Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

训练模型时出错 #315

Open
xingxindrst opened this issue Apr 3, 2023 · 2 comments
Open

训练模型时出错 #315

xingxindrst opened this issue Apr 3, 2023 · 2 comments

Comments

@xingxindrst
Copy link

我在训练以前的模型时遇到了
2023-04-03 17:53:46.366387: W tensorflow/core/framework/op_kernel.cc:1830] OP_REQUIRES failed at ctc_loss_op.cc:215 : INVALID_ARGUMENT: Saw a non-null label (index >= num_classes - 1) following a null label, batch: 1 num_classes: 1425 labels: 353,1111,657,1059,1135,842,911,1424,1059,187,540 labels seen so far: 353,1111,657,1059,1135,842,911
Traceback (most recent call last):
File "D:\biyeshixi\f14f1116-77dd-11ed-b094-52540066b139\CASR-DEMO-master\speech_env\main\SpeechModel251.py", line 429, in
ms.TrainModel(datapath, epoch = 50, batch_size = 2, save_step = 500)
File "D:\biyeshixi\f14f1116-77dd-11ed-b094-52540066b139\CASR-DEMO-master\speech_env\main\SpeechModel251.py", line 175, in TrainModel
self._model.fit_generator(yielddatas, save_step)
File "E:\python\lib\site-packages\keras\engine\training.py", line 2604, in fit_generator
return self.fit(
File "E:\python\lib\site-packages\keras\utils\traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
File "E:\python\lib\site-packages\tensorflow\python\eager\execute.py", line 52, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:

Detected at node 'model_1/ctc/CTCLoss' defined at (most recent call last):
File "D:\biyeshixi\f14f1116-77dd-11ed-b094-52540066b139\CASR-DEMO-master\speech_env\main\SpeechModel251.py", line 429, in
ms.TrainModel(datapath, epoch = 50, batch_size = 2, save_step = 500)
File "D:\biyeshixi\f14f1116-77dd-11ed-b094-52540066b139\CASR-DEMO-master\speech_env\main\SpeechModel251.py", line 175, in TrainModel
self._model.fit_generator(yielddatas, save_step)
File "E:\python\lib\site-packages\keras\engine\training.py", line 2604, in fit_generator
return self.fit(
File "E:\python\lib\site-packages\keras\utils\traceback_utils.py", line 65, in error_handler
return fn(*args, **kwargs)
File "E:\python\lib\site-packages\keras\engine\training.py", line 1650, in fit
tmp_logs = self.train_function(iterator)
File "E:\python\lib\site-packages\keras\engine\training.py", line 1249, in train_function
return step_function(self, iterator)
File "E:\python\lib\site-packages\keras\engine\training.py", line 1233, in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "E:\python\lib\site-packages\keras\engine\training.py", line 1222, in run_step
outputs = model.train_step(data)
File "E:\python\lib\site-packages\keras\engine\training.py", line 1023, in train_step
y_pred = self(x, training=True)
File "E:\python\lib\site-packages\keras\utils\traceback_utils.py", line 65, in error_handler
return fn(*args, **kwargs)
File "E:\python\lib\site-packages\keras\engine\training.py", line 561, in call
return super().call(*args, **kwargs)
File "E:\python\lib\site-packages\keras\utils\traceback_utils.py", line 65, in error_handler
return fn(*args, **kwargs)
File "E:\python\lib\site-packages\keras\engine\base_layer.py", line 1132, in call
outputs = call_fn(inputs, *args, **kwargs)
File "E:\python\lib\site-packages\keras\utils\traceback_utils.py", line 96, in error_handler
return fn(*args, **kwargs)
File "E:\python\lib\site-packages\keras\engine\functional.py", line 511, in call
return self._run_internal_graph(inputs, training=training, mask=mask)
File "E:\python\lib\site-packages\keras\engine\functional.py", line 668, in _run_internal_graph
outputs = node.layer(*args, **kwargs)
File "E:\python\lib\site-packages\keras\utils\traceback_utils.py", line 65, in error_handler
return fn(*args, **kwargs)
File "E:\python\lib\site-packages\keras\engine\base_layer.py", line 1132, in call
outputs = call_fn(inputs, *args, **kwargs)
File "E:\python\lib\site-packages\keras\utils\traceback_utils.py", line 96, in error_handler
return fn(*args, **kwargs)
File "E:\python\lib\site-packages\keras\layers\core\lambda_layer.py", line 209, in call
result = self.function(inputs, **kwargs)
File "D:\biyeshixi\f14f1116-77dd-11ed-b094-52540066b139\CASR-DEMO-master\speech_env\main\SpeechModel251.py", line 148, in ctc_lambda_func
return K.ctc_batch_cost(labels, y_pred, input_length, label_length)
File "E:\python\lib\site-packages\keras\backend.py", line 7023, in ctc_batch_cost
tf.compat.v1.nn.ctc_loss(
Node: 'model_1/ctc/CTCLoss'
Saw a non-null label (index >= num_classes - 1) following a null label, batch: 1 num_classes: 1425 labels: 353,1111,657,1059,1135,842,911,1424,1059,187,540 labels seen so far: 353,1111,657,1059,1135,842,911
[[{{node model_1/ctc/CTCLoss}}]] [Op:__inference_train_function_2809]
2023-04-03 17:53:51.148622: W tensorflow/core/kernels/data/generator_dataset_op.cc:108] Error occurred when finalizing GeneratorDataset iterator: FAILED_PRECONDITION: Python interpreter state is not initialized. The process may be terminated.
[[{{node PyFunc}}]]

Process finished with exit code 1
麻烦大家可以帮我看看吗,谢谢

@nl8590687
Copy link
Owner

你是不是改了dict.txt里面的列表数目,这种问题一般都是改代码和配置文件改出来的

@xingxindrst
Copy link
Author

你是不是改了dict.txt里面的列表数目,这种问题一般都是改代码和配置文件改出来的
没有改过dict.txt 代码也只改了报错的地方

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants