1 系统环境
硬件环境(Ascend/GPU/CPU): Ascend310
软件环境: MindSpore= 2.4.1、Mindnlp=0.4.0
执行模式： PyNative
Python 版本: 3.9.2
操作系统平台: linux
2 报错信息
2.1 问题描述
执行下方链接中的qwen1.5-0.5b.py，会出现如下报错。
https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/14-qwen1.5-0.5b
2.2 报错信息
Prefix dict has been built successfully.
Qwen2ForCausalLM has generative capabilities, as prepare_inputs_for_generation is explicitly overwritten. However, it doesn't directly inherit from GenerationMixin.PreTrainedModel will NOT inherit from GenerationMixin, and this model will lose the ability to call generate and other related functions.
- If you are the owner of the model architecture code, please modify your model class such that it inherits from GenerationMixin (after PreTrainedModel, otherwise you'll get an exception).
- If you are not the owner of the model architecture class, please contact the model code owner to update it.
Sliding Window Attention is enabled but not implemented for eager; unexpected results may be encountered.
[WARNING] DEVICE(4480,e7fff25f2020,python):2024-12-30-14:44:43.528.024 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:116] Initialize] Free memory size is less than half of total memory size.Device 0 Device HBM total size:16367894528 Device HBM free size:7189217280 may be other processes occupying this card, check as: ps -ef|grep python
The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results.
Traceback (most recent call last):
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/queueing.py", line 536, in process_events
response = await route_utils.call_process_api(
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/blocks.py", line 1935, in process_api
result = await self.call_function(
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/blocks.py", line 1532, in call_function
prediction = await utils.async_iteration(iterator)
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/utils.py", line 671, in async_iteration
return await iterator.anext()
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/utils.py", line 776, in asyncgen_wrapper
response = await iterator.anext()
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/chat_interface.py", line 653, in _stream_fn
first_response = await async_iteration(generator)
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/utils.py", line 671, in async_iteration
return await iterator.anext()
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/utils.py", line 664, in anext
return await anyio.to_thread.run_sync(
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 2441, in run_sync_in_worker_thread
return await future
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 943, in run
result = context.run(func, *args)
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/utils.py", line 647, in run_sync_iterator_async
return next(iterator)
File "/home/HwHiAiUser/Desktop/qwen/qwen1.5-0.5b.py", line 48, in predict
for new_token in streamer:
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/mindnlp/transformers/generation/streamers.py", line 282, in next
value = self.text_queue.get(timeout=self.timeout)
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/queue.py", line 179, in get
raise Empty
_queue.Empty
Traceback (most recent call last):
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/queueing.py", line 536, in process_events
response = await route_utils.call_process_api(
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/blocks.py", line 1935, in process_api
result = await self.call_function(
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/blocks.py", line 1532, in call_function
prediction = await utils.async_iteration(iterator)
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/utils.py", line 671, in async_iteration
return await iterator.anext()
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/utils.py", line 776, in asyncgen_wrapper
response = await iterator.anext()
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/chat_interface.py", line 653, in _stream_fn
first_response = await async_iteration(generator)
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/utils.py", line 671, in async_iteration
return await iterator.anext()
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/utils.py", line 664, in anext
return await anyio.to_thread.run_sync(
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 2441, in run_sync_in_worker_thread
return await future
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/anyio/_backends/asyncio.py", line 943, in run
result = context.run(func, *args)
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/gradio/utils.py", line 647, in run_sync_iterator_async
return next(iterator)
File "/home/HwHiAiUser/Desktop/qwen/qwen1.5-0.5b.py", line 29, in predict
input_ids = tokenizer.apply_chat_template(
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/mindnlp/transformers/tokenization_utils_base.py", line 1768, in apply_chat_template
rendered_chat = compiled_template.render(
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/jinja2/environment.py", line 1304, in render
self.environment.handle_exception()
File "/home/HwHiAiUser/miniconda3/envs/myconda/lib/python3.9/site-packages/jinja2/environment.py", line 939, in handle_exception
raise rewrite_traceback_stack(source=source)
File "
3 根因分析
因为这个单开了个thread，这个timeout时间设置的比较短（timeout=120），还没生成就timeout。
4 解决方案
在qwen1.5-0.5b.py中设置的timeout=120，还没生成就timeout，可以设置为300。
streamer = TextIteratorStreamer(tokenizer, timeout=300, skip_prompt=True, skip_special_tokens=True)