Error for the example from hugging face
I am getting the following error whenever I attempt to run this model with the given prompt.
/opt/conda/conda-bld/pytorch_1728945377988/work/aten/src/ATen/native/cuda/TensorCompare.cu:110: _assert_async_cuda_kernel: block: [0,0,0], thread: [0,0,0] Assertion probability tensor contains either inf, nan or element < 0 failed.
Traceback (most recent call last):
File "/u1/a8tariq/LoginCrawler/OS-ATLAS/test.py", line 116, in
generated_ids = model.generate(**inputs, max_new_tokens=128)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/u1/a8tariq/.conda/envs/new-env/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/u1/a8tariq/.conda/envs/new-env/lib/python3.12/site-packages/transformers/generation/utils.py", line 2215, in generate
result = self._sample(
^^^^^^^^^^^^^
File "/u1/a8tariq/.conda/envs/new-env/lib/python3.12/site-packages/transformers/generation/utils.py", line 3195, in _sample
while self._has_unfinished_sequences(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/u1/a8tariq/.conda/envs/new-env/lib/python3.12/site-packages/transformers/generation/utils.py", line 2413, in _has_unfinished_sequences
elif this_peer_finished:
^^^^^^^^^^^^^^^^^^
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.