Skip to content

Recipe CommonVoice/ASR/transformer fails with error 'WhisperTokenizerFast' object has no attribute 'normalize' #2884

Description

@Craya

Describe the bug

Since speechbrain==1.0.3, I face an error when launching recipe CommonVoice/ASR/transformer/:
AttributeError: 'WhisperTokenizerFast' object has no attribute 'normalize'

Expected behaviour

Recipe works fine ;-)

To Reproduce

Host-0:~/speechbrain/recipes/CommonVoice/ASR/transformer$ torchrun train_with_whisper.py hparams/train_hf_whisper.yaml --language=en --skip_prep=True --save_folder=/tmp/speechbrain/ --whisper_hub=openai/whisper-large-v2 --data_folder=/tmp/datasets --train_csv=/tmp/datasets/dataset_train.csv --valid_csv=/tmp/datasets/dataset_val.csv --test_csv=/tmp/datasets/dataset_test.csv --output_folder=/tmp/outputs/

Environment Details

speechbrain==1.0.3

Relevant Log Output

/usr/local/lib/python3.8/dist-packages/huggingface_hub/file_download.py:797: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
speechbrain.lobes.models.huggingface_transformers.whisper - whisper encoder is frozen.
speechbrain.utils.quirks - Applied quirks (see `speechbrain.utils.quirks`): [allow_tf32, disable_jit_profiling]
speechbrain.utils.quirks - Excluded quirks specified by the `SB_DISABLE_QUIRKS` environment (comma-separated list): []
speechbrain.core - Beginning experiment!
speechbrain.core - Experiment folder: /tmp/outputs
speechbrain.core - Info: precision arg from hparam file is used
speechbrain.core - Info: eval_precision arg from hparam file is used
speechbrain.core - Info: max_grad_norm arg from hparam file is used
speechbrain.core - Info: ckpt_interval_minutes arg from hparam file is used
speechbrain.core - Info: grad_accumulation_factor arg from hparam file is used
speechbrain.core - Gradscaler enabled: `True`
speechbrain.core - Using training precision: `--precision=fp16`
speechbrain.core - Using evaluation precision: `--eval_precision=fp16`
speechbrain.core - Exception:
Traceback (most recent call last):
  File "train_with_whisper.py", line 296, in <module>
    asr_brain = ASR(
  File "/usr/local/lib/python3.8/dist-packages/speechbrain/core.py", line 770, in __init__
    self.training_ctx = TorchAutocast(
  File "/usr/local/lib/python3.8/dist-packages/speechbrain/utils/autocast.py", line 79, in __init__
    self.context = torch.autocast(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/amp/autocast_mode.py", line 241, in __init__
    raise RuntimeError(
RuntimeError: User specified an unsupported autocast device_type 'cuda:0'
[2025-04-11 10:10:58,397] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 15988) of binary: /usr/bin/python3
Traceback (most recent call last):
  File "/usr/local/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 347, in wrapper
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/run.py", line 812, in main
    run(args)
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/run.py", line 803, in run
    elastic_launch(
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/launcher/api.py", line 135, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/usr/local/lib/python3.8/dist-packages/torch/distributed/launcher/api.py", line 268, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
train_with_whisper.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2025-04-11_10:10:58
  host      : Host-0
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 15988)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================

Additional Context

It worked fine with speechbrain <=1.0.2

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions