WebSafe 3.7github.com
|
|
🏠
Skip to content

fix issue with ctc_forward_step#2505

Merged
mravanelli merged 5 commits intospeechbrain:developfrom
Adel-Moumen:fix_ctc_forward_step
Apr 13, 2024
Merged

fix issue with ctc_forward_step#2505
mravanelli merged 5 commits intospeechbrain:developfrom
Adel-Moumen:fix_ctc_forward_step

Conversation

@Adel-Moumen
Copy link
Collaborator

@Adel-Moumen Adel-Moumen commented Apr 11, 2024 •
edited
Loading

What does this PR do?

This PR aims to fix one issue with the ctc_segmentation module due to the introduction of the new beam search part. Indeed, there's no longer the function ctc_forward_step inside the beam search. To address this issue, I added new code to make it work with SB 1.0.

Fixes #2440

repro:

from speechbrain.inference.ASR import EncoderDecoderASR
from speechbrain.alignment.ctc_segmentation import CTCSegmentation
# load an ASR model
pre_trained = "speechbrain/asr-transformer-transformerlm-librispeech"
asr_model = EncoderDecoderASR.from_hparams(source=pre_trained)
aligner = CTCSegmentation(asr_model, kaldi_style_text=False)
# load data
audio_path = "/content/speechbrain/tests/samples/single-mic/example1.wav"
text = ["THE BIRCH CANOE", "SLID ON THE", "SMOOTH PLANKS"]
segments = aligner(audio_path, text, name="example1")

Before

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[<ipython-input-4-cb3639878d03>](https://localhost:8080/#) in <cell line: 6>()
      4 pre_trained = "speechbrain/asr-transformer-transformerlm-librispeech"
      5 asr_model = EncoderDecoderASR.from_hparams(source=pre_trained)
----> 6 aligner = CTCSegmentation(asr_model, kaldi_style_text=False)
      7 # load data
      8 audio_path = "/content/speechbrain/tests/samples/single-mic/example1.wav"

1 frames
[/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py](https://localhost:8080/#) in __getattr__(self, name)
   1693             if name in modules:
   1694                 return modules[name]
-> 1695         raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
   1696 
   1697     def __setattr__(self, name: str, value: Union[Tensor, 'Module']) -> None:

AttributeError: 'S2STransformerBeamSearcher' object has no attribute 'ctc_forward_step'

After

The dictionary has 5000 tokens with a max length of 15. This may lead to low alignment performance and low accuracy.
example1_0000 example1 0.78 1.61 -5.7213 THE BIRCH CANOE
example1_0001 example1 1.61 1.85 -11.5366 SLID ON THE
example1_0002 example1 1.85 2.81 -3.2060 SMOOTH PLANKS
Before submitting
  • Did you read the contributor guideline?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Does your code adhere to project-specific code style and conventions?

PR review

Reviewer checklist
  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified
  • Confirm that the changes adhere to compatibility requirements (e.g., Python version, platform)
  • Review the self-review checklist to ensure the code is ready for review
@Adel-Moumen Adel-Moumen marked this pull request as draft April 11, 2024 19:59
@Adel-Moumen Adel-Moumen marked this pull request as ready for review April 11, 2024 20:32
@mravanelli mravanelli self-requested a review April 12, 2024 01:32
@mravanelli mravanelli added the bug Something isn't working label Apr 12, 2024
@mravanelli
Copy link
Collaborator

mravanelli commented Apr 13, 2024

Thank you @Adel-Moumen, I tested the latest version and everything seems fixed!
@mravanelli mravanelli merged commit d187cba into speechbrain:develop Apr 13, 2024
@Mickaelh51
Copy link

Mickaelh51 commented Apr 13, 2024

Hi,
Thanks a lot for this PR.
It works like a charm
++
pchampio pushed a commit to pchampio/speechbrain that referenced this pull request Apr 17, 2024
@Adel-Moumen @pchampio
fix issue with ctc_forward_step (speechbrain#2505)
4554c1e 
* fix issue with ctc_forward_step

* improve documentation

* add check

* add checks

* fix checks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

3 participants

@Adel-Moumen @mravanelli @Mickaelh51

Comments