transformers.js
c5ed1d70 - Add support for CLAP (`zero-shot-audio-classification`) and Audio Spectrogram Transformer (`audio-classification`) (#427)

Commit
2 years ago
Add support for CLAP (`zero-shot-audio-classification`) and Audio Spectrogram Transformer (`audio-classification`) (#427) * Add FFT unit tests * Refactor maths.js and audio.js * Refactor audio processors * Add support for AST models * Add another audio-classification example * Add audio processing unit tests * Implement `log_mel='dB'` in `spectrogram` function * Add `ClapFeatureExtractor` * Implement `ClapFeatureExtractor` unit tests * Add support for `CLAP` * Add `ZeroShotAudioClassificationPipeline` * Add listed support for `zero-shot-audio-classification` pipeline tag * Cleanup * `let` -> `const` * Update `mel_filter_bank` unit test * Add `'Xenova/tiny-random-ClapModel'` * Add `ClapAudioModelWithProjection` and `ClapTextModelWithProjection` * Move audio validation to helper function * Optimize `mel_filter_bank` computation -30ms * Update mel filters unit test * Cleanup * Optimizations * Fix jsdoc * Optimizations * Add WIP conversion scripts Will be updated once https://github.com/huggingface/optimum/pull/1552 is merged
Author
Parents
Loading