The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Hey! This is great. Is this already in alpha?
Team, is there any tentative time to release this v3 alpha ???
I can't wait anymore :) Please update me when it will be released!
@xenova Can I test v3-alpha by using NPM? When I try to run, I get this issue.
@xenova Can I test v3-alpha by using NPM? When I try to run, I get this issue.
use this https://github.com/kishorekaruppusamy/transformers.js/commit/7af8ef1e5c37f3052ed3a8e38938595702836f09 commit to resolve this issue ...
Thanks for your reply @kishorekaruppusamy I tried with your branch and I got other issues.
Please give me your advise!
Thanks for your reply @kishorekaruppusamy I tried with your branch and I got other issues.
Please give me your advise!
https://github.com/kishorekaruppusamy/transformers.js/blob/V3_BRANCH_WEBGPU_BUG_FIX/src/backends/onnx.js#L144
change this url to local dist dir inside build ..
Thanks @kishorekaruppusamy
I downloaded the latest wasm from onnxruntime and added in local directory but I got same issue
I realized transformer.js v3 uses onnxruntime 1.16.3 so I created wasm by using onnxruntime 1.16.3 and tested and I got same issue
please give your advise. Thanks
Hi everyone! Today we released our first WebGPU x Transformers.js demo: The WebGPU Embedding Benchmark (online demo). If you'd like to help with testing, please run the benchmark and share your results! Thanks!
@xenova can this bench pick the GPU 1 instead of 0? For the laptops with dGPU
@xenova can this bench pick the GPU 1 instead of 0? For the laptops with dGPU
Not currently, but this is being worked on here: microsoft/onnxruntime#19857. We will add support here once ready.
@beaufortfrancois - I've added the source code for the video background removal demo. On my device, I get ~20fps w/ WebGPU support (w/ fp32 since fp16 is broken). Here's a screen recording (which drops my fps to ~14):
webgpu-modnet.mp4
- Model used: https://huggingface.co/Xenova/modnet (~4 years old, and it clearly struggles on hands moving quickly). I will try on more up-to-date models soon.
- Video tested: https://www.youtube.com/watch?v=NXpdyAWLDas
- Online demo: https://huggingface.co/spaces/Xenova/webgpu-video-background-removal
@beaufortfrancois - I've added the source code for the video background removal demo. On my device, I get ~20fps w/ WebGPU support (w/ fp32 since fp16 is broken). Here's a screen recording (which drops my fps to ~14):
You rock. Thanks! It's a cool demo! ๐
I've been wondering how we could improve it:
- I've noticed you read the current frame of the video on the main thread. Would it help to move the entire demo to a web worker?
output[0].mul(255).to('uint8')takes some non negligible time to run. Is there a faster path?- How much you expect fp16 to improve perf? In https://developer.chrome.com/blog/new-in-webgpu-120#support_for_16-bit_floating-point_values_in_wgsl, we've noticed on an Apple M1 Pro device that the f16 implementation of Llama2 7B models used in the WebLLM chat demo is significantly faster than the f32 implementation, with a 28% improvement in prefill speed and a 41% improvement in decoding speed.
- A way to feed a
GPUExternalTextureto the model as an input could also come handy.
| 1 | /** | ||
| 2 | * @typedef {'cpu'|'gpu'|'wasm'|'webgpu'|null} DeviceType |
beaufortfrancois1 year agoOut of curiosity, what is 'gpu'?
xenova1 year ago๐ 3It's meant to be a "catch-all" for the different ways that the library can be used with GPU support (not just in the browser with WebGPU). The idea is that it will simplify documentation, as transformers.js will select the best execution provider depending on the environment. For example, DML/CUDA support in onnxruntime-node (see microsoft/onnxruntime#16050 (comment))
Of course, this is still a work in progress, so it can definitely change!
device: 'webgpu',
For some environement it better be list.
Because there not all execution proveders support all oprtators.
For my use-case I'm give a list of EP orderd by priority, let onnxruntime auto fallback.
For example: ['nnapi', 'xnnpack', 'cpu'] for Android / ['qnn', 'dml', 'xnnpack', 'cpu'] for Windows ARM64 (custom build)
UPDATE: Looks like some kernels are not supported for quant operations :/
I tested WebGPU version on https://huggingface.co/Xenova/wav2vec2-bert-CV16-en with changes from v3 and model (quantized) is loaded without errors but after running transcribe it throws error with message:
An error occurred during model execution: "Error: [WebGPU] Kernel "[Split] /wav2vec2_bert/encoder/layers.0/conv_module/glu/Split" failed. Error: no GPU data for output: 0".
[E:onnxruntime:, sequential_executor.cc:514 ExecuteKernel] Non-zero status code returned while running Split node. Name:'/wav2vec2_bert/encoder/layers.0/conv_module/glu/Split' Status Message: Failed to run JSEP kernel
Is it some quantization error or onnxruntime error?
Logs localhost-1710758687772.log
Env: Windows, Chrome 122, Nvidia Geforce 3090
@young-developer Thanks for the report. I will cc @guschmue for this unsupported operator. It may already be fixed in the dev branch of onnxruntime-web.
@hans00 For more advanced use-cases, you can update the session options directly with session_options: {...} in the model options.
FYI @xenova I was able to load the model in fp32 and got the same error. I also tried to load in fp16 but it throws input error is (float) instead of (float16) so I assume inputs should be converted to fp16 too.
Exciting news ๐ฅณ We've got Musicgen working! Example usage:
import { AutoTokenizer, MusicgenForConditionalGeneration } from '@xenova/transformers';
// Load tokenizer and model
const tokenizer = await AutoTokenizer.from_pretrained('Xenova/musicgen-small');
const model = await MusicgenForConditionalGeneration.from_pretrained(
'Xenova/musicgen-small', { dtype: 'fp32' }
);
// Prepare text input
const prompt = '80s pop track with bassy drums and synth';
const inputs = tokenizer(prompt);
// Generate audio
const audio_values = await model.generate({
...inputs,
max_new_tokens: 512,
do_sample: true,
guidance_scale: 3,
});
// (Optional) Write the output to a WAV file
import wavefile from 'wavefile';
import fs from 'fs';
const wav = new wavefile.WaveFile();
wav.fromScratch(1, model.config.audio_encoder.sampling_rate, '32f', audio_values.data);
fs.writeFileSync('musicgen_out.wav', wav.toBuffer());Samples:
sample_1.mp4
sample_2.mp4
sample_3.mp4
Would it be helpful if I created an example for MusicGen? (based on your example code, but as a small stand-along html page)
@xenova There is new version 1.17.3 of onnxruntime-web. I tested with wav2vec and there is new error so looks like a progress ๐
Segment Anything Encoder now works with WebGPU: up to 8x faster! (online demo)
sam-webgpu.mp4
Phi-3 WebGPU support is now working! Demo: https://huggingface.co/spaces/Xenova/experimental-phi3-webgpu
phi3-webgpu-demo.mp4
Does anyone have a guide for how to get this bundled into a script, akin to a JSDelivr URL? Here's what I tried:
// index.js
export * from 'transformers.js'; // Adjust if the import path differsnpm install xenova/transformers.js#v3
npm install rollup @rollup/plugin-node-resolve rollup-plugin-terser --save-dev
// rollup.config.js
import resolve from '@rollup/plugin-node-resolve';
import { terser } from 'rollup-plugin-terser';
export default {
input: 'index.js',
output: {
file: 'bundle.js',
format: 'esm',
sourcemap: true
},
plugins: [
resolve({
browser: true,
}),
terser()
]
};And in package.json:
"scripts": {
"build": "rollup -c"
}
And then:
npm run build
And that produced a bundle.js, but it was looking for webgpu.proxy.min.js on jsdelivr, which doesn't exist where it was looking. I tried manually adjusting to URL in the bundle to point to the ort.webgpu.min.js file, but no luck (I also tried esm/ort.webgpu.min.js). I'm guessing there are some tricky things due to the dynamic nature of backend loading that bundlers struggle to automatically pick up.
@xenova Alternatively, I wonder if you'd be able to do some v3 alpha/prealpha releases via github tags so that jsdelivr picks them up? Since there's no way (IIUC) to simply reference a branch via jsdelivr (due to immutability requirement I assume).
The latest commits add support for Moondream2, a small vision language model by @vikhyat designed to run efficiently on edge devices.
Try it out yourself with the live demo: https://huggingface.co/spaces/Xenova/experimental-moondream-webgpu
moondream-webgpu-2.mp4
Usage:
import { AutoProcessor, AutoTokenizer, Moondream1ForConditionalGeneration, RawImage } from '@xenova/transformers';
// Load processor, tokenizer and model
const model_id = 'Xenova/moondream2';
const processor = await AutoProcessor.from_pretrained(model_id);
const tokenizer = await AutoTokenizer.from_pretrained(model_id);
const model = await Moondream1ForConditionalGeneration.from_pretrained(model_id, {
dtype: {
embed_tokens: 'fp16', // or 'fp32'
vision_encoder: 'fp16', // or 'q8'
decoder_model_merged: 'q4', // or 'q4f16' or 'q8'
},
device: 'webgpu',
});
// Prepare text inputs
const prompt = 'Describe this image.';
const text = `<image>\n\nQuestion: ${prompt}\n\nAnswer:`;
const text_inputs = tokenizer(text);
// Prepare vision inputs
const url = 'https://huggingface.co/vikhyatk/moondream1/resolve/main/assets/demo-1.jpg';
const image = await RawImage.fromURL(url);
const vision_inputs = await processor(image);
// Generate response
const output = await model.generate({
...text_inputs,
...vision_inputs,
do_sample: false,
max_new_tokens: 64,
});
const decoded = tokenizer.batch_decode(output, { skip_special_tokens: false });
console.log(decoded);
// [
// '<|endoftext|><image>\n\n' +
// 'Question: Describe this image.\n\n' +
// 'Answer: A hand is holding a white book titled "The Little Book of Deep Learning" against a backdrop of a balcony with a railing and a view of a building and trees.<|endoftext|>'
// ]
VLMs now support PKV caching. Demo: https://huggingface.co/spaces/Xenova/experimental-nanollava-webgpu
nanollava-webgpu.mp4
Example code
import { AutoProcessor, AutoTokenizer, LlavaForConditionalGeneration, RawImage } from '@xenova/transformers';
// Load tokenizer, processor and model
const model_id = 'Xenova/nanoLLaVA';
const tokenizer = await AutoTokenizer.from_pretrained(model_id);
const processor = await AutoProcessor.from_pretrained(model_id);
const model = await LlavaForConditionalGeneration.from_pretrained(model_id, {
dtype: {
embed_tokens: 'fp16', // or 'fp32' or 'q8'
vision_encoder: 'fp16', // or 'fp32' or 'q8'
decoder_model_merged: 'q4', // or 'q8'
},
// device: 'webgpu',
});
// Prepare text inputs
const prompt = 'What does the text say?';
const messages = [
{ role: 'system', content: 'Answer the question.' },
{ role: 'user', content: `<image>\n${prompt}` }
]
const text = tokenizer.apply_chat_template(messages, { tokenize: false, add_generation_prompt: true });
const text_inputs = tokenizer(text);
// Prepare vision inputs
const url = 'https://huggingface.co/qnguyen3/nanoLLaVA/resolve/main/example_1.png';
const image = await RawImage.fromURL(url);
const vision_inputs = await processor(image);
// Generate response
const { past_key_values, sequences } = await model.generate({
...text_inputs,
...vision_inputs,
do_sample: false,
max_new_tokens: 64,
return_dict_in_generate: true,
});
// Decode output
const answer = tokenizer.decode(
sequences.slice(0, [text_inputs.input_ids.dims[1], null]),
{ skip_special_tokens: true },
);
console.log(answer);
// The text reads "Small but mighty".
const new_messages = [
...messages,
{ role: 'assistant', content: answer },
{ role: 'user', content: 'How does the text correlate to the context of the image?' }
]
const new_text = tokenizer.apply_chat_template(new_messages, { tokenize: false, add_generation_prompt: true });
const new_text_inputs = tokenizer(new_text);
// Generate another response
const output = await model.generate({
...new_text_inputs,
past_key_values,
do_sample: false,
max_new_tokens: 256,
});
const new_answer = tokenizer.decode(
output.slice(0, [new_text_inputs.input_ids.dims[1], null]),
{ skip_special_tokens: true },
);
console.log(new_answer);
// The context of the image is that of a playful and humorous illustration of a mouse holding a weightlifting bar. The text "Small but mighty" is a playful reference to the mouse's size and strength.
@xenova For some models, the performance may be a blocker. Since model downloads can be quite large, I wonder if there should be a way for web developers to know their machine performance class for running a model without downloading it completely first.
I believe this would involve running the model code with zeroed-out weights, which would still require buffer allocations but would allow the web app to catch out-of-memory errors or such. The model architecture would still needed to generate shaders, but this be much smaller than model weights.
Essentially, knowing the model architecture and testing with empty weights would allow for assessing performance capability without downloading the full model.
I thought I could use from_config for that but I wonder now if this should be a built-in V3 feature. What are your thoughts?
@beaufortfrancois That would be amazing to have! Although, it's probably best suited as a feature request for onnxruntime-web. The way one could do it is to use the external data format to save models into two parts: graph-only (<1MB usually) and weights, and then initialize an empty session from the graph without loading the weights. @guschmue might have additional insights.
Thank you @xenova for your support โค๏ธ
@guschmue What are your thoughts on #545 (comment)?
I'm happy to file a feature request in https://github.com/microsoft/onnxruntime
@beaufortfrancois, yes some utility class that helps applications to decide what hardware capabilities are there before a model is loaded is on my wish list since some time.
We have not gotten to it but should find time soon I hope.
It would need to tell how mighty your gpu is, if there is a npu (in the future if there is webnn), if it is visible to run the model on wasm.
Not trivial to get this right on first try so I'd expect a few iterations on it.
It also would need a lot of feedback and help from application developers.
Filing a feature request would be good, then we have a place to track it.
@guschmue I've filed microsoft/onnxruntime#20998 to track this feature request. How would we be able to help out there?
we'd need to come up with a nice api. the info one can get from webgpu is very sparse and imo not good enough to make this work.
The way I see this work goes like this:
we define a couple of model classes, ie:
llm, vision, speech
based on this we'd run some shaders briefly to determine relevant flops that are used in the selected class.
the result would be some raw flops number, or some class based on some heuristics a class like 'good enough for 500M parameters' and some hints from the webgpu info ...
Application could cache this so the detection needs to run only the first time.
Maybe there would be an offline tool that you can run your model through to capture some data what the model needs.
We would need help defining this (ie what classes) and then get a lot of feedback to tune this to practial values.
But this is just what I think how this would work, very open to other suggestions.
@guschmue Sounds great, +1 on the need to integrate feedback from application developers. One early comment/question on the potential API, wrt your "good enough for 500M parameters" example: are you referring to "fast enough"? If so, it may be convenient for application developers to not only get as an output a bucketized speed estimate (e.g. for a given 500MB params model, x-slow / slow / medium/ fast), but also to be able access the raw timings. How long end-users are willing to wait for an output may be use-case specific.
Experimental Florence2 support has been added! ๐ฅณ (closes #815)
Example code:
import {
Florence2ForConditionalGeneration,
AutoProcessor,
AutoTokenizer,
RawImage,
} from '@xenova/transformers';
// Load model, processor, and tokenizer
const model_id = 'onnx-community/Florence-2-base-ft';
const model = await Florence2ForConditionalGeneration.from_pretrained(model_id, {
dtype: 'fp32',
});
const processor = await AutoProcessor.from_pretrained(model_id);
const tokenizer = await AutoTokenizer.from_pretrained(model_id);
// Load image
const url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/car.jpg?download=true";
const image = await RawImage.fromURL(url);
// Process inputs
const prompts = "Describe with a paragraph what is shown in the image.";
const text_inputs = tokenizer(prompts);
const vision_inputs = await processor(image);
// Generate text
const generated_ids = await model.generate({
...text_inputs,
...vision_inputs,
max_new_tokens: 100,
});
// Decode generated text
const generated_text = tokenizer.batch_decode(generated_ids, { skip_special_tokens: true });
console.log(generated_text);
generates
'A green car is parked in front of a tan building. There is a brown door on the building behind the car. There are two windows on the front of the building. '
I'm still working on adding support for other tasks and improving processing methods, but this is a good start. Another issue is that the vision encoder doesn't work on WebGPU (but other submodules do). cc @guschmue for this.
Early dereferencing for performance boosts
cleanup
Move quantization logic to `quantize.py`
update deps
Fix q4 quantization
save q4 quantization
Add decode ASR test
Do not process last chunk unnecessarily
fp16 disable_shape_infer if model is too large
Use `check_and_save_model` for saving fp16 model
Reorder functions
formatting
Remove debug log
Fix q8 quantization for models > 2GB
correct attribute
Fix `TextGenerationPipeline`
Fix pauses in whisper word-level timestamps
Formatting
Sort added tokens by length to avoid early partial matches
Add new tokenizer test
Only finish with newline if running in Node.js
Consider token timestamps when selecting longest common sequence
Create whisper word-level timestamps demo
cleanup
Fallback to WASM if WebGPU not supported
Reload model for each quantization mode
Update converstion script requirements
Separate IO and Quantization args
Use `const` where possible
Add `InterruptableStoppingCriteria`
`@xenova/transformers` -> `@huggingface/transformers`
Override semver version
Add support for pyannote models
Update README.md
Add listed support for pyannote
Add pyannote example code
Support specifying `min_num_frames`
Support simultaneous instantiation of multiple inference sessions
Support broadcasting encoder outputs over decoder inputs
Fix test
fix bundler config for latest ORT
Only check fp16 support for webgpu device
Remove default chat templates
Add support for gemma2
Add gemma2 generation test
Update gemma2 config mapping
Prioritize high-performance adapter when possible
Set defaults for `tools` and `documents` in `apply_chat_template`
bump `@huggingface/jinja` -> 0.3.0
Add `apply_chat_template` default parameters unit test
Merge branch 'v3' into @huggingface/transformers
Add prettier
prettier format config files
remove incorrect comment
Merge branch 'pr/864' into @huggingface/transformers
Update onnxruntime-web version
Update webpack.config.js
Fix copy path
Run `npm ci`
Fix bundling
Do not set `preferredOutputLocation` if we are proxying
Merge branch 'v3' into @huggingface/transformers
Update `@webgpu/types`
Update SAM example
Use `??=` operator where possible
Fix commonjs usage
Mark `onnxruntime-node` and `sharp` as externals
Move `externals` into config
Downgrade to onnxruntime 1.18.0
Finalize module/commonjs build
Separate web and node builds
[version] Update to 3.0.0-alpha.1
Default to CDN-hosted .wasm files
[version] Update to 3.0.0-alpha.2
bump versions
[version] Update to 3.0.0-alpha.3
Merge branch 'improve-conversion-script' into v3
Consolidate conversion and quantization script
Downgrade `onnxconverter-common`
Link to types in exports
Update list of supported tasks
Fixed unit tests
Update imports
Bump versions to `3.0.0-alpha.4`
[version] Update to 3.0.0-alpha.4
Fix "Default condition should be last one"
Bump versions
[version] Update to 3.0.0-alpha.5
Update next.js client-side demo
Initial WebNN Support
Mark fs, path and url as external packages for node build
Move content type map outside of `FileResponse` object
Add GPU support for Node.js
Bump versions
[version] Update to 3.0.0-alpha.6
Fix conflicts
bump dependency versions
Add support for device auto-detection
Fix default device selection
Merge branch 'pr/ibelem/890-1' into v3
Improve WebNN selection
Skip token callback if `skip_prompt` is set
Bump versions
[version] Update to 3.0.0-alpha.7
bump versions
[version] Update to 3.0.0-alpha.8
bump versions
[version] Update to 3.0.0-alpha.9
Add support for Sapiens
Update default ONNX env
Fix types
Topologically sort fp16 nodes
Add marian unit test
Re-order imports
Fix `NoBadWordsLogitsProcessor`
Update package.json
[jest] Disable coverage
Bump versions
[version] Update to 3.0.0-alpha.10
Improve node/web interoperability
Fix scripts/requirements.txt
Bump versions
[version] Update to 3.0.0-alpha.11
Add support for JAIS models (#906)
Add JAIS to README
Fix node/web interop (again)
Bump versions
[version] Update to 3.0.0-alpha.12
Set `SapiensForNormalEstimation` to encoder-only
Implement `sub` tensor operation
Bump versions
[version] Update to 3.0.0-alpha.13
Improve typing for `wrap` helper function
Update `preferredOutputLocation` type
Make `wrap` type more generic
Re-use `segmentation_data`
Fix `min` type
Add support for Hiera models
Fix reused loop variable (closes #910)
Add logits processor test file
Fix test imports
Bump versions
[version] Update to 3.0.0-alpha.14
Add another `bad_words` logits processor test (closes #913)
Add support for GroupViT
Add zero-shot-image-classification unit test
Add maskformer model definitions
Support universal image segmentation in `image-segmentation` pipeline
Add support for PVT models
Add `post_process_instance_segmentation` function template
Add `library_name` option to convert.py
Wrap onnxslim with try block
Use const where possible
Use const where possible (again)
Create `MaskFormerFeatureExtractor`
Add support for MaskFormer
Improve tool-use chat template detection
Add object detection pipeline unit test
Add support for ViTMSN and VitMAE
@xenova the option of use a quantized model doesn't exists anymore?
I'm trying to use https://huggingface.co/Xenova/trocr-base-handwritten/blob/main/onnx/encoder_model_quantized.onnx
Bump ORT versions
Create `get_chat_template` helper function
Fix CI
Run prettier on `tests/**`
move certain tests to utils subfolder
Bump onnxruntime-web version
Bump `onnxruntime==1.19.2` in scripts/requirements.txt
Merge branch 'main' into v3
Merge branch 'main' into v3
Sort `this.added_tokens` before creating regex (`.toSorted` is not avโฆ
Rather make a copy of `this.added_tokens`
Fix `.tokenize` with `fuse_unk=true`
Add blenderbot tokenizer tests
Add t5 tokenizer tests
Add falcon tokenizer tests
Run prettier
Add ESM tokenizer tests
Run unit tests in parallel
Fix `fuse_unk` for tokenizers with `byte_fallback=true` but no byte fโฆ
Add llama tokenizer unit tests
Update emoji test string names
Move whisper-specific unit tests to subfolder
Code formatting
Bump versions
[version] Update to 3.0.0-alpha.15
Add emoji tokenizer test cases for LlamaTokenizer
Attempt to fix encoder-decoder memory leak
Remove unused code
Fix BertNormalizer (strip `Mn` unicode characters)
Handle ZERO WIDTH JOINER (U+200D) characters
Add more spm normalization characters
Add emoji unit tests for bert/t5
[WebNN] Add support for specifying `free_dimension_overrides` in config
Log warning if webnn is selected by `free_dimension_overrides` is notโฆ
Fix unigram for multi-byte tokens
Add gemma tokenizer tests
Allow user to specify device and dtype in config.json
Update dependency versions
Bump versions
[version] Update to 3.0.0-alpha.16
Add CLIP tokenizer unit tests
Add more tokenizer tests
Bump onnxruntime-web version
Bump versions
[version] Update to 3.0.0-alpha.17
Add support for new `tokenizers>=0.2.0` BPE serialization format
Bump onnxruntime-web version
Bump versions
[version] Update to 3.0.0-alpha.18
Keep encoder outputs on GPU
Update whisper-webgpu demo dependencies
Bump versions
[version] Update to 3.0.0-alpha.19
Support to load ONNX APIs based on JS runtime (#947)
Allow specification of `use_external_data_format` in custom config
Update deberta unit tests
Update roberta tokenizer tests
Support inferringunigram tokenizer type
Reuse tokenizer tests for original t5-small
Remove redundant null coalesce
Enable unit test coverage reports
Use `PROBLEMATIC_REGEX_MAP` for bloom tokenizer
Improve tokenizer unit tests
Update tokenizer unit tests
Remove unused code
Add m2m_100 tokenizer unit tests
Add m2m translation pipeline unit test
Add support for Depth Pro models
Add whisper turbo alignment heads
Remove in-library list of supported models
Bump versions
[version] Update to 3.0.0-alpha.20
Add function to map tensor data array.
Merge branch 'main' into v3
Optimise loop to reduce calls to `this`
Merge branch 'pr/966' into v3
Add back tensor map test
Add support for granite models
Allow multiple optional configs to be passed (+ reduce code duplication)
Bump dependencies
Bump versions
[version] Update to 3.0.0-alpha.21
Add support for per-dtype `kv_cache_dtype`
Add text streamer unit test
Bump ORT web version
Bump versions
[version] Update to 3.0.0-alpha.22
Update repo name to `@huggingface/transformers.js`
xenova
changed the title
Update tested node versions
Bump versions
[version] Update to 3.0.0
Let's gooo ๐ ๐ ๐ Awesome work!!!
I nearly thought it would never happen ๐ An amazing achievement and thank you for your persistence!
๐ฅ ๐
WOOHOO!!! Congrats!! WebGPU all the things!
This is a huge milestone๐ Thank you for all the fantastic work in this great project!
๐ ๐ ๐
Login to write a write a comment.
In preparation for Transformers.js v3, I'm compiling a list of issues/features which will be fixed/included in the release.
onnxruntime-webto 1.17.0).onnxruntime-webโ 1.17.0). Closes:topk->top_kparameter.transpose->permuteUseful commands:
npm version prerelease --preid=alpha -m "[version] Update to %s"How to use WebGPU
First, install the development branch
Then specify the
deviceparameter when loading the model. Here's example code to get started. Please note that this is still a WORK IN PROGRESS, so the following usage may change before release.