Skip to content

Commit 93d4ca8

Browse files
juntaoclaude
andcommitted
Add Quick Start section to README with prebuilt binary instructions
Signed-off-by: Michael Yuan <michael@secondstate.io> Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 75e4795 commit 93d4ca8

1 file changed

Lines changed: 116 additions & 81 deletions

File tree

README.md

Lines changed: 116 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -6,121 +6,108 @@ Pure Rust implementation of [Qwen3-ASR](https://github.com/QwenLM/Qwen3-ASR) aut
66

77
Supports two backends: **libtorch** (via the `tch` crate, cross-platform with optional CUDA) and **MLX** (Apple Silicon native via Metal GPU). Loads model weights directly from safetensors files and re-implements the complete neural network forward pass in Rust.
88

9-
## Architecture
9+
## Quick Start
1010

11-
The implementation ports the Qwen3-ASR encoder-decoder architecture from PyTorch/Transformers to Rust with libtorch (via the `tch` crate):
11+
### 1. Download the binary
1212

13-
- **Audio Encoder** (Whisper-style): 3x Conv2d downsampling → sinusoidal positional embeddings → 18 transformer encoder layers → output projection (896 → 1024)
14-
- **Text Decoder** (Qwen3): 28 transformer decoder layers with Grouped Query Attention (16 Q heads / 8 KV heads), QK-normalization, MRoPE (Multimodal Rotary Position Embeddings), and SwiGLU MLP
15-
- **Audio preprocessing**: FFmpeg decodes any audio format → resampled to mono 16kHz f32 → 128-bin log-mel spectrogram (Whisper-style)
13+
Download the latest release for your platform from [GitHub Releases](https://github.com/second-state/qwen3_asr_rs/releases/latest) and extract:
1614

17-
## Supported Models
15+
**macOS (Apple Silicon)**
1816

19-
| Model | Parameters | HuggingFace |
20-
|-------|-----------|-------------|
21-
| Qwen3-ASR-0.6B | 0.6B | [Qwen/Qwen3-ASR-0.6B](https://huggingface.co/Qwen/Qwen3-ASR-0.6B) |
22-
| Qwen3-ASR-1.7B | 1.7B | [Qwen/Qwen3-ASR-1.7B](https://huggingface.co/Qwen/Qwen3-ASR-1.7B) |
17+
```bash
18+
curl -LO https://github.com/second-state/qwen3_asr_rs/releases/latest/download/asr-macos-aarch64.zip
19+
unzip asr-macos-aarch64.zip
20+
# Contains: asr-macos-aarch64/asr and asr-macos-aarch64/mlx.metallib
21+
```
2322

24-
## Prerequisites
23+
**Linux x86_64 (CPU)**
2524

26-
### Backend
25+
```bash
26+
curl -LO https://github.com/second-state/qwen3_asr_rs/releases/latest/download/asr-linux-x86_64.zip
27+
unzip asr-linux-x86_64.zip
28+
# Contains: asr-linux-x86_64/asr
29+
```
2730

28-
Choose one backend:
31+
**Linux x86_64 (CUDA)**
2932

30-
| Backend | Feature flag | Platforms | GPU |
31-
|---------|-------------|-----------|-----|
32-
| libtorch | `tch-backend` (default) | Linux, macOS, Windows | CUDA |
33-
| MLX | `mlx` | macOS Apple Silicon | Metal |
34-
35-
### libtorch (for `tch-backend`)
33+
```bash
34+
curl -LO https://github.com/second-state/qwen3_asr_rs/releases/latest/download/asr-linux-x86_64-cuda.zip
35+
unzip asr-linux-x86_64-cuda.zip
36+
# Contains: asr-linux-x86_64-cuda/asr
37+
```
3638

37-
The `tch` crate (v0.20) requires **libtorch 2.7.1**. Download and extract for your platform:
39+
**Linux ARM64**
3840

3941
```bash
40-
# macOS (Apple Silicon)
41-
curl -LO https://download.pytorch.org/libtorch/cpu/libtorch-macos-arm64-2.7.1.zip
42-
unzip libtorch-macos-arm64-2.7.1.zip
42+
curl -LO https://github.com/second-state/qwen3_asr_rs/releases/latest/download/asr-linux-aarch64.zip
43+
unzip asr-linux-aarch64.zip
44+
# Contains: asr-linux-aarch64/asr
45+
```
46+
47+
### 2. Download libtorch (Linux only)
4348

49+
macOS uses the MLX backend and does not need libtorch.
50+
51+
```bash
4452
# Linux x86_64 (CPU)
4553
curl -LO https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-2.7.1%2Bcpu.zip
4654
unzip libtorch-cxx11-abi-shared-with-deps-2.7.1+cpu.zip
4755

48-
# Linux ARM64 (CPU)
49-
curl -LO https://github.com/second-state/libtorch-releases/releases/download/v2.7.1/libtorch-cxx11-abi-aarch64-2.7.1.tar.gz
50-
tar xzf libtorch-cxx11-abi-aarch64-2.7.1.tar.gz
51-
5256
# Linux x86_64 (CUDA 12.8)
5357
curl -LO https://download.pytorch.org/libtorch/cu128/libtorch-cxx11-abi-shared-with-deps-2.7.1%2Bcu128.zip
5458
unzip libtorch-cxx11-abi-shared-with-deps-2.7.1+cu128.zip
55-
```
5659

57-
### FFmpeg
58-
59-
Install FFmpeg development libraries:
60-
61-
```bash
62-
# macOS
63-
brew install ffmpeg
64-
65-
# Ubuntu/Debian
66-
sudo apt-get install libavcodec-dev libavformat-dev libavutil-dev libswresample-dev pkg-config
60+
# Linux ARM64
61+
curl -LO https://github.com/second-state/libtorch-releases/releases/download/v2.7.1/libtorch-cxx11-abi-aarch64-2.7.1.tar.gz
62+
tar xzf libtorch-cxx11-abi-aarch64-2.7.1.tar.gz
6763
```
6864

69-
### Model Weights
70-
71-
Download a model from HuggingFace:
65+
### 3. Download model weights
7266

7367
```bash
74-
# 0.6B model
75-
huggingface-cli download Qwen/Qwen3-ASR-0.6B --local-dir Qwen3-ASR-0.6B
68+
pip install huggingface_hub transformers
7669

77-
# 1.7B model (sharded safetensors, auto-detected)
78-
huggingface-cli download Qwen/Qwen3-ASR-1.7B --local-dir Qwen3-ASR-1.7B
79-
```
80-
81-
Generate `tokenizer.json` (required):
70+
huggingface-cli download Qwen/Qwen3-ASR-0.6B --local-dir Qwen3-ASR-0.6B
8271

83-
```bash
8472
python -c "
8573
from transformers import AutoTokenizer
8674
tok = AutoTokenizer.from_pretrained('Qwen3-ASR-0.6B', trust_remote_code=True)
8775
tok.backend_tokenizer.save('Qwen3-ASR-0.6B/tokenizer.json')
8876
"
8977
```
9078

91-
## Build
92-
93-
### libtorch backend (default)
79+
### 4. Transcribe
9480

9581
```bash
96-
# Set environment
97-
export LIBTORCH=$(pwd)/libtorch
98-
export LIBTORCH_BYPASS_VERSION_CHECK=1
99-
export LD_LIBRARY_PATH=$LIBTORCH/lib:$LD_LIBRARY_PATH # Linux
100-
export DYLD_LIBRARY_PATH=$LIBTORCH/lib:$DYLD_LIBRARY_PATH # macOS
82+
# macOS (MLX backend — no extra env needed)
83+
./asr-macos-aarch64/asr Qwen3-ASR-0.6B input.wav
10184

102-
# Build (dynamically links FFmpeg)
103-
cargo build --release
85+
# Linux
86+
LD_LIBRARY_PATH=$(pwd)/libtorch/lib:$LD_LIBRARY_PATH \
87+
./asr-linux-x86_64/asr Qwen3-ASR-0.6B input.wav
88+
```
10489

105-
# Build with statically linked FFmpeg
106-
cargo build --release --features static-ffmpeg
90+
Output:
10791

108-
# Build FFmpeg from source and link statically (most self-contained)
109-
cargo build --release --features build-ffmpeg
92+
```
93+
Language: English
94+
Text: Thank you for your contribution to the most recent issue of Computer.
11095
```
11196

112-
### MLX backend (macOS Apple Silicon)
97+
## Architecture
11398

114-
```bash
115-
# Initialize mlx-c submodule
116-
git submodule update --init --recursive
99+
The implementation ports the Qwen3-ASR encoder-decoder architecture from PyTorch/Transformers to Rust with libtorch (via the `tch` crate):
117100

118-
# Build with MLX (no libtorch needed)
119-
cargo build --release --no-default-features --features mlx
101+
- **Audio Encoder** (Whisper-style): 3x Conv2d downsampling → sinusoidal positional embeddings → 18 transformer encoder layers → output projection (896 → 1024)
102+
- **Text Decoder** (Qwen3): 28 transformer decoder layers with Grouped Query Attention (16 Q heads / 8 KV heads), QK-normalization, MRoPE (Multimodal Rotary Position Embeddings), and SwiGLU MLP
103+
- **Audio preprocessing**: FFmpeg decodes any audio format → resampled to mono 16kHz f32 → 128-bin log-mel spectrogram (Whisper-style)
120104

121-
# With statically linked FFmpeg
122-
cargo build --release --no-default-features --features mlx,static-ffmpeg
123-
```
105+
## Supported Models
106+
107+
| Model | Parameters | HuggingFace |
108+
|-------|-----------|-------------|
109+
| Qwen3-ASR-0.6B | 0.6B | [Qwen/Qwen3-ASR-0.6B](https://huggingface.co/Qwen/Qwen3-ASR-0.6B) |
110+
| Qwen3-ASR-1.7B | 1.7B | [Qwen/Qwen3-ASR-1.7B](https://huggingface.co/Qwen/Qwen3-ASR-1.7B) |
124111

125112
## Usage
126113

@@ -132,19 +119,10 @@ asr ./Qwen3-ASR-0.6B input.wav
132119
asr ./Qwen3-ASR-0.6B input.wav chinese
133120
asr ./Qwen3-ASR-0.6B input.wav english
134121

135-
# Any audio format (FFmpeg handles conversion)
136-
asr ./Qwen3-ASR-0.6B input.mp3
137-
asr ./Qwen3-ASR-0.6B input.flac
138-
asr ./Qwen3-ASR-0.6B input.m4a
139-
140122
# Enable debug logging
141123
RUST_LOG=debug asr ./Qwen3-ASR-0.6B input.wav
142124
```
143125

144-
### Input Audio Requirements
145-
146-
The `asr` binary accepts **any audio format** supported by FFmpeg. The audio is automatically converted to mono 16 kHz f32 for the model's mel spectrogram computation.
147-
148126
### Output Format
149127

150128
```
@@ -156,6 +134,63 @@ Text: 你好世界
156134

157135
Qwen3-ASR supports 30 languages: Chinese, English, Cantonese, Arabic, German, French, Spanish, Portuguese, Indonesian, Italian, Korean, Russian, Thai, Vietnamese, Japanese, Turkish, Hindi, Malay, Dutch, Swedish, Danish, Finnish, Polish, Czech, Filipino, Persian, Greek, Romanian, Hungarian, Macedonian.
158136

137+
## Build from Source
138+
139+
### Backend
140+
141+
Choose one backend:
142+
143+
| Backend | Feature flag | Platforms | GPU |
144+
|---------|-------------|-----------|-----|
145+
| libtorch | `tch-backend` (default) | Linux, macOS, Windows | CUDA |
146+
| MLX | `mlx` | macOS Apple Silicon | Metal |
147+
148+
### Prerequisites
149+
150+
**libtorch** (for `tch-backend`): See [Step 2](#2-download-libtorch-linux-only) above for download links.
151+
152+
**FFmpeg** development libraries:
153+
154+
```bash
155+
# macOS
156+
brew install ffmpeg
157+
158+
# Ubuntu/Debian
159+
sudo apt-get install libavcodec-dev libavformat-dev libavutil-dev libswresample-dev pkg-config
160+
```
161+
162+
### libtorch backend (default)
163+
164+
```bash
165+
# Set environment
166+
export LIBTORCH=$(pwd)/libtorch
167+
export LIBTORCH_BYPASS_VERSION_CHECK=1
168+
export LD_LIBRARY_PATH=$LIBTORCH/lib:$LD_LIBRARY_PATH # Linux
169+
export DYLD_LIBRARY_PATH=$LIBTORCH/lib:$DYLD_LIBRARY_PATH # macOS
170+
171+
# Build (dynamically links FFmpeg)
172+
cargo build --release
173+
174+
# Build with statically linked FFmpeg
175+
cargo build --release --features static-ffmpeg
176+
177+
# Build FFmpeg from source and link statically (most self-contained)
178+
cargo build --release --features build-ffmpeg
179+
```
180+
181+
### MLX backend (macOS Apple Silicon)
182+
183+
```bash
184+
# Initialize mlx-c submodule
185+
git submodule update --init --recursive
186+
187+
# Build with MLX (no libtorch needed)
188+
cargo build --release --no-default-features --features mlx
189+
190+
# With statically linked FFmpeg
191+
cargo build --release --no-default-features --features mlx,static-ffmpeg
192+
```
193+
159194
## Project Structure
160195

161196
```

0 commit comments

Comments
 (0)