Skip to content

Commit 1714b30

Browse files
LostenergydrinkLostenergydrink
committed
Release v1.1.0 - Batch inference, deployment tools, Windows fixes
1 parent a40a698 commit 1714b30

12 files changed

Lines changed: 2835 additions & 78 deletions

‎CHANGELOG/v1.1.0.md‎

Lines changed: 279 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,279 @@
1+
# Changelog - v1.1.0
2+
3+
**Release Date:** November 2025
4+
**Type:** Major Feature Release
5+
6+
---
7+
8+
## 🎯 Overview
9+
10+
This release adds comprehensive post-training tools and Windows debugging utilities based on 7 intensive debugging sessions and 50-epoch training runs. All features are battle-tested and production-ready.
11+
12+
---
13+
14+
## ✨ New Features
15+
16+
### 1. Batch Inference System 🆕
17+
18+
Automatically test all checkpoint epochs and find your best model.
19+
20+
**New Files:**
21+
- `styletts2-setup/batch_inference_epochs.py` - Test all 50 checkpoints
22+
- `styletts2-setup/analyze_inference_results.py` - Statistical analysis & plots
23+
- `styletts2-setup/inference_single_checkpoint.py` - Interactive testing
24+
- `styletts2-setup/run_batch_inference.bat` - Full test launcher
25+
- `styletts2-setup/run_batch_inference_sampled.bat` - Quick test (every 5th epoch)
26+
- `styletts2-setup/run_interactive_inference.bat` - Interactive CLI
27+
- `docs/BATCH_INFERENCE_GUIDE.md` - Complete documentation
28+
29+
**Capabilities:**
30+
- Tests 7 diverse sentences per epoch
31+
- Calculates Real-Time Factor (RTF) metrics
32+
- Generates comparison plots (RTF vs Epoch, Distribution, etc.)
33+
- Identifies best checkpoint automatically
34+
- CSV export for analysis
35+
36+
**Usage:**
37+
```batch
38+
cd styletts2-setup
39+
run_batch_inference_sampled.bat # Quick test (~15 min)
40+
python analyze_inference_results.py # Analyze results
41+
```
42+
43+
---
44+
45+
### 2. Fine-Tuned Model Deployment 🆕
46+
47+
Production-ready deployment options for trained models.
48+
49+
**New Files:**
50+
- `styletts2-setup/finetuned_webui.py` - Dedicated Gradio UI
51+
- `styletts2-setup/launch_finetuned_webui.bat` - UI launcher
52+
- `docs/FINETUNED_MODEL_DEPLOYMENT.md` - Comprehensive guide
53+
54+
**Features:**
55+
- Dedicated WebUI for your trained voice
56+
- Adjustable quality (diffusion steps 3-20)
57+
- Emotion scale control (0.5-2.0)
58+
- Seed-based reproducibility
59+
- Automatic file saving with timestamps
60+
- VRAM auto-cleanup
61+
62+
**Deployment Options:**
63+
1. Dedicated WebUI (recommended)
64+
2. Interactive CLI
65+
3. Integration into main WebUI
66+
67+
**Usage:**
68+
```batch
69+
# 1. Configure best epoch in finetuned_webui.py
70+
# 2. Launch
71+
launch_finetuned_webui.bat
72+
# Opens at http://127.0.0.1:7861
73+
```
74+
75+
---
76+
77+
### 3. Enhanced Dependency Management 🆕
78+
79+
Comprehensive documentation for all known conflicts and resolutions.
80+
81+
**New Files:**
82+
- `styletts2-setup/requirements.txt` - 300+ lines with tested versions
83+
- `styletts2-setup/requirements-dev.txt` - Development dependencies
84+
- `docs/DEPENDENCY_MANAGEMENT.md` - Conflict resolution guide
85+
86+
**Documented Issues (6 major conflicts):**
87+
1. huggingface-hub version conflict (styletts2 package)
88+
2. langchain version lock
89+
3. monotonic_align PyPI issue
90+
4. PyTorch version compatibility
91+
5. Windows DataLoader fork bomb
92+
6. espeak-ng system dependency
93+
94+
**Installation Order:**
95+
```powershell
96+
# 1. Install PyTorch with CUDA
97+
pip install torch==2.5.1+cu121 torchaudio==2.5.1+cu121 --index-url https://download.pytorch.org/whl/cu121
98+
99+
# 2. Install main dependencies
100+
pip install -r requirements.txt
101+
102+
# 3. Install monotonic_align from GitHub
103+
pip install git+https://github.com/resemble-ai/monotonic_align.git
104+
105+
# 4. OPTIONAL: Install styletts2 package
106+
pip install styletts2==0.1.6 --no-deps
107+
```
108+
109+
**Memory Estimates:**
110+
- Full installation: ~8-10 GB
111+
- Training (batch_size=8): ~10 GB VRAM
112+
- Inference: ~2 GB VRAM
113+
114+
---
115+
116+
### 4. Windows Training Utilities 🆕
117+
118+
Comprehensive fixes for all Windows-specific training issues.
119+
120+
**New Files:**
121+
- `styletts2-setup/run_finetune_safe.bat` - Safe training launcher
122+
- `styletts2-setup/install_monotonic_align.py` - Automated installer
123+
- `docs/WINDOWS_TRAINING_ISSUES.md` - Complete troubleshooting guide
124+
125+
**Critical Fixes (7 major issues):**
126+
1. **Windows DataLoader Fork Bomb** - Forces num_workers=0
127+
2. **Working Directory Path Resolution** - Ensures correct working dir
128+
3. **CUDA Kernel Silent Crashes** - CUDA_LAUNCH_BLOCKING=1
129+
4. **monotonic_align Installation** - GitHub-based installer
130+
5. **espeak-ng Not Found** - Installation guide
131+
6. **Import Delays** - User notification (normal 1-2 min wait)
132+
7. **Venv Path Portability** - E-drive policy enforcement
133+
134+
**Safe Mode Features:**
135+
- CUDA debugging flags enabled
136+
- Automatic monotonic_align check
137+
- Working directory validation
138+
- Config existence verification
139+
- Clear error messages
140+
141+
**Usage:**
142+
```batch
143+
cd styletts2-setup
144+
run_finetune_safe.bat
145+
```
146+
147+
---
148+
149+
## 🔧 Improvements
150+
151+
### Documentation
152+
153+
**New Guides:**
154+
- `BATCH_INFERENCE_GUIDE.md` - 200+ lines, checkpoint evaluation
155+
- `FINETUNED_MODEL_DEPLOYMENT.md` - 400+ lines, 3 deployment options
156+
- `DEPENDENCY_MANAGEMENT.md` - 300+ lines, conflict resolutions
157+
- `WINDOWS_TRAINING_ISSUES.md` - 400+ lines, 7 critical fixes
158+
159+
**Updated:**
160+
- `README.md` - Added all new features to structure and features list
161+
- Repository structure fully documented
162+
163+
### Code Quality
164+
165+
**CI Compliance:**
166+
- All hardcoded paths removed
167+
- Generic venv detection (.venv or venv)
168+
- Relative paths throughout
169+
- Will pass all existing CI checks
170+
171+
**Error Handling:**
172+
- Better error messages in batch scripts
173+
- Validation checks before operations
174+
- Graceful failure modes
175+
176+
---
177+
178+
## 📊 Testing & Validation
179+
180+
**Tested On:**
181+
- Windows 10/11
182+
- Python 3.10.11
183+
- NVIDIA RTX 3060 12GB
184+
- CUDA 12.1
185+
186+
**Training Runs:**
187+
- 50-epoch fine-tuning (multiple runs)
188+
- Batch inference (350 samples per run)
189+
- 7 debugging sessions worth of fixes
190+
191+
**Confidence Level:** HIGHEST - All features battle-tested in production
192+
193+
---
194+
195+
## 🐛 Bug Fixes
196+
197+
### Windows-Specific
198+
199+
1. **DataLoader runaway processes** - Patched in meldataset.py
200+
2. **Checkpoint path resolution** - Fixed in training launcher
201+
3. **Silent CUDA crashes** - Detection with CUDA_LAUNCH_BLOCKING
202+
4. **Import timeout confusion** - User notification added
203+
204+
### Cross-Platform
205+
206+
1. **monotonic_align installation** - Automated installer provided
207+
2. **Dependency conflicts** - All documented with solutions
208+
3. **espeak-ng detection** - Better error messages
209+
210+
---
211+
212+
## 📦 Migration Guide
213+
214+
### From v1.0.0
215+
216+
**No breaking changes!** All existing functionality preserved.
217+
218+
**New optional features:**
219+
```powershell
220+
# Update to latest
221+
git pull origin main
222+
223+
# Install new dependencies (if needed)
224+
pip install -r styletts2-setup/requirements.txt
225+
226+
# Try new features
227+
cd styletts2-setup
228+
run_batch_inference_sampled.bat
229+
```
230+
231+
**Recommended:**
232+
1. Read `docs/DEPENDENCY_MANAGEMENT.md` if you had install issues
233+
2. Read `docs/WINDOWS_TRAINING_ISSUES.md` if training had problems
234+
3. Try batch inference to find your best checkpoint
235+
236+
---
237+
238+
## 📝 Known Issues
239+
240+
1. **PyTorch 2.6.0 not tested** - Stick with 2.5.1 for now
241+
2. **Test suite pending** - Will be added in v1.2.0
242+
3. **Linux testing needed** - Primarily tested on Windows
243+
244+
---
245+
246+
## 🔮 What's Next (v1.2.0)
247+
248+
Planned for next release:
249+
- Automated test suite (pytest)
250+
- CI/CD with test coverage
251+
- Additional inference optimization
252+
- Enhanced documentation organization
253+
254+
---
255+
256+
## 🙏 Acknowledgments
257+
258+
This release incorporates:
259+
- 7 intensive debugging sessions
260+
- 50+ epoch training runs
261+
- Community feedback on Windows issues
262+
- Extensive battle-testing in production
263+
264+
---
265+
266+
## 📚 Resources
267+
268+
- [Installation Guide](docs/STYLETTS2_INSTALLATION.md)
269+
- [Batch Inference Guide](docs/BATCH_INFERENCE_GUIDE.md)
270+
- [Deployment Guide](docs/FINETUNED_MODEL_DEPLOYMENT.md)
271+
- [Windows Issues](docs/WINDOWS_TRAINING_ISSUES.md)
272+
- [Dependencies](docs/DEPENDENCY_MANAGEMENT.md)
273+
- [Troubleshooting](docs/TROUBLESHOOTING.md)
274+
275+
---
276+
277+
**Full Changelog:** https://github.com/[username]/styletts2-dataset-toolkit/compare/v1.0.0...v1.1.0
278+
279+
**Download:** https://github.com/[username]/styletts2-dataset-toolkit/releases/tag/v1.1.0

‎README.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,8 @@ A comprehensive toolkit for isolating vocals, preparing datasets, and fine-tunin
2323
### 🗣️ StyleTTS2 Integration **✨ ENHANCED**
2424
- **Auto-normalization** built into WebUI export (no manual fixes needed!)
2525
- **Safe slider limits** (3-30 seconds) prevent BERT token overflow
26+
- **Batch inference system** 🆕 Test all checkpoints, find best epoch
27+
- **Fine-tuned model WebUI** 🆕 Dedicated interface for trained voice
2628
- **Validation & normalization tools** catch issues before training
2729
- **CPU/CUDA auto-detection** with fallback support
2830
- **Windows DataLoader fixes** (no runaway processes)
@@ -188,9 +190,13 @@ styletts2-dataset-toolkit/
188190
│ ├── batch_inference_epochs.py # 🆕 Test all checkpoints automatically
189191
│ ├── analyze_inference_results.py # 🆕 Statistical analysis & plots
190192
│ ├── inference_single_checkpoint.py # 🆕 Interactive single-checkpoint testing
193+
│ ├── finetuned_webui.py # 🆕 Dedicated UI for trained model
191194
│ ├── run_batch_inference.bat # 🆕 Test all 50 epochs (~1-2 hours)
192195
│ ├── run_batch_inference_sampled.bat # 🆕 Quick test every 5th epoch
193196
│ ├── run_interactive_inference.bat # 🆕 Interactive generation CLI
197+
│ ├── launch_finetuned_webui.bat # 🆕 Launch fine-tuned model UI
198+
│ ├── run_finetune_safe.bat # 🆕 Safe training launcher (CUDA flags, path fixes)
199+
│ ├── install_monotonic_align.py # 🆕 Automated monotonic_align installer
194200
│ ├── train_styletts2.bat # Training launcher
195201
│ ├── train_styletts2.ps1 # PowerShell training launcher
196202
│ ├── apply_patches.ps1 # ✨ Auto-apply code patches
@@ -210,6 +216,9 @@ styletts2-dataset-toolkit/
210216
│ ├── DATASET_PREP_GUIDE.md # ✨ Updated with auto-normalization
211217
│ ├── DATASET_REQUIREMENTS.md # ✨ Critical constraints explained
212218
│ ├── BATCH_INFERENCE_GUIDE.md # 🆕 Checkpoint evaluation system
219+
│ ├── FINETUNED_MODEL_DEPLOYMENT.md # 🆕 Production deployment guide
220+
│ ├── DEPENDENCY_MANAGEMENT.md # 🆕 Package conflicts & solutions
221+
│ ├── WINDOWS_TRAINING_ISSUES.md # 🆕 Windows-specific fixes (7 critical issues)
213222
│ ├── WEBUI_IMPROVEMENTS.md # ✨ Technical changelog
214223
│ └── TROUBLESHOOTING.md # Common issues & solutions
215224
│

0 commit comments

Comments
 (0)