Project Roadmap

[supreme-gg-gg]

This tracking issue aims to document the project's upcoming features. We ship frequent updates because the LLM ecosystem evolves very rapidly, and a lot of our dependent frameworks & libraries are still being developed.

Completed

• Separate evaluation & inference service + new features Improved evaluation and inference service #53 Support vLLM for inference & eval engine #27
• Deploy on your own GCP project with Terraform IaC to facilitate custom cloud deployment #17
• New separate export service for better flexibility Separate export feature that merges the model or exports it in GGUF format #43 Convert HF models to GGUF with llama.cpp #31
• Documentation Setting up documentation! #61

Work in progress

• Loading fine tuned models to continue training, and with a different trainer [feature] Load saved model to fine tune #48
• Ways to integrate deployment of fine tuned models Deploy fine tuned model to Cloud Run #28
• Fixing a few technical debt with GCP architecture: Replace GCS + Jobs API with a proper Pub/Sub queue #69 Integrate training and export service #67
• Fixing datasets technical debt [question] Change vision dataset formatting in format converter #50

Planned features

• Sharing datasets (and training jobs) on cloud Sharing datasets and training jobs #40
• Instruction tuning explicit support Support instruction tuning datasets #37 Add custom chat template support for instructing tuning #62
• New log streaming system to avoid dependency on WandB [feature] built-in tracker with HF Trackio as alternative to WandB #58
• Benchmark feature in evaluations Add benchmarking in evaluation service #60

[supreme-gg-gg]

^The backlog tag means we are not presently working on it, i.e. for grabs 😄