Overview
Soup is a revolutionary command-line interface designed to simplify and democratize LLM fine-tuning. It eliminates the complexities of SSH and intricate configuration files, offering a streamlined, single-command workflow. The project's standout innovation, "layer streaming," enables the fine-tuning of substantial models, such as an 8B parameter model, on hardware as modest as a 4GB laptop GPU.
Key Features
- Effortless Fine-Tuning: Train and post-train LLMs with a single YAML configuration file and one command.
- Low-Resource Training: Features "layer streaming" to dramatically reduce VRAM requirements, enabling 8B models to run on a 4GB GPU with impressive speed (e.g., 119.6 tok/s measured on an RTX 3050 Laptop).
- Zero Configuration Hell: Automates batch sizing, GPU detection, and quantization, removing common setup hurdles.
- Local Development Friendly: Train directly on your own GPU using QLoRA, negating the need for cloud infrastructure.
- Extensive Model Support: Compatible with virtually any text-generation model available on the HuggingFace Hub that loads with
AutoModelForCausalLM. - Rich Template System: Over 20 pre-defined templates for various tasks like chat, code generation, tool-calling, and more.
- Comprehensive Documentation: Detailed guides covering advanced training methods, PEFT techniques, data engineering, serving, and export options.
Typical Use Cases
Soup is ideal for:
- Individual developers and researchers: Fine-tune LLMs on personal hardware without complex setups, making advanced AI accessible to everyone.
- Small to medium-sized teams: Rapidly iterate on LLM customization for specific applications without dedicated MLOps expertise.
- Educational purposes: Learn and experiment with LLM fine-tuning on widely available hardware.
- Prototyping and experimentation: Quickly adapt open-source models for niche tasks or novel research directions.
- Deploying custom LLM solutions: Easily merge, export, and serve fine-tuned models in various formats (GGUF, ONNX, etc.) for integration into applications.
