Overview

MTPLX is a cutting-edge, native application and command-line tool designed to accelerate local Large Language Model (LLM) inference on Apple Silicon. It achieves remarkable speed improvements by leveraging the seldom-utilized Multi-Token Prediction (MTP) heads present in modern LLMs like Qwen. Unlike other solutions, MTPLX natively integrates these MTP capabilities without requiring external draft models, ensuring no additional RAM overhead and preserving the original output distribution of the model. Experience dramatically faster token generation while maintaining output fidelity.

Key Features

• Native MTP Speculative Decoding: Unlocks significant speedups by utilizing a model's built-in MTP heads for speculative token generation and batched verification.
• Unparalleled Speed: Achieves up to 2x faster inference speeds on Apple Silicon (e.g., 1.6x on M4, 2.24x on M5 Max), delivering measurable performance gains.
• RAM Efficiency: Eliminates the need for separate drafter models, preventing additional memory consumption.
• Output Fidelity: Guarantees exact rejection sampling with residual correction, ensuring that output distributions remain identical to standard decoding at any temperature or top_p setting.
• User-Friendly Mac App: A seamless experience with automatic hardware detection, model recommendation, download management, Python engine setup, fan control, and PATH configuration.
• Powerful CLI: Offers comprehensive control for advanced users, including model management, tuning, MTP model forging, benchmarking, and server configuration.
• Auto-Tuning: Intelligently measures and configures the optimal MTP draft depth for your specific Apple Silicon hardware.
• Forge MTP Models: Empowers users to convert Hugging Face models into MTPLX-ready MTP models, including training MTP adapters and verifying performance gains.
• OpenAI & Anthropic API Compatibility: Serves a fully compatible API, allowing seamless integration with existing tools and frameworks like OpenAI Python clients, vLLM, and popular UIs.
• Retrieval Model Support: Can serve embedding and reranker models alongside LLMs, simplifying RAG and agent-memory setups without a separate inference server.
• Advanced Sampler Controls: Fine-tune generation with temperature, top_p, top_k, and penalty parameters, configurable per request or via server defaults.
• Transparent Compatibility Reporting: mtplx inspect provides clear classification of model compatibility before any weights are loaded.

Typical Use Cases

• Accelerated Local LLM Chat: Experience significantly faster responses from your favorite LLMs directly on your Mac for conversational AI.
• Enhanced Coding Assistance: Boost productivity with quicker code generation and analysis from models optimized for programming tasks, such as the Qwen 3.8 27B Optimized Speed.
• LLM Development and Experimentation: Easily build, verify, and deploy your own MTP-enabled models using the "Forge" feature.
• Rapid Prototyping with RAG: Integrate LLMs with retrieval-augmented generation pipelines by leveraging MTPLX's built-in embedding and reranker serving capabilities.
• API-Driven Applications: Power applications that rely on local LLM inference through the OpenAI and Anthropic API-compatible server.
• Benchmarking and Performance Analysis: Accurately measure and compare LLM performance on your specific Apple Silicon hardware, with tools like the AIME benchmark runner.