Skip to main content
EN · FR Start for free
Menu
Lesson 14 of 14 · Mistral Vibe Advanced: Agents, Skills, and Local Mode

Local Alternatives: llama.cpp, LM Studio, Ollama

Lesson 14 of 14 Updated November 2, 2026 Advanced
On this page
  1. Beyond vLLM
  2. llama.cpp: Maximum Portability
  3. LM Studio: Everything with a Mouse
  4. Ollama: Fastest Setup
  5. Choosing, and Why the Choice Matters Little
  6. Course Recap
  7. Key Takeaways
  8. Test Your Knowledge

Lesson updated November 2, 2026 based on the Using offline models page in the Vibe documentation. Devstral has been removed from the Mistral API, but its weights remain published: Devstral Small 2 and Mistral Small 4 are the two recommended open models for local use. Vibe does not bundle a local model: you connect it to a compatible OpenAI server via a preset in config.toml, then switch using /model. Source: Using offline models.

Beyond vLLM

vLLM is the framework Vibe documentation recommends for serving Devstral, but it isn’t the only option. Three alternatives cover scenarios where vLLM struggles: an Apple Silicon Mac, a Windows machine where installation gets stuck, or the desire to test a model in five minutes. They all share the one thing that matters—they expose an OpenAI-compatible API, making them interchangeable from Vibe’s perspective.

llama.cpp: Maximum Portability

llama.cpp is an inference engine written in pure C/C++, designed to run on as many platforms as possible with minimal dependencies. This constraint is its strength: it works on Linux, macOS, and Windows, including ARM—Apple Silicon and Raspberry Pi alike. Its native GGUF quantization in Q4, Q5, or Q8 makes it particularly suitable for consumer GPUs where VRAM is limited. It requires neither Python nor a full CUDA toolkit, and its performance on Apple Silicon via Metal, as well as on NVIDIA, is excellent. The trade-off is a compilation step.

# Clone and compile
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make LLAMA_CUDA=1  # With CUDA support

# Launch the server
./llama-server \
  -m devstral-small-2-24b-instruct.Q4_K_M.gguf \
  --port 8080 \
  -ngl 99 \
  -c 32768

Key flags:

  • -m: path to the model in GGUF format
  • -ngl 99: number of layers offloaded to the GPU (99 = all)
  • -c 32768: context window size
  • --port 8080: API port

In practice, you adjust -ngl: if the model refuses to load, reduce the number of layers sent to the GPU. Choose llama.cpp if you’re on an Apple Silicon Mac (M1 to M4), want to avoid any Python dependencies, need the most aggressive quantization for a small GPU, or are working on ARM—Graviton server, Raspberry Pi.

LM Studio: Everything with a Mouse

LM Studio is a desktop application that reduces local deployment to a few clicks. Its interface handles download, configuration, and launch; its catalog lets you browse and fetch models from Hugging Face without the command line; its built-in OpenAI-compatible API server requires no tweaking. The app runs on Windows, macOS, and Linux. Setup happens entirely in the interface:

  1. Open LM Studio
  2. Download Devstral Small 2 from the catalog
  3. Load the model
  4. Enable the local server (“Start Server” button)
  5. In Vibe: declare the LM Studio server as an OpenAI-compatible provider in config.toml (its address appears in LM Studio), then select the model with /model

Watch the port: LM Studio listens on 1234 by default. If your Vibe expects port 8080, adjust either LM Studio’s configuration or Vibe’s—this is the most common cause of “Vibe can’t find the server” even though the model is loaded. LM Studio suits you if you’re new to local models, prefer a GUI over the command line, want to quickly test several models, or are on Windows and vLLM installation is giving you trouble.

Ollama: Fastest Setup

Ollama borrows Docker’s ergonomics: one command to fetch a model, another to run it. Installation takes one line, management uses ollama pull, ollama run, and ollama list, the OpenAI-compatible API is built in and active by default, and models update like Docker images.

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Download and run Devstral Small 2
ollama pull devstral-small-2

# Launch the model (API server starts automatically)
ollama run devstral-small-2

The server listens on port 11434, with the OpenAI-compatible endpoint at http://localhost:11434/v1/: this is the api_base to declare in the Vibe provider, with name = "devstral-small-2" in the model preset. Pick Ollama if you want to get started as fast as possible, juggle multiple models to test, are comfortable with Docker workflows, or want an inference server running permanently in the background.

Choosing, and Why the Choice Matters Little

Each framework occupies a distinct niche. vLLM delivers the best production performance with native multi-GPU support: it’s the choice for dedicated servers equipped with H100s or A100s. llama.cpp wins on portability, shines on Apple Silicon and consumer GPUs, and skips Python. LM Studio wins on ergonomics, with a full GUI ideal for discovering local models. Ollama offers the fastest setup and simplified model management, great for testing and prototyping. Yet this choice matters little, because they all expose the same OpenAI-format API—this common denominator lets Vibe connect without knowing anything about the engine running behind it.

GET  /v1/models              # List available models
POST /v1/chat/completions    # Chat completion (main)
GET  /health                 # Server status (vLLM)

Configuration in Vibe is therefore identical across frameworks: only the port changes.

vLLM      → http://localhost:8080
LM Studio → http://localhost:1234
Ollama    → http://localhost:11434
llama.cpp → http://localhost:8080

You can start with Ollama to get your feet wet, migrate to llama.cpp to optimize quantization, then switch to vLLM the day your team shares a GPU server—without touching your Vibe configuration.

Course Recap

This journey took you through Mistral Vibe’s advanced ecosystem. You learned to control who executes your tasks and with what permissions, via built-in agents then custom agents, and to calibrate risk with real locks: tool lists, per-tool permissions, and trusted folders, with the safety field serving only as a visual label. Unified modes bundle tools, permissions, and behaviors into activatable profiles; subagents delegate specialized tasks independently. You then built reusable components with skills and their SKILL.md, launched by slash commands in a single gesture, and assembled modes tailored to your development cycle—review, deploy, lint. Multi-choice clarification showed you how Vibe asks before acting when things are ambiguous, and local mode with Devstral opened the path to a confidential, offline, sovereign environment. You now have all the tools to build an AI-assisted development environment that respects your security, performance, and sovereignty constraints.

Key Takeaways

  • llama.cpp: Pure C++, cross-platform, excellent on Apple Silicon
  • LM Studio: Graphical interface, ideal for beginners
  • Ollama: Fastest setup, Docker-like workflow
  • vLLM: Recommended for production with server GPUs
  • All expose an OpenAI-compatible API—Vibe connects transparently
  • Choose your framework based on your hardware and comfort level

Test Your Knowledge

Agents, skills, modes, local: advanced Vibe in five points.

1. What do custom agents in TOML allow?

Answer: Define your own agents—instructions, tools, behavior—in a TOML file: your team’s specialized agent becomes a versioned artifact.

2. Does an agent's safety field (safe, neutral, destructive, smart, yolo) limit its actions?

Answer: No. It’s a visual cue that colors the input border. What actually limits an agent are enabled_tools and disabled_tools, the [tools.<tool>] permissions (always, ask, never), the working directory scope, and trusted folders.

3. Skills and slash commands: how do they work together?

Answer: A skill (SKILL.md) encapsulates a reusable procedure; the slash command triggers it in one gesture—discovery, activation, and priority manage the catalog.

4. What are subagents for?

Answer: Delegating specialized subtasks to dedicated agents—the main agent orchestrates, subagents execute their specialty.

5. How do you run Vibe entirely locally?

Answer: By serving a recommended open model (Devstral Small 2 or Mistral Small 4) with vLLM or an alternative (llama.cpp, LM Studio, Ollama), declaring it as a provider in config.toml, then disabling telemetry and update checks (enable_telemetry = false, enable_update_checks = false).

Agents declared, security graded, skills tooled, local possible: advanced Vibe is a platform—configure it to match your team.

← Previous lessonDeploy Devstral Locally with vLLM
← Back to catalog