Yukkai Local (MLX)

Yukkai Local provides on-device inference via Apple's MLX framework — it runs a downloaded open-weight model entirely on your Mac's GPU/CPU. No API calls, no account, no cost, fully offline at inference time.

What is Yukkai Local?

Yukkai Local is a completely different way to use AI:

• Fully offline — once the model is downloaded, nothing about your conversation leaves your device.
• Fully free — no per-token charge, no Wallet/credit involvement at all.
• Fully private — your data never touches a cloud server.
• On-device inference — the model runs on your Mac's GPU/CPU using Apple's MLX framework.

But it comes with significant limitations — this is the part users get surprised by.

Available Models

The model picker features a curated shortlist of ~8 recommended open-weight models, each tagged with:

• Parameter count — model size (3B, 4B, 30B, etc.)
• RAM requirement — how much memory it needs to run
• Download size — how much disk space it needs

Featured Models (Examples)

Model	RAM Required	Download Size	Notes
Llama 3.2 3B	~4 GB	~2 GB	Fast, light, good for simple tasks
Qwen3 4B	~4 GB	~2.5 GB	Strong general-purpose model
Qwen3-Coder-30B	~20 GB	~18 GB	Specialized for code generation
Gemma 4 27B 8-bit	~30 GB	~28 GB	Large, capable, requires lots of RAM

Beyond the Shortlist

The curated list is just a starting point — not the limit. The Local Models settings screen has a free-text field to enter any Hugging Face model id (e.g. mlx-community/DeepSeek-R1-Distill-Qwen-7B-4bit).

Hundreds of MLX-converted models beyond the 8 featured ones are downloadable this way, with a direct link to browse huggingface.co/mlx-community.

Note: None of the curated/featured local models support vision.

Storage, Download & Activation Cost

Only One Model Active at a Time

Only one local model can be active (loaded into memory) at a time — each one occupies several GB of RAM on its own, so switching means unloading the current one first.

First-Time Activation is Slow

Activating a model — especially the first time — can realistically take anywhere up to several tens of minutes:

1. Download — the model downloads from Hugging Face into a local cache (~/.cache/huggingface/hub on macOS, Documents/huggingface/hub on iOS). This requires internet.
2. Load into RAM — several GB are loaded into memory before the model is ready to answer anything.

Set expectations accordingly — this isn't an instant model switch like picking a different cloud model.

Subsequent Activations are Faster

Once downloaded, the model stays on disk permanently. Re-activating it later:

• Skips the download (it's already cached).
• Still needs the RAM-loading step (several seconds to a minute, depending on model size).

RAM Check

The picker checks the device's actual physical RAM and flags/warns on any model that likely won't fit before you commit to a multi-GB download.

Limitations — Read This Carefully

A local model is plain text completion only — a big step down from the main cloud models.

❌ No Tools

No tools at all in this version of the Apple MLX integration:

• No file editing
• No bash execution
• No web browsing
• No swarm/sub-agents
• No desktop automation
• No media generation
• Nothing from the normal ToolRegistry

If you ask a local model to read a file, run a command, or browse the web, it cannot do it — it will just generate text describing what it would do.

❌ No Vision

None of the curated/featured local models support vision. You cannot send images to a local model.

❌ No Streaming

The response comes back as one block, not word-by-word. You'll wait for the full response before seeing anything.

❌ No Media Generation

Image, music, and video generation never run on a local MLX model at all — regardless of which chat model (local or cloud) is active, those features always go through Yukkai's paid Google models. There's no local/offline path for media generation, and picking a local chat model doesn't change that.

⚠️ Explicit Warning

The picker shows an explicit warning before switching to a local model:

"Local models are limited and do not support all Yukkai tools (agents, web search, file editing…). Continue?"

Take that warning at face value — it is accurate.

When to Use Yukkai Local

Use a local model when:

• Privacy is paramount — you're working with sensitive data that cannot leave your device.
• You're offline — no internet connection available (after the initial download).
• Cost is a concern — you want unlimited free inference.
• The task is simple text completion — answering questions, writing text, brainstorming, basic conversation.

Do NOT use a local model when the task needs:

• File access
• Web browsing
• Code execution
• Desktop automation
• Image/music/video generation
• Multi-agent swarm
• Any tool from the ToolRegistry

If the task needs any tool, use a cloud model instead (Fusion, Frontier, Free, or a personal model).

Cost & Privacy

• Fully free — no per-token charge, no Wallet/credit involvement at all.
• Fully offline at inference time — once the model is on disk, nothing about the conversation leaves the device.
• First use requires internet — to download the model from Hugging Face. All later use is fully offline.

Storage Locations

Platform	Cache Location
macOS	~/.cache/huggingface/hub
iOS	Documents/huggingface/hub

Switching to/from Local

You can switch between Local and cloud models at any time from the model picker:

• Cloud → Local — switch to offline inference (expect a delay while the model loads).
• Local → Cloud — switch back to full tool support (instant).

The switch takes effect on your next message.

How It Works Internally

Internally, the conversation loop detects a local:-prefixed model id and routes to a completely separate, much simpler code path than the normal cloud tool-calling loop. There's no fallback or partial tool support — it's all-or-nothing.

Examples

User asks a local model to read a file

User: "Read the file ~/Documents/report.txt and summarize it."

Yukkai (local model active): "I can't read files when a local model is active — local models don't have access to tools like file reading. To do this, switch to a cloud model like Yukkai Fusion or Frontier, which can access your filesystem."

User asks about media generation

User: "Generate an image of a sunset."

Yukkai (local model active): "Image generation doesn't run on local models — it always uses Yukkai's paid Google models, regardless of which chat model is active. The image generation will be billed to your Wallet balance if you have credits."

User asks about RAM requirements

User: "Can my Mac run Gemma 4 27B?"

Yukkai: "Gemma 4 27B 8-bit requires ~30 GB of RAM. Check your Mac's memory in System Settings → General → About. If you have less than 32 GB, this model won't fit — consider a smaller model like Llama 3.2 3B (~4 GB RAM) or Qwen3 4B (~4 GB RAM) instead."

Good to Know

• Local models are best for privacy-sensitive, offline, simple text tasks — not for complex agent workflows.
• The first download is slow — plan for up to several tens of minutes depending on model size and internet speed.
• Only one model at a time — switching means unloading the current model first.
• Media generation is separate — even with a local chat model, image/music/video always uses paid cloud models.
• Hundreds of models are available — the 8 featured ones are just a starting point.

Learn More

• Model Selector — Overview of all model options.
• Yukkai Fusion — Cloud model routing with full tool support.
• Platforms & Requirements — Hardware requirements for local models.