RAG — Vector Knowledge Base

Yukkai's RAG (Retrieval-Augmented Generation) tool provides a vector knowledge base with semantic search. Unlike keyword search, semantic search finds content by meaning — you can search "how do I handle errors" and find a document titled "Error Handling Best Practices" even if the exact words don't match.

How It Works

1. Ingestion: Source content is split into chunks (~512 tokens, 64-token overlap) → embedded into vectors → stored in SQLite.
2. Search: Your query is embedded → compared against all stored vectors using cosine similarity → top-K results returned.

---

Actions

Action	Description	Key Parameters
search	Semantic search across the knowledge base	query (required), top_k (default 5, max 20), source_type (optional)
index	Index a file or folder from disk	path (required), source_type (optional)
index_text	Index raw text directly — no file needed	content (required), title (optional), source_type (optional)
delete	Remove a source by ID	source_id (required, use "all" to purge everything)
reindex	Purge and rebuild Yukkai's internal knowledge base	—
status	Database statistics (source count, chunk count, size, embedding model)	—
sources	List all indexed sources with their IDs	—

---

Source Types

The source_type parameter scopes searches and categorizes indexed content:

Source Type	Description
yukkai_knowledge_agent	Technical/operational docs about how Yukkai's tools and features work. Used by the agent for its own reasoning.
yukkai_knowledge_user	Simple, product-level docs safe to relay verbatim to end users. Use when answering "what can Yukkai do?" questions.
skill	Knowledge files from activated skills.
project	Indexed project files.
document	Standalone or pasted content.
rule	User-given standing rules.

Scoping Searches

• For the agent's own reasoning (how a tool works, internal docs): use source_type: "yukkai_knowledge_agent".
• For end-user questions about Yukkai capabilities: use source_type: "yukkai_knowledge_user". This prevents leaking technical detail.
• General "have we discussed this before?" queries: omit source_type to search everything.

---

Examples

Search the Knowledge Base

// Agent recalling how a feature works
rag(action: "search", query: "how does the swarm tool decompose tasks", source_type: "yukkai_knowledge_agent")

// Answering a user question about capabilities
rag(action: "search", query: "can Yukkai browse the web for me", source_type: "yukkai_knowledge_user")

// General search across everything
rag(action: "search", query: "API authentication")

Index Content

// Index a file
rag(action: "index", path: "~/Documents/spec.pdf")

// Index a folder
rag(action: "index", path: "~/Documents/GIT/MyProject/docs")

// Index raw text the user pasted in chat
rag(action: "index_text", content: "<pasted spec text>", title: "Q3 API spec")

Manage Sources

// List all indexed sources
rag(action: "sources")

// Delete a specific source
rag(action: "delete", source_id: "abc123")

// Purge and rebuild Yukkai's internal knowledge base
rag(action: "reindex")

// Check database stats
rag(action: "status")

---

Rules

• Use sources before delete if you don't already know the exact source_id — deleting the wrong source is not reversible.
• Use reindex (not delete(source_id: "all")) when refreshing Yukkai's own internal knowledge base — it purges and rebuilds in one step.
• top_k is capped at 20. Request a narrower query instead of relying on a larger result count.
• Always pass source_type when the query is about Yukkai itself, to avoid retrieving content from the wrong pool.

---

Don'ts

• ❌ Searching without source_type when answering an end user's product question — you may retrieve agent-pool content and leak technical detail.
• ❌ Using the agent pool's content verbatim in a reply meant for the end user — translate it into simpler, user-facing language.
• ❌ Calling delete(source_id: "all") to "start fresh" on Yukkai's own docs — use reindex instead.
• ❌ Assuming a project file is already indexed — check sources or status before relying on RAG search to surface it.