Workflow Engine

Overview

The Workflow Engine lets you build reusable, multi-step pipelines that combine AI prompts, tool calls, agent swarms, conditional logic, human checkpoints, and more — all defined once as a JSON file and executed on demand.

Unlike one-off conversations or ad-hoc agent swarms, a workflow is a persisted, declarative blueprint. You design it, save it, and re-run it whenever you need. The engine handles sequencing, error recovery, and supervision automatically.

What makes it different?

Traditional automation tools (Zapier, n8n, etc.) connect static API endpoints with rigid rules. Yukkai workflows are AI-driven:

• The AI builds the workflow step by step under your control.
• It is stored as a JSON file on your disk for reuse or manual editing.
• The AI launches and monitors execution in real time.
• If a step fails, the AI can retry, skip, modify, or escalate — autonomously or with your input.
• After each run, the AI analyzes execution to improve future runs.

How it works

Stages and sequencing

A workflow is composed of stages, each with an action type. Stages run sequentially based on their dependency order (dependsOn), not in parallel. Parallelism happens inside specific stage types (swarm and dynamicSwarm) where multiple agents fan out concurrently.

The Agent-in-the-Middle (AITM)

An optional supervisor agent can watch over the entire pipeline. When a stage fails or produces unexpected results, the AITM can:

• Retry the stage automatically
• Skip it and move on
• Escalate to you for a decision

This keeps workflows resilient without requiring manual intervention for every hiccup.

Storage

Workflow files live in ~/YukkaiDocs/workflows/ as JSON files. Each file is self-contained and can be version-controlled, shared, or hand-edited.

Scale limits

A single workflow can orchestrate up to 1,000 agents total (for example, 10 swarms of 100 agents each). Dynamic swarms are capped at 50 concurrent agents and 1,000 total items.

How to use it

You interact with the Workflow Engine through five actions:

Action	What it does	Key parameters
Run	Execute a workflow from start to finish	workflow_name (required), inputs (optional key-value object)
Validate	Check a workflow file for errors before running	workflow_name (required) — always validate before the first run of a new or edited file
List	Show all available workflow files with stage counts and supervisor flags	None
Status	Check progress of a running or completed workflow; surfaces pending human questions if paused	None
Stop	Gracefully stop a running workflow — the in-flight stage finishes first	None

Stage action types

Each stage in a workflow uses one of eight action types:

Type	Purpose
Prompt	Run an AI prompt — a single LLM task or instruction
Swarm	Fan out multiple agents in parallel to tackle a task from different angles
Dynamic Swarm	Like a swarm, but the number of agents or items is determined at runtime
Tool Call	Execute a specific tool (e.g., web search, file write, API call)
Sub-Workflow	Invoke another workflow as a nested stage
Human in the Loop	Pause execution and wait for your approval or input before continuing
Wait	Delay for a fixed duration or until a condition is met
Conditional Branch	Route to different stages based on a simple condition (true/false, equals/not-equals, non-empty-is-truthy)

Resuming paused workflows

When a workflow hits a Human in the Loop stage, it pauses. You can check status to see the pending question, then answer via the ask-user mechanism and re-run the workflow to resume from where it stopped.

Examples

Daily research digest

A workflow that gathers news on a topic, summarizes findings with a swarm of agents, branches based on sentiment, and sends you the result:

1. Tool Call — Fetch latest articles on a keyword
2. Swarm — Five agents each summarize a different article
3. Conditional Branch — If overall sentiment is negative, route to a deeper analysis prompt
4. Prompt — Compose a final digest from all summaries
5. Tool Call — Send the digest to your Telegram

Content approval pipeline

1. Prompt — Draft a blog post from an outline
2. Human in the Loop — Pause for your review and edits
3. Prompt — Polish the approved draft
4. Tool Call — Publish to your CMS

Multi-source data aggregation

1. Sub-Workflow — Run a separate "scrape LinkedIn" workflow
2. Sub-Workflow — Run a separate "scrape GitHub" workflow
3. Swarm — Cross-reference and deduplicate results across sources
4. Prompt — Generate a consolidated report
5. Tool Call — Save the report to disk

Tips

• Always validate before the first run. The validate action catches schema errors and missing dependencies before execution, saving you from mid-pipeline failures.
• Use Human in the Loop for high-stakes decisions. Pause the workflow before irreversible actions (publishing, sending emails, deleting files) so you stay in control.
• Keep stages small and focused. Each stage should do one thing well. Complex logic is easier to debug when split across multiple stages with clear conditions.
• Leverage sub-workflows for reuse. If you find yourself repeating the same sequence across workflows, extract it into its own workflow file and call it with Sub-Workflow.
• Let the AITM handle transient failures. Enable the supervisor for workflows that depend on flaky external APIs — it will retry automatically so you don't have to babysit.
• Review after each run. The AI analyzes execution post-run to identify bottlenecks. Pay attention to which stages fail or take longest, and iterate.
• Name workflows clearly. Since you reference them by filename (without the .json extension), descriptive names like daily-news-digest make management easier.
• Use conditional branches for adaptive logic. Instead of building separate workflows for different scenarios, branch within a single workflow based on intermediate results.
• Mind the agent budget. With a 1,000-agent cap per workflow, plan swarm sizes carefully. Large dynamic swarms (50 concurrent) are powerful but consume resources fast.