Core Concept
- Nodes represent discrete work units (agent calls, API requests, data processing)
- Edges represent dependencies between nodes
- Variables flow between nodes via Jinja2 interpolation
- Execution follows a tick-based scheduling model driven by Pub/Sub
Why Use Workflows?
Automate Complex Processes
Chain multi-step operations without manual intervention
Coordinate Multiple Agents
Have different AI agents work together on the same task
Handle Async Operations
Wait for long-running tasks (web scraping, external APIs)
Reuse & Compose
Build subworkflows and compose them into larger systems
Real-time Monitoring
Track execution progress step-by-step with detailed logs
Data Transformation
Process and enrich data through multiple stages
Workflow vs Single Agent
Single Agent Chat
- Direct response
- Real-time only
- No state persistence between calls
- Linear reasoning
Workflow
- Multi-step orchestration
- Handles async operations
- Persistent state across steps
- Parallel and sequential execution
- Detailed monitoring per step
Key Components
1. Nodes
Individual units of work:- Agent Nodes: Call your agents with specific inputs
- Data Nodes: Process or transform data
- Integration Nodes: Call external APIs or scrape websites
- Container Nodes: Group other nodes (sequential, parallel, loops)
2. Dependencies
Define execution order:3. Variables
Share data between steps:4. Input Schema
Define what your workflow accepts:Workflow Lifecycle
1. Creation
Define your workflow with nodes, dependencies, and parameters2. Execution
Trigger the workflow with input data3. Tracking
Monitor progress in real-time4. Completion
Workflow either:- Succeeds: All nodes completed, output available
- Fails: A node failed, workflow halted
Common Use Cases
Case 1: Market Research
Case 2: Customer Support
Case 3: Content Creation
Case 4: Data Pipeline
Workflow Structure
Execution Model at a Glance
Workflows use tick-based scheduling:- Workflow is triggered via API
- System publishes execution start message to Pub/Sub
- Workflow engine processes one βtickβ (up to 5 nodes)
- Completed nodes trigger dependent nodes
- Process repeats until all nodes complete or a node fails
- Real-time monitoring shows progress at each step
- Asynchronous execution - Long operations donβt block
- Scalability - Multiple workflows execute concurrently
- Observability - Each step is logged and tracked
- Resilience - Failures are isolated to affected branches
Next Steps
- Execution Engine - Understand how workflows run
- Node Types - Learn about available node types
- Dependencies - Define execution order
- Execution Variables - Share data between steps
- Input Variables - Define workflow inputs