Building Jarvis: Multi-Agent System
What this gives you: Your own team of 7 AI specialists, each handling a different part of your business. One manages your LinkedIn. One handles YouTube. One does your accounting. One plans your strategy. They work independently, 24/7, and cost a fraction of hiring real people.
One AI that handles everything becomes slow, expensive, and confused. Seven specialist agents, each knowing exactly what it does, become something genuinely powerful.
The Problem With Single-Agent Systems
Most people build their AI assistant as one big monolith. One Claude instance, one system prompt, one context window that contains everything: your business context, your clients, your content strategy, your financials, your technical knowledge. Everything.
The problem is the context window becomes enormous. Every message costs more because you are paying for thousands of tokens of irrelevant context on every single call. The agent gets confused because it is holding 10 different roles simultaneously. Response quality degrades. And when you want to update how the agent handles, say, LinkedIn content, you are editing a single massive prompt that affects everything.
The better model is specialisation. The same way a real team works. You have a LinkedIn strategist. You have an accountant. You have a YouTube editor. Each one is excellent at their specific domain and has no opinion about things outside it.
Multi-Agent Architecture Overview
The architecture has two layers: a gateway and a set of specialist agents.
The gateway is a lightweight process that runs permanently. It connects to your messaging interface (Telegram, Slack, Discord - whatever you use to talk to your assistant). Its only job is to receive messages and route them to the correct specialist agent. It does not run Claude itself. It just decides who should handle the message.
The specialist agents are invoked on demand. When the gateway routes a message to the LinkedIn agent, it spawns a claude -p process with the LinkedIn system prompt and the relevant context. That process handles the task, responds, and terminates. No idle cost, no permanent memory pressure.
Gateway Pattern: Routing by Topic Thread
Telegram has a feature called Topics - you can create a group with multiple topic threads, each acting like a separate chat. This is the perfect routing mechanism. Each agent gets its own topic. When you post in the LinkedIn topic, the gateway knows to use the LinkedIn agent.
Running Agents with claude -p
Each agent is invoked using the -p (print) flag, which runs Claude in non-interactive mode, outputs the response, and exits. This is what makes the architecture lightweight - agents are processes, not permanent servers.
The 7 Specialist Agents
Each agent is a markdown file that serves as its system prompt. These files define personality, capabilities, tools, and focus area. Here is the structure for each:
General Agent
The catch-all. Handles anything that does not fit a specific topic. Knows about all the other agents and can hand off tasks. Has broad knowledge of the business but is not expert in any single domain.
LinkedIn Agent
Specialises in LinkedIn content strategy. Knows the posting cadence, which content formats work, how to write connection requests, how to analyse profile views and engagement. Has access to the content calendar and past post performance.
YouTube Agent
Scripts, titles, thumbnails, and analytics. Knows the channel's existing content, audience demographics, best-performing formats, and the content pipeline. Can pull YouTube Analytics data when given API access.
Newsletter Agent
Manages the Beehiiv newsletter. Writes editions, plans content calendar, tracks open rates and click rates, suggests repurposing opportunities. Knows the audience segment and content preferences.
Accountant Agent
The most powerful one for business owners. Pulls live data from Stripe (revenue, MRR, churn), Xero (expenses, invoices), and presents clean financial summaries. Can flag unusual expenses, compare month-on-month performance, and project revenue.
Strategy Agent
Long-horizon thinking. Reviews decisions, analyses options, suggests pivots. Has access to the full business context, historical performance, and can be prompted with specific strategic questions. This one benefits most from a longer context window.
Shared Context vs Agent-Specific Context
Not all context belongs in every agent. Here is the split:
Shared context (injected into every agent):
- MEMORY.md - core facts about the business, tech stack, preferences
- business-context.md - revenue model, ICP, current clients
- formatting-rules.md - content tone, style rules, what to avoid
Agent-specific context (only in the relevant agent):
- LinkedIn: post archive, platform algorithms, content calendar
- Accountant: chart of accounts, subscription list, client billing schedule
- YouTube: channel analytics, script templates, thumbnail style guide
Keeping these separate reduces token cost and keeps each agent focused. The LinkedIn agent does not need to know how Stripe webhooks work.
Skills Directory: Reusable Workflows
Some tasks follow the same multi-step process every time. These become skills - markdown files that describe the workflow in enough detail that any agent can execute them.
When an agent needs to execute a skill, the gateway loads the skill file and includes it in the context:
Data Helpers: Bash Scripts for Live Data
Agents are only useful if they have current data. Static context files go stale. The solution is bash scripts that pull live data from APIs at the moment the agent needs it.
The agent runner calls these scripts before invoking Claude, then includes the output as part of the context:
Cron Automations
The best agents are not reactive - they are proactive. Set up cron jobs for tasks that should happen on a schedule, without anyone asking:
Cost Comparison
This is why the architecture matters from a business perspective.
| Single Agent | Multi-Agent (Specialist) | |
|---|---|---|
| System prompt | ~8,000 tokens | ~2,000 tokens |
| Per message | ~12,000 tokens | ~4,000 tokens |
| Cost per message | ~$0.15 | ~$0.03 |
| Monthly (100 msgs/day) | ~$450 | ~$90 |
That is roughly 60% cheaper per message, at higher quality because each agent is focused on its domain. At 50 messages per day, the savings compound to hundreds of dollars per month.
How to Add a New Agent
Adding a new specialist is straightforward:
- Create the agent system prompt at
/home/jarvis/agents/new-agent.md - Add a new Telegram topic in your group and note its thread ID
- Add the topic ID to route mapping in
gateway/router.ts - If the agent needs live data, create a bash script in
/home/jarvis/data/ - Update shared context (MEMORY.md) if new facts are relevant to multiple agents
- Restart the gateway:
pm2 restart jarvis-gateway - Test by posting a message in the new Telegram topic