Why a Markdown File Beats a Message Bus

You have five AI coding agents. They need to know what to work on, what's already taken, and what's done. How do you coordinate them?

The popular answer is a message bus. Agents publish and subscribe. They negotiate tasks, share context, broadcast status. It's the architecture you'd find in CrewAI, AutoGen, or any framework with "multi-agent" in the tagline.

I tried that approach. Then I replaced it with a directory of markdown files — kanban-driven task dispatch for AI agents, using the filesystem instead of a broker. It's been six months, and I haven't looked back.

The O(n²) problem with message buses

When agents coordinate through messages, every agent potentially talks to every other agent. Agent A finishes a task and broadcasts "task 27 done." Agents B, C, D, and E all receive it. Agent B claims the next task and broadcasts "I'm taking task 28." Now everyone else needs to hear that, update their state, and avoid claiming the same task.

With 5 agents, that's manageable. With 10, it's 90 potential message pairs. With 20, it's 380. The communication overhead grows as O(n²), and every message is a chance for race conditions, stale state, or lost updates.

Worse: you can't see what's happening. The coordination state lives in flight — in message queues, in-memory buffers, agent context windows. When something goes wrong, you're debugging invisible state.

The O(1) alternative: read a file

Here's how Batty dispatches tasks instead. Every task is a markdown file in a directory:

.batty/board/tasks/
├── 027-add-jwt-auth.md          # status: in-progress, claimed_by: eng-1
├── 028-user-registration.md     # status: todo
├── 029-add-rate-limiting.md     # status: backlog
└── 030-fix-dashboard-css.md     # status: done

Each file has YAML frontmatter for machine-readable fields and a markdown body for the task description:

id: 28
title: User registration endpoint
status: todo
priority: high
depends_on: [27]
claimed_by:
tags: [api, auth]

# User registration endpoint

Add POST /api/register with email validation,
password hashing, and duplicate detection.

## Done when

- Endpoint returns 201 with user object
- Duplicate email returns 409
- Tests cover happy path and validation errors

An agent doesn't subscribe to a topic or negotiate with peers. It reads a file. One file, one read, one task. O(1).

How kanban-driven dispatch actually works

Batty's daemon runs a polling loop — every 10 seconds, it reads the board and makes decisions:

1. Scan the task directory
2. Find idle agents (no active task)
3. For each idle agent, find the highest-priority task that is:
   - status: backlog or todo
   - not claimed by anyone
   - not blocked
   - dependencies resolved (all depends_on tasks are done)
4. Update the task file: status → in-progress, claimed_by → eng-1
5. Launch the agent with the task context

That's the entire dispatch algorithm. Priority sorting is deterministic: critical tasks dispatch before high, high before medium. Ties break by task ID, so the oldest unblocked task wins. The board is always consistent because the daemon updates the file before launching the agent — if the launch fails, the task stays claimed and the daemon retries next cycle.

No message broker. No pub/sub. No consensus protocol. The filesystem is the coordination layer, and grep is your monitoring tool:

# What's in progress right now?
grep -rl "status: in-progress" .batty/board/tasks/

# Who's working on what?
grep -rn "claimed_by:" .batty/board/tasks/*.md

# How many tasks in each status?
grep -rh "^status:" .batty/board/tasks/ | sort | uniq -c

Try doing that with a message bus.

Why agents understand markdown natively

This is the insight that makes the whole approach work: LLMs already know markdown. It's the dominant format in their training data — README files, GitHub issues, documentation, Stack Overflow posts. When you hand an AI coding agent a markdown task file, it reads the title, parses the acceptance criteria, and starts working. No serialization format to teach it. No API client to configure.

Compare this to handing an agent a message from a coordination bus:

{"type": "task_assignment", "payload": {"id": 28, "title": "User registration endpoint", "priority": "high", "context": {"depends_on": [27], "tags": ["api", "auth"]}, "description": "Add POST /api/register..."}}

The agent can parse this, but the format carries no information about the task. It's overhead. The markdown version is the task description — the agent reads it the same way a human developer would read a ticket.

What happens when things go wrong

Message buses need sophisticated error handling. What if a message is lost? What if two agents claim the same task? What if an agent crashes mid-task?

With file-based dispatch, the answers are simple:

Lost updates: Can't happen. The file is on disk. If the daemon crashes mid-write, the file is either updated or it isn't. On restart, the daemon reads the board and picks up where it left off.

Double claims: The daemon is the only writer for dispatch operations. It claims the task (updates claimed_by in the file) before launching the agent. If the launch fails, the claim is already on the board — the daemon retries or escalates. No two agents race for the same task.

Crashed agents: Every poll cycle, the daemon reconciles. If an agent is idle but has an in-progress task, the daemon re-assigns it. If a task is claimed by an agent that no longer exists, it gets unclaimed and returned to the queue. Orphaned state is impossible because the board is always the source of truth.

Dependency violations: Before dispatching task 28, the daemon checks that all tasks in depends_on: [27] have status: done. If task 27 is still in progress, task 28 stays in the queue. No message ordering to worry about — just a field comparison.

What humans can do that message buses can't

Your kanban board is a directory of text files. This means:

Reprioritize on the fly. Open 029-add-rate-limiting.md, change priority: medium to priority: critical. Next dispatch cycle, it jumps the queue. No API call, no admin panel, no "drag the card."

Add context mid-task. An agent is working on task 28, and you realize it needs additional context. Edit the markdown file — add a note, clarify an acceptance criterion, paste a code snippet. The agent reads the updated file on its next reference.

Debug with cat. When something goes wrong at 11pm, the difference between "open a file" and "connect to a monitoring dashboard and reconstruct message flow" is the difference between fixing the problem and going to bed.

Block a task instantly. Add blocked: "waiting on API key from vendor" to the frontmatter. The daemon skips it on every dispatch cycle until you remove the field. No "pause" button to find, no workflow to trigger.

Version-control everything. git log -- .batty/board/tasks/ shows every task creation, status change, and priority shift. git diff shows exactly what changed. git blame shows who changed it. Your project management history is in the same repo as your code, with the same tools.

When this doesn't work

Honest limitations:

Real-time collaboration. If you need agents to share intermediate results — "I just changed the API schema, everyone update your types" — file polling with a 10-second interval isn't fast enough. You need something push-based.

High agent counts. At 50+ agents, scanning a directory of task files every 10 seconds starts to matter. Batty is built for teams of 3-10 agents, not swarms.

Cross-project coordination. If agents span multiple repositories or machines, a shared filesystem isn't available. You'd need a networked coordination layer.

For the typical use case — a developer running 3-8 AI coding agents on a single project — a directory of markdown files handles dispatch better than any message bus I've tried. It's simpler to operate, simpler to debug, and agents read it as naturally as you read a README.

Try it

cargo install batty-cli
batty init
batty up

Define tasks as markdown files. Batty dispatches them to your agents, gates on tests, and moves them through the board. No message bus required.

How does your multi-agent setup handle task coordination? I'm curious whether anyone else has landed on file-based approaches — or if there's a message bus setup that's actually simple to debug.

Links: GitHub | Demo | Docs