Running a small company of AI agents: the boring infrastructure that makes it trustworthy

Most AI tooling assumes you’re sitting in front of it. That’s fine for a chat. It’s a poor fit for work that has to happen every week whether or not you’re at your desk: the usage report, the research digest, the timesheet.

If you want agents to behave like colleagues rather than toys, they need what colleagues have. Someone who assigns the work, tools to do it, written instructions, a record of what they did and a person who answers for the result. This is the setup I’ve settled on, and where it still falls short.

I run the same pattern in two places, and the contrast is the interesting part. One host is on a corporate account, doing back-office work such as reports and timesheets. The other runs my own business, Run with Mort, a running-coaching practice where the agents work with real client records through tools I built for them. Same architecture, very different risk.

The org chart, in three layers

1. The orchestrator is management. Paperclip holds the schedules, the agents that run each job and an issue for every run. That gives me a history of what ran, what it said and whether it succeeded, without building any of it myself. It is also the audit trail: I can open any run and see what an agent did and when.

2. The image is the toolbox. Everything an agent might need is baked into one Docker image, once:

Claude, for synthesis and reasoning work.
Chromium and the OS libraries it needs, for jobs that have to drive a web UI behind single sign-on, such as downloading a report or submitting a form that has no API.
The Snowflake Cortex Code CLI, so an agent can work against warehouse data. A small script turns bound secrets into a connection when an agent starts. I haven’t wired an agent to it yet, so treat that as the next module, not a result.
A run wrapper that tells the orchestrator when a job finished cleanly.

3. Modules are the job descriptions. Each automation is its own repo with its own tests. A modules.yaml file pins every module to an exact commit SHA, and the image build pulls those commits and nothing else. Adding a job means adding a line and rebuilding. The Dockerfile doesn’t change.

Why the split works

Tools and logic change at different speeds. Browsers and CLIs change when their vendors say so. A job’s logic changes when your needs do, say when a holiday calendar needs a new entry. Separate layers mean a script fix doesn’t touch the toolchain, and a toolchain bump doesn’t touch the scripts.

Pinning makes production boring. What runs is exactly what I reviewed and merged. If something misbehaves, I know the commit. The alternative is cloning each module’s latest default branch on every build, which means a push you forgot about can ship to production at 3am, running as root during install. Pinning turns every change into a deliberate, reviewable act.

Secrets stay out of the code. Agents get credentials through references resolved at launch. Modules never contain them, so a module repo is safe to share or review.

State has an explicit home. Anything a job must remember, such as login sessions, run history and config, lives on a persistent volume. Anything baked into the image is replaced on every build. A volume mounted over a path hides whatever the image put there, so I keep code under /opt and state on the volume. Getting this wrong fails silently, which makes it the worst kind of bug.

A coaching business as a place for agents to work

The corporate host mostly runs scripts that an agent supervises. The other host is closer to hiring staff. Run with Mort has its own MCP server, a service that exposes the business as a set of tools an agent can call. The coaching agent on the host connects to it over OAuth, with the client credentials supplied to the container as secrets, so no credentials live in a module or a prompt.

Through those tools an agent can work with the things a coach’s assistant would:

Athletes and plans. Profiles, preferences, notes, training plans and the planned workouts for each week.
Real training data. Activities and samples synced from the athletes’ watches, plus fitness estimates and compliance reports that compare what was planned with what was done.
Coaching memory. Notes, coaching events and summaries of each training block, so context survives from one week to the next instead of living in my head.
The business side. Communications history, survey responses, pricing and invoices.

The tool list is also a risk register. Reading an activity or a compliance report is harmless. Writing a plan or a note is reversible. Sending an email to an athlete, creating an invoice or voiding one reaches a real person and a real payment, and those are the actions that get a gate, a review, or no autonomy at all. Designing the tool surface is the same job as designing a role: you decide what this colleague may see, what they may change and what they must ask about.

Two things follow from handling client data:

Where the data lives matters. The athletes are real people, so backups go to a US bucket, encrypted before they leave the server, with the key kept offline.
The organization’s rules decide the auth. On this host the agent runs on my own subscription and uses a long-lived token, because I set the policy. On the corporate host the policy forbids that. Same software, different answer, and the pattern has to accommodate both.

Because the business is mine, it is also the safest place to push on autonomy. A mistake here costs me an apology and an hour, not a compliance review.

Who is accountable?

Right now: me, and only me. I’m the board, the manager and the on-call engineer. That is worth saying plainly, because “zero-human company” is more aspiration than description of what’s running today.

What that means in practice:

Every agent has a human owner. With one person that’s trivial, but the question matters the moment a second person joins. An unowned agent is the one that quietly keeps doing the wrong thing.
Autonomy is tiered by consequence. Reading data and drafting a digest run unattended. Anything that changes a system of record, such as submitting a timesheet, goes through a gate. Single sign-on pushes a prompt to my phone on every run, and the job waits. I treat that as the security model working, not as friction.
I don’t rely on the platform alone for approvals. Paperclip describes board-style approval for structural changes such as adding an agent, and I treat that as a useful layer. For the actions that matter, my gates are the sign-on prompt and my own review of each run’s issue. I’d want to see how far the platform’s approvals go before leaning on them with more than one person involved.
Credentials are scoped and short-lived by policy. This runs on a corporate account where long-lived tokens aren’t allowed. The practical cost is that the Claude login has to be renewed by a person rather than set once and forgotten, and a scheduled run fails until that happens. I’d rather have that failure than a year-long token sitting in an env file.

What it costs

Money. There are two lines: the server that hosts everything, and the model usage the agents consume. Both are small next to the hours the automations replace, but the second one is the one that can surprise you. Budget is a first-class concept in the orchestrator, since each agent can be given a spending limit, so a runaway loop should hit a ceiling rather than produce a surprise bill. Set the limits before you schedule anything, and check the cost view the same way you’d check a team’s expenses.
Deploys are slow. Immutable image tags mean a rebuild per change, and a swap takes the server down for about eight minutes while the load balancer drains.
Browser jobs are the fragile ones. A headless browser on a server starts slower than on a laptop, and a check that works locally can fail there. A “is this button visible?” test answers instantly instead of waiting, so the fix is an explicit wait. Browsers also use real memory, and my first container was too small for one.
Some steps stay human. See above. The sign-on prompt and the login renewal are the price of staying inside corporate policy.

When it goes wrong

Agents fail in ordinary ways, so the useful question is how fast you find out and how cheaply you undo it.

You need a real check. A dry run proves the script ran. Only looking at the actual system proves it did the right thing. In one case a saved screenshot showed a row the target system had never stored, because the last unsaved row is lost when the browser closes. The run reported success. It wasn’t.
Make health checks tell the truth. On a Paperclip host I found that the obvious /health URL serves the web app’s shell and returns 200 even when the backend is broken. The deploy script polls /api/health instead. A green check that can’t go red is decoration.
Roll back without rebuilding. The previous image is pulled from the registry and swapped in; if the new version fails its health check, the old one goes back automatically. The app container is stateless, so this is safe. The database is not rolled back, which is why I treat schema migrations as the one change that needs a human to think first.
Back up the key, not just the data. Nightly encrypted offsite backups include the database and the secrets master key. Without the key, the secrets in the database can’t be decrypted, so a database-only backup doesn’t restore. I don’t trust a backup until I’ve rehearsed restoring it.
Get told, don’t go looking. Backups and deploys post a short success or failure message to my phone. If nothing arrives, that is itself a signal.

What I’d tell someone starting out

Decide the boundaries first. What belongs in the image, what belongs in a module, what belongs on the volume. Most of my problems came from getting one of those wrong.
Name the accountable human for every agent, even if it’s you.
Gate by consequence, not by habit. Let reads run free and make writes earn their approval. Check the gates still bite, because approvals that are always granted are not oversight.
Keep a record of every run. If you can’t answer “what did it do on Tuesday?”, you don’t have a colleague, you have a cron job.
Treat the first success as unproven until you’ve looked at the real system.

On the corporate host, three automations run: a weekly usage report, a weekly research digest and a timesheet submission that knows about company holidays. On the Run with Mort host, an agent works inside the coaching business through its own MCP tools. None of it runs on a laptop, and each automation is a short module rather than a deployment project. A new automation is a repo, a pin and a schedule. The next ones I’d try pair a model for judgment with a warehouse for the data, using the same host and the same rules.