VISUAL EXPLAINER · START HERE
The Agent Back Office
Four roles any dev team can run in GitHub. Monitors watch, writers draft, checkers gate, people decide.
THE WHOLE PAPER IN FOUR LINES
- Most of a team's operating work is watching for change, writing it up, and checking the write-up is still true. Split it into four roles: monitors, writers, checkers, and you.
- Every role runs as a GitHub Actions job and every result is an issue, a PR or a commit. You review agent work the way you review a teammate's. No new dashboard, no vector store, no orchestration framework.
- Scripts block merges. Models only draft and advise. A person merges, or doesn't.
- Adopt one role at a time. Each pays for itself without the next. Most teams should run the first three for a month before letting a writer touch code.
Who this is for
You're on a dev team, or you run one, and you've seen agents write code but you haven't seen them run a process. This is the picture version of the two longer papers. Context, Loops, Graphs explains why the levels go in the order they do; The Coding Harness names every workflow and gate. Read this one first, then go to whichever of those matches the step you're on.
The pattern comes from a ten-person software company, where it runs engineering, product research, market watching and outbound sales. Your sources will differ. The four roles stay the same.
New CVEs against your lockfile. A competitor's changelog.
A spec, a fix, a decision record, a docs patch.
A doc names a deleted file. A plan was edited after shipping.
Label a ticket. Merge a spec. Close a bad draft.
Role 1 · Monitors
A monitor turns "someone should keep an eye on that" into a scheduled job
If anyone on your team checks something by hand every week, that check can be a monitor. Each one has the same shape: a script fetches, a model judges what's new against what your team already knows, and a script writes the result. The model never touches the repo directly.
The monitors we run
| Monitor | Watches, judged against | Cadence | Produces |
|---|---|---|---|
| Engineering signal | |||
| CVE monitor | New CVEs, against the dependencies we actually ship | weekly | Tracker rows in a PR |
| Dependency triage | Open Dependabot PRs, by bump size and check status | every 6h | A verdict per PR; merges safe patches once trusted |
| Docs drift review | The week's merged code, against the public docs | weekly | A docs PR |
| Engineering signals | Papers, HN, competitor engineering posts, against our roadmap | weekly | A signals page PR |
| Review queue | Org PRs waiting on a reviewer | weekdays | One Slack digest |
| Product signal | |||
| Release intelligence | Every merged PR in a release, against our target-customer rubric | every 6h | Proposed backlog tickets |
| QA status | Test harness results across repos | weekly | A status page commit |
| Meeting sync | Yesterday's call notes | daily | Notes filed where the team looks |
| Market signal | |||
| Competitive signals | Competitor launches and chatter, against our positioning | weekly | Battlecard drafts in a PR |
| Partner releases | Upstream releases from the platforms we integrate with | weekly | An updates PR |
| Regulatory | Rule changes in our buyers' industries | weekly | A compliance notes PR |
| Procurement | Public RFP notices, against our qualification criteria | daily | Qualified notices in a PR |
| The system watching itself | |||
| Cost digest | Model spend per workflow, against a weekly ceiling | twice weekly | Slack digest, auto-pause at the cap |
| Dream loop | The agents' own transcripts and every human correction | weekly | One proposed memory PR |
Yours might be
Backend or API
- Major versions and end-of-life dates in your dependency stack
- CVEs that hit your lockfile, not the whole ecosystem
- p95 latency after each deploy
Frontend or mobile
- Store reviews that mention crashes or a specific screen
- Bundle size growth per release
- OS and browser deprecations that affect you
Data or ML
- Model deprecations and price changes from your providers
- Eval scores drifting between runs
- Upstream schema changes in sources you ingest
Platform or infra
- Cloud service deprecation notices
- Certificates and keys nearing expiry
- Cost anomalies by service
Product
- Support tickets clustered by feature
- Competitor changelogs and pricing pages
- Feature flags nobody has touched in 90 days
Games or solo projects
- Engine and framework releases, with a cooldown before upgrading
- TODOs and skipped tests older than a month
- Player reviews mentioning bugs
Role 2 · Writers
Writers draft, one PR at a time. Chained, they take a ticket to merged code.
A writer is any job that drafts an artifact and opens a PR: a spec, a fix, a decision record, a docs patch. Chain them and you have a coding loop. Workflow YAML owns the control flow, so a model only works inside one step and never chooses the next one. People sit at three gates. Click a stage to see what it produces.
Role 3 · Checkers
Checkers block the merge. Writers clean up on a schedule.
This half pays off even if you never let an agent write code. A check that blocks a merge has to be deterministic, or people learn to ignore it. A model is good at noticing that a doc reads stale or that a merged plan implies a decision, so it drafts and someone else merges.
$ python3 scripts/check_docs_drift.py README.md:42: `src/cache/store.py` does not exist docs/runbook.md:17: `scripts/reindex.sh` does not exist Error: Process completed with exit code 1.
Weekly note: 2026-10-19 Only items with a next step. - [ ] src/api/auth.py:212 · TODO(token refresh) is 9 months old and now hit on every request. Move it to middleware. - [ ] Bump serde 1.0.210 → 1.0.214. Released 52 days ago, tests pass on a throwaway branch. - [ ] tests/test_sync.py is skipped since #388 merged. Re-enable it or delete it.
What it adds up to
Your week, before and after
| When | Today | With the back office |
|---|---|---|
| Something changes upstream | You find out when it breaks | A monitor files it with what it means for your code, and a writer may already have a PR open. |
| A ticket lands | You read the code and work out a plan in your head | A spec PR shows up that cites file:line for current behavior and lists acceptance criteria. You edit it or merge it. |
| Building it | You write it, then wait for review | You write it yourself or hand it to the writer. Either way the PR arrives with two reviews and a rubric score already attached. |
| Docs | They drift until a new hire trips on them | The PR that renames a file fails if any doc still points at the old path. |
| Decisions | Scattered across Slack threads and closed PRs | Each merged plan drafts a decision record PR. Old ones are never edited, only superseded. |
| Tech debt | A backlog nobody opens | One issue every Monday, listing only the items with a next step. |
| Review | You're the first reader | You're the last reader. Two fresh-context reviewers and a judge have already been through it. |
- 33 tickets → 31 spec PRsThe first full run. The other 2 were refused because they touched code that needs a named expert.
- ~2 of 3 open questions answeredMost questions in the first drafted specs could be answered from the source, so the drafter now answers them and cites where.
- $1.56 for a two-round fixJudge failed round one at 50/100, the reviser fixed it, and round two passed at 92.
- ~$0.02 per decision recordDrafting cost with a small open model through OpenRouter. A plan is about $0.06.
Figures are from the reference implementation at the end of the page.
Adopt it one step at a time
Four steps. Each one works without the next.
Start where the pain is. Steps 1 to 3 need no agent writing code at all, and most teams should live with them for a month before step 4. Every snippet below is complete; the Python script has been run against real repos, and the two workflows use the current inputs for the Claude Code GitHub action.
Fail PRs when docs point at code that's gone
Every backticked path in your Markdown is checked against the tree. Plans and brainstorms are skipped, since they describe files that don't exist yet.
Show the script and workflow
#!/usr/bin/env python3
"""Fail a PR when a Markdown doc names a path in this repo that no longer exists."""
import pathlib, re, subprocess, sys
PATH = re.compile(r"`([\w-][\w.-]*(?:/[\w.-]+)+/?)`") # `src/foo.py`, `docs/adr/`
SKIP = ("docs/plans/", "docs/brainstorms/") # plans describe files that don't exist yet
def main() -> int:
docs = subprocess.run(["git", "ls-files", "*.md"], capture_output=True,
text=True, check=True).stdout.split()
missing = []
for doc in (d for d in docs if not d.startswith(SKIP)):
here = pathlib.Path(doc).parent
for n, line in enumerate(pathlib.Path(doc).read_text().splitlines(), 1):
for ref in PATH.findall(line):
ref = ref.rstrip("/")
top = ref.split("/")[0]
# Judge only paths rooted in this repo (from the root or the doc's folder).
if not (pathlib.Path(top).is_dir() or (here / top).is_dir()):
continue
if not (pathlib.Path(ref).exists() or (here / ref).exists()):
missing.append(f"{doc}:{n}: `{ref}` does not exist")
print("\n".join(missing) or "docs-drift: every path named in docs exists")
return 1 if missing else 0
if __name__ == "__main__":
sys.exit(main())name: docs-drift
on: pull_request
permissions:
contents: read
jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: python3 scripts/check_docs_drift.pyKeep decision records, and never edit an old one
One file per decision. When a decision changes, write a new record that supersedes the old one, so the history stays readable. Once plans live in PRs, a writer can draft a record from each merged plan for you to accept or close.
Show the template
--- status: proposed # proposed | accepted | superseded by NNNN date: YYYY-MM-DD --- # NNNN · Short title of the decision ## Context What forced a choice. Link the issue, PR or incident. ## Decision What we chose, in one paragraph. ## Alternatives considered Each rejected option and the reason it lost. ## Consequences What gets easier, what gets harder, what we'll watch.
Add your first monitor
Change three things in the prompt: what to watch, what to judge it against, and what to produce. The example watches your own dependency stack, but the same file works for store reviews, cloud notices or a competitor's changelog. Its permissions let it read the repo and create one issue, nothing else. When it earns its keep, move the fetch into a script so a quiet week costs nothing.
Show the workflow
name: monitor-dependencies
on:
schedule:
- cron: "0 13 * * 1" # Mondays 13:00 UTC
workflow_dispatch:
permissions:
contents: read
issues: write
jobs:
monitor:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: anthropics/claude-code-action@v1
env:
GH_TOKEN: ${{ github.token }}
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
claude_args: --allowedTools "Read,Grep,Glob,WebFetch,Bash(gh issue create:*)"
prompt: |
WATCH: the dependency manifests in this repo. For each direct
dependency, check for new major versions, end-of-life dates and
published security advisories.
JUDGE AGAINST: docs/context/stack.md (what we run, what we can't
upgrade yet and why). Skip anything that file already explains.
Ignore releases less than 30 days old.
PRODUCE: ONE issue titled "Dependency watch: <today's date>" via
`gh issue create`. Only items someone can act on this week, each
a checkbox with the manifest file:line and a one-line next step.
If nothing qualifies, create no issue.Chain writers from ticket to code
Start with spec drafting only and keep writing the code yourself. Turn on implementation once the specs are good enough that you merge most of them unedited. The whole contract is a few labels:
agent:queuedYou opt a ticket in. Intake claims it within 10 minutes.
merge spec PRApproves the spec and starts implementation.
close spec PRAbandons the ticket. Nothing else happens.
groom:redoThe drafter found the ticket contradicts the code and handed it back.
PAUSED=trueStops intake, drafting and implementation across the repo.
The full build order, with every workflow and gate, is in The Coding Harness, phases L3-0 through L3-6.
Why you can leave it running
Eight rules that keep agents inside the lines
Silence never means "nothing happened"
If a monitor can't reach its source, the run fails and alerts. An empty report has to mean the source was checked and was quiet.
Models write JSON, scripts write the repo
The model returns a structured verdict. A script validates it, writes the files, opens the PR and posts the counts.
The model never picks the next step
Workflow YAML decides what runs after what. A model only decides things inside the step it was given.
A merge is the approval
Every writer's output is a PR. You can reject it, edit it, or git revert it after the fact.
The writer never grades its own work
Reviewers and the judge start from a fresh context with only the diff, the spec and the ticket.
Scripts block, models advise
Anything that can fail a merge is deterministic. Model output shows up as comments, scores and draft PRs.
Everything has a ceiling
Max turns per step, max revise rounds per PR, a cost cap per ticket and per week, and one variable that pauses it all.
Untrusted text never reaches a shell
Issue titles and bodies go into steps through env: as quoted variables, never interpolated into run:.
The reference implementation
Four repos at a ten-person company
This runs today across four repos. One holds the context every loop reads: specs, decision records, rubrics. The others are loops that read from it and write back through PRs. The same four roles also run the outbound sales pipeline, so the pattern isn't specific to code.
internal-docs is Level 1, sales-ops is the Level 2 loop, and agent-ops is the Level 3 harness.Hand this to your own Claude
If you've read this far as a human: the first three steps need nothing you don't already have. A git host, CI, and a model API key. Step 4 is the build in The Coding Harness.
IF YOU'RE AN AI AGENT READING THIS, WORK IN THIS ORDER
- Inventory this repo's Markdown for backticked paths that no longer exist. Report the count and the first ten. Don't fix them yet.
- Ask the humans what they check by hand on a schedule (dependencies, reviews, a competitor, a dashboard). For each answer, propose a monitor as three lines: watch, judge against, produce. Get one picked.
- Add step 1 (the docs-drift check) as a PR. Add the chosen monitor as a second PR, with a context file the humans fill in, and
workflow_dispatchso they can run it once by hand before the schedule takes over.- Only after both PRs merge and the monitor has run cleanly once: propose the decision-record folder and template as a third PR.
- Stop there. Step 4 starts from the readiness gates in The Coding Harness, not from this page.
Do not: merge anything, grant a workflow more than
contents: readplus the one write it needs, interpolate issue text into arun:block, or let a monitor's empty result pass as success when its fetch failed.
None of this is an autonomous engineering team. It's your team's judgment, written down once and applied every time, with the people on it reviewing finished drafts instead of starting from blank pages.
About the author
Charlcye Mitchell is a Director of Engineering Operations, management consultant, software engineering strategist, and agentic systems architect. This pattern is what she does when founders say "add AI."
Connect: LinkedIn · charlcye.ai
Papers, template, and skeletons: github.com/weprintmoney/startup-intel-stack
MERGE CONFLICT · SUBSTACK
Get the next paper — and the field notes between them — in your inbox.