adapt your codebase and environment for agents
focus on designing the environment where ai agents can do reliable work
when ai gives you bad code or makes a wrong assumption, the question is never "how do i fix this?"
the question is "what's missing that would prevent ai from making this mistake next time?"
you have 2 main levers to do this:
- static code checks + feedback loop (how can the agent validate the result of its work?)
- the right context/instructions (does the agent know what you know?)
lever 1 - static code checks + feedback loop
llms are good at looping until they meet specific goals. the better the guardrails you give them, the better the result will be
do you have static code checks that are so opinionated that a coding agent will always produce code that is exactly at the level your senior engineers would produce?
do you have tests that will fail when ai slop has been introduced?
does the agent have all the tools an engineer has to validate its work?
- let the agent:
- run static code analysis. examples, mostly from the js world:
- oxfmt/prettier https://oxc.rs/docs/guide/usage/formatter
- oxlint/eslint https://oxc.rs/docs/guide/usage/linter + custom oxlint/eslint rules
- ts
- dead code detector https://github.com/webpro-nl/knip
- copy/paste detector https://github.com/kucherenko/jscpd
- github actions checker https://github.com/rhysd/actionlint
- vulnerability scanner https://github.com/google/osv-scanner
- sql linter https://github.com/sqlfluff/sqlfluff
- fancy js linter https://github.com/fallow-rs/fallow
- fancy react linter https://github.com/millionco/react-doctor
- oxlint rules https://github.com/dmmulroy/anti-slop
- shadcn linter https://github.com/shadcn-ui/lint
- build/compile/run the application locally, grep the logs
- deploy to a test environment
- interact with the application in the browser (or desktop/mobile app), take screenshots, record videos. i highly recommend https://agent-browser.dev
- query the db, logs, analytics
- dry run db migrations against a copy of prod db
- tighten the static code analysis rules. there’s really no limit here
- you don’t need to refactor the whole codebase. just baseline the current errors and come back later. i strongly prefer code comment baselines if the tool supports them. oxlint example:
// oxlint-disable-next-line typescript/no-deprecated -- baseline. why it’s good:
- it only ignores 1 line
- it requires you to name the rule ignored
- it requires you to name the reason
- the agent reading the code understands that this is not desired
- the tool errors if the rule violation has been fixed but the comment has been left, so it’s super easy to maintain
- i strongly recommend avoiding setting any rules as warnings. error or nothing. people, and even more so agents, don’t care about warnings. they only care when ci is red
- add strong type safety, static + runtime type checking (could be the hardest item on the list)
- let the agent work until everything is green. only then will the code be worth looking at
you can start with this prompt:
analyse the last 1000 pr comments left by humans in this repository. group the comments by recurring issue and put each group into 1 of 3 buckets:
- we can add a deterministic check in ci for it. we can use things like formatters, linters, custom checkers, or dedicated packages. this is the best option and should be considered first. we will baseline all of the current violations of this rule, if any, to avoid large refactors
- we can add clear agent instructions like
AGENTS.md or skills
- neither
for every group, include:
- number of prs
- roi of applying your suggestion (0-10)
divide the work across 5 sub agents and double-check their findings
lever 2 - the right context/instructions
AGENTS.md
this is the standard recognised by most agents. just place this file in the root of your repository and it will be included in the system prompt
start with an AGENTS.md that’s 10-50 lines long, then commit it