I made the setup steps a lookup so a model can't invent a command
Building the tool setup engine for the Riskiest Page Builder, and checking every block against vendor docs.
Learn · Oct 28, 2026
I almost let a model write the setup instructions for my page builder. Then I thought about what a wrong install command does to someone's first hour.
The Riskiest Page Builder helps you pick a stack and hands you a PRD. The next piece was the part that tells you how to get each tool running. CLI install, login, the command that connects an MCP server, the env var names, the check that proves it worked.
That's the exact content a model gets slightly wrong. A flag that got renamed. A login step that moved. It reads fine and fails on your machine.
Write the prompt before the build step
Before any of this, I wrote the prompt for the step. Not after. Before. It's a habit I've been pushing lately, including in the PRD for The Automated CMO, where every feature gets its own Claude Code prompt in build order. The reason is simple. Gaps in your thinking show up on paper instead of in your codebase. For this layer, writing the prompt forced one question I'd been dodging. Who is allowed to be the source of truth for an install command? The answer was never the model. Once that was on the page, the file structure almost designed itself. It's slower for about ten minutes. Then it pays that time back, because the build step stops being a conversation and starts being a checklist.
Put the setup facts in a file, not in a prompt
The first decision was where the facts live. I didn't want a model producing install steps at request time. So the setup blocks became plain data in one module, lib/tool-setup.ts. Each tool gets the same shape. How to install the CLI and log in. The command that connects its MCP server. Notes for the connector. The names of the env vars it needs. And one verify step. Same shape every time is the point. A reader learns the layout once. And I can scan the file for a missing field instead of rereading prose.
Check every block against the vendor's own docs
This is the slow part. Each block was checked against the vendor's documentation on 2026-10-26, and the date is written into the work. I'm not claiming the docs can't change. They can, and they will. A date on the check tells the next person how old the claim is. That's more honest than a block that just looks confident. It also makes the file reviewable. Stale means a date I can see, not a feeling.
Turn the stack into setup blocks with one function
Then a function, getSetupBlocks(stack), takes the tools someone ticked and returns the matching blocks. Nothing is generated. It's a lookup. A lookup can't invent a command that doesn't exist. If a tool isn't in the file, it isn't in the output. I'd rather show less than show something I never verified. That's the same instinct from the guardrails work on this app, where the renderer only draws content that already passed validation. It also keeps the cost of being wrong small. One bad block is one bad block. It can't spread into ten confident paragraphs I'd have to audit.
Render the blocks as markdown
The last piece of the engine is a markdown renderer. The blocks have to end up somewhere a person will copy from, and a PRD is markdown. Keeping rendering separate from the data mattered here. The data file only holds facts. The renderer only holds layout. When I fix a command, I touch one place. When I change the layout, I touch the other.
Make the UI point at what changed
The next deploy was small, and it was a usability fix. When someone ticks tools, the page now scrolls to the PRD stack section and highlights it. Before that, ticking a tool changed something off screen. The feature worked and nobody could see it work. That's a bug even when the code is right. I test this kind of thing the dull way. Open the live page. Tick a tool. Ask whether a stranger would notice anything happened.
What broke: Nothing crashed, but one thing was wrong. The tool engine worked and the page didn't show it. Ticking a tool updated the PRD stack section below the fold, so it looked like the click did nothing. The data was right and the experience was broken. I'm glad I clicked through it like a visitor before moving on.
The fix: A follow-up change scrolls to the PRD stack section and highlights it whenever tools are ticked. It's a small fix. But it changed whether the feature felt real. Here's the thing. The harder decision was upstream. Writing the setup facts down as checked data, with a date, meant I never had to trust a model to remember a command. Going into the next layer, the rule is simple: if a wrong answer costs the reader an hour, it's a lookup, not a generation.
I'm building a tool setup module for my app. Don't generate install steps from memory.
Step 1. Create one TypeScript data file. Each tool is an object with the same fields: cliInstall, cliLogin, mcpConnect, connectorNotes, envVarNames, verifyStep, and checkedOn (a date string).
Step 2. Leave any field you can't confirm from the vendor's official docs as TODO. Do not guess a command or a flag.
Step 3. Write getSetupBlocks(stack) that takes a list of tool names and returns only the matching objects. If a tool isn't in the file, return nothing for it.
Step 4. Write a separate function that renders the blocks as markdown. Keep layout out of the data file.
Step 5. List every TODO you left and which doc page I should check for it.Get new guides and builds in your inbox
No noise. New artifacts, skills, and lessons when they drop — and early access when agents launch.
Want a full structured path instead of one-off guides?
That's Learn AI with AI — a sequenced course that takes you from zero to building real things, with community and accountability built in.
Check out Learn AI with AI →