Skip to content
← All writings

My 23 Billion Lesson

AUTHOR: RYAN GOOD · August 27, 2026

AI isn't God. And if a $2 Trillion Anthropic IPO estimate gives us any hints, it's creators don't want you to save on tokens.

Read this if you want to cut token usage by up to 75%.

Let me be your guinea pig. I'm about halfway into what has developed into a massive undertaking that I have over a decade of deep domain expertise to conceptualize and very little of the technical skills to actually implement -- hence why I'm vibe coding, Reddit Foruming and YouTubing my way through it like a chimp on a bicycle.

What I'm building: A full CRE all-in-one Underwriting Suite. 23 Billion tokens. 1.3 Million lines of code deep (a real investment, mind you). The result that's unfolding: A full stack application with structure and governance guardrails that stitches both proprietary and open-source tooling with a schema and libraries that would make your CRM blush, AI import capabilities and best-in class IC-friendly analysis and auditing functions across all asset types, scenarios and across various modules and submodules covering operations (historical and proforma), development budgets and draw sizing, markets and comps, and institutional multi-lifecycle and complex stack capital structuring capability. It's cool. It's intuitive. It's user friendly. But it's not done yet.

But I digress. I was WELL past due for a comprehensive audit (PM me if you'd like the script I ran for this, courtesy of hours of my own research). So I ran one on my last 30 days of sessions. Here's what I found that will SAVE YOU MONEY. FYI keeping code and specs off of this so I don't get banned (or hacked), but feel free to slide into my DM's for nerd talk (prompt lines, scripts, system architecture, etc... happy to share):

1. The Lightbulb. AI has ADHD:

If you and I sit down for a coffee and I tell you my problems, my goals and what I'm working on, chances are (if you like me), you'll remember at least parts of that conversation -- and enough context to fill in the blanks the next time we get together.

AI does things differently. When you have that same coffee date with an LLM, it has to record the entire conversation. Each time you talk to that same LLM in a session (your "date"), before it responds, it has to play back the full recording of what you said before. In other words, every turn in a coding session replays the ENTIRE CONVERSATION. Turns out, that's expensive, but there are commands to fix that.

2. Marathon Sessions (the biggest cost):

  • The issue: Auto Mode has a Yin and Yang. While you will never get anything done if you're forced to sit and hit "approve" 80 times in a session, mind the dangers of Auto Mode. And mind your usage of "Ultracode" in Claude or ultracode skills in Codex. You might THINK you're done (because YOU are), but if you have agents and tasks fanned out (as you should be doing -- consciously), your coding agents probably aren't. My audit returned an alarming finding. I ran one session for 21 hours straight AFTER my last prompt. Oops.
  • What it costs: Every single exchange re-reads the entire conversation from the beginning. My average request was 435,000 tokens, and only 38,000 of that was actual setup. The rest was history. Cost grows roughly with the square of session length, so a wasted exchange early gets re-read hundreds of times. The longer the session, the more this can compound.
  • The fixes: Your goals, your architecture, your schemas, your libraries, your rules: Should all be easily accessible in your repo docs for each new session to reference -- and should both grow and get trimmed over time. End sessions at natural boundaries and instruct the gates from the start. Mind "ultracode" (if using Claude Code): this will automatically fan out agents without you asking. This is cool. This also needs to be treated with care. Write a handoff note, check context to make sure you aren't breaking anything that shouldn't be turned off, clear work processes when you hit your gate safely, and start fresh in a new session on the next task. This will save you from data greedy little minions scouring your machine while you're away.

3. Redundant Prompts:

  • The issue: I told agents the same thing more than twenty times in one case ("stop giving me this bad script to run. it doesn't work"). It kept getting it wrong.
  • What it costs: 35 logged incidents of it taking an approach I'd already rejected. One misread instruction cost me >hour to unwind.
  • The fix: Anything you've said twice belongs in a rules file ("put this in the 'constitution' or 'commandments' doc" etc.) the AI reads at the start of every session. A rule stated in chat dies with that chat.

4. No Automatic Checks:

  • The issue: Nothing sat between the AI making a change and me noticing it broke something.
  • What it costs: 70 self-inflicted bugs. My single largest source of friction, and most of them were mechanical: formatting artifacts, broken references, silent failures.
  • The fix: A automated check that you put into your docs that runs after every edit. Mine would have caught most of those 70 in seconds instead of three exchanges later.

5. Hand-Verifications:

  • The issue: I don't accept "it's fixed." I open the browser and check. That instinct was right and it caught real problems. But there's a better way.
  • What it costs: Over 1,000 manual verification round trips, each one re-reading that 435,000-token history. My median response time was five minutes, and a lot of that was me personally auditing claims.
  • The fix: Delegate verification to a separate cheap process that only checks (how YOU would) and never edits. Same rigor, fraction of the cost, and none of it clogs your main session.

6. Expensive Model Use (Despite Thinking Otherwise):

  • The issue: When the AI spins up helper tasks and agents, they inherit whatever model you're using by default unless you set explicit instructions to the contrary.
  • What it costs: I was running a frontier model to do the equivalent of searching for a filename
  • The fix: One line of config per helper task. Cheap model for simple work, expensive model where being wrong really matters. Fable for heavy orchestration. Opus or even Sonnet for the cheaper writes. Think of it like any organizational hierarchy. Fable shouldn't be shoveling coal. It's sitting in the foreman's office. And to save face here, I'll add the caveat that I DID instruct this hierarchy, but only on the tasks and agents I KNEW about. I never explicitly instructed "ALL OTHER AGENTS AND TASKS". Just that simple addition would have saved me significant token cost.

7. Parallel Work in a Shared Space:

  • The issue: In one example, I ran three full sessions at once against the same file.
  • What it costs: A production outage. Broken builds. Duplicated stylesheets.
  • The fix: Isolated workspaces, one per task. Standard practice in software, and I skipped it because I was moving too fast and am admittedly still learning some UI/UX basics.

8. Optimizing the Wrong Thing Entirely:

  • The issue: My instinct on hearing "context is expensive" was to trim my instructions file. Everyone does this. But it's not enough.
  • The measurement: My instructions were 1.4% of the cost. In fact, I should have been ADDING to and UPDATING it. All configuration combined was 5.8%. The actual work product, the code it wrote, was 7.6%.
  • The fix: Trim your sessions, not your instructions.

THE AUDIT MATH

  • 90% of token spend went to re-reading.
  • ~40% of my volume went to correction rather than production.
  • And 40-50% of my usage could also be attributed to bad gate etiquette: not setting the STOPPING place from the start correctly, or leaving AUTO idly running invisible agents that were fanned out by Ultracode automatically.

The estimate for fixing all of it: about a third fewer exchanges to the same output, and 60 to 75% lower cost.

If you're running these tools at any real scale, your instincts about where the waste lives are probably wrong. Mine were. By a lot. I assumed it was in my instructions and my model choice. And it was -- partly. But most of my waste ran on session lengths, fanned out agents acting rogue, and idle times. The efficiency paradox I slapped myself with: I was fanning out agents like a scrum master with a 30-man team, but they went rogue because I didn't give them simple bi-laws to follow ("don't get drunk at the water cooler" would have been a good start). Constitution: Created, but not properly minded. Lessons learned.

Happy to share more technical configs, prompts, skills or creator notes with anyone who wants them. We're all learning this stuff together. Comment or DM.