Project Brief: Building a Pipeline of Private Development Projects and Their Sponsors

Summary

“Tremendous” isn’t a very technical word. So here’s how I’ll ‘technify’ it. In my first HOUR (as of writing this, I’m >24 hours running), I used >250 Million tokens at a cost of only $8.50. As of today, after guarding several redundant processes that were re-reading documents, I have continued the process at an additional expenditure of only ~25 million tokens, which is roughly where I imagine I’ll land daily going forward. Keep in mind, these are mostly Jev tokens, not trad LLM tokens, so that equates to about $10 / day, of which only $1.05 of this (and the bulk of the compute) is attributed to System 1 scrubbing.

The equivalent task over just this initial period using a traditional (can we say “traditional” yet?) model would have cost upwards of $1400 and taken 2-4 days, running several parallel agents. Extrapolate that to the ~10 days I plan on running this. Hence: “Tremendous”.

What this tackles (maybe I should have started with this)

Every local government publishes new-construction proposals before anything gets built: rezonings, plats, site plans, special exceptions and annexations. Each of these filings names the owner, developer, applicant and consultants. More often than not, this is in unstructured PDF uploads, with handwritten contact information.

My pipeline turns those scattered public records into one database of projects with contacts, with every field traced back to its source, with relevant project data for outreach context. It finds projects at the land-approval stage, months to years before a building permit, and it costs very little per project.

The problem

• Development data is spread across about 22,000 local governments, each with its own meeting system, file formats and habits.
• Commercial data vendors cover permits well but cover early land approvals poorly. They seldom give the sponsor’s direct phone or email.
• Agendas are mostly noise. Even in busy counties, under 10% of items are private development applications.

Approach

1. Choose where to look. This requires work. And thinking. Census Building Permits Survey volumes rank jurisdictions, and work starts where the most gets built. But if we take a macro-down view, we would organize our source research by jurisdiction: Federal agencies (public) / National (private, third-party vendors) → State → County → City.
2. One reader per vendor, not per city. Only a handful of platforms host most public meetings and records: Legistar/Granicus, CivicClerk, ArcGIS REST and Accela-type portals. A reader for one platform covers all its clients, so adding a new city is usually one line of configuration.
3. Cute, cached downloading. Each file is downloaded once and kept. Requests to a site are spaced 2 to 3 seconds apart, robots.txt is followed, and any login or CAPTCHA is logged and skipped.
4. Screening in tiers, cheapest first:
   • A free keyword filter drops procedural items (minutes, bids, appointments, budgets).
   • One short Jev question: “Is this a site-specific private project?”
   • Seven more questions only for items that pass: new construction, government, single home, size floor, project type, application type, relevance.
   • Answers are cached one question per record. Rewording a question re-asks only that question, and only for uncertain records. Changing a cut-off is just arithmetic on stored answers.
5. One record per project. Records are matched on application number, then on normalized address + parcel, then by a Jev same-project check.
6. Pulling contacts out of documents:
   • Text is extracted from PDFs, and local OCR (Tesseract in Python) handles scans and fillable forms (zero token spend).
   • Fixed-layout staff reports are read by plain code at no cost.
   • Everything else, once scrubbed, is extracted by a language model, and a value is kept only if it appears word for word on the cited page.
   • Paid reading happens only after a ~14-document test shows the jurisdiction’s files actually contain contact details.

Quality controls

• Every field links to its source URL and PDF page. Names, phones and emails are never guessed.
• After each batch, a sample of what Jev passed and rejected is spot-checked and scored against an answer key (431 items, about 96–97% agreement).
• Each batch’s spot-check has turned up roughly 0.2–4% missed projects.
• A newer dated value replaces an older one and the change is logged. A filled field is never overwritten with a blank.
• Spending guard: every model job prints its estimated tokens first and stops for approval above 5 million.

Economics

• Screening: about 2 to 4 million tokens per large jurisdiction, roughly $0.10–0.20 on Jev. National screening comes to tens of dollars.
• Contact extraction: this is the main cost, and the test-first rule controls it.
• Labor: the main cost of adding coverage is mapping the sites that have no readable feed.

Results so far: Wave 1, Florida

~24 hours of project (a total of apx. 8 hours of actual running)

• About 8,200 new-construction projects and more than 4,200 contacts across about 20 cities and counties.
• Contact yield varies a lot by place. Hollywood had a phone or email for 81% of projects (147 of 181). Hillsborough’s test found phones for 4 of 14 developers and no emails.

Limits

• Browser-only or script-blocking sites need hand downloads or custom work.
• Agendas name projects but often not phones or emails. Those are mostly in attached applications and exhibits.
• Reading is throttled per site, which slows deep document collection but not coverage across many jurisdictions at once.

Path to national coverage

Coverage grows fastest by adding more clients of platforms already mapped.

New vendors and browser-only sites each need their own one-time effort. Kept fresh with regular re-pulls, the database becomes a running national feed of early-stage projects with named contacts behind each one.

Well. Anyways. That’s how I got your uncle’s email address.