Featured Work — Factory

A self-referential spec-to-code engine / personal AI control plane. Specs in, reviewed PRs out, with an inspectable substrate of goals, specs, decisions, and accumulated context underneath. Factory builds itself.

• Multi-contour review loop: generation → cross-model review → human review → analyst verdict, with review comments injected back into the next agent round until the change is green. The same fractal structure recurs at step, goal, and release levels.
• Evidence-bound validation: an analyst role returns DONE / NOT_DONE / INCONCLUSIVE with file-level evidence rather than plausible-sounding claims; a negative-control discipline guards against the system flattering itself.
• Goal-coverage analysis: raises “is this done?” from a single spec to a whole goal, checking each described feature against the code with evidence.
• Engineering: ~133k lines of TypeScript across three modules (UI, orchestrator on Cloudflare Workers, agent runner), with ~1,900 automated tests, 880 commits since April and 63 D1 migrations. Notably, the git history includes commits authored by Factory’s own agents alongside mine: the system genuinely participates in building itself. A hardening pass added a test harness for git-mutation paths, a CI gate, and shared secret-scrubbing across every log/error boundary.
• Specification-driven by discipline, not slogan: a separate specs repository holds 404 specs across 61 goals (~49k lines) — roughly one line of specification for every three lines of code. Intent is negotiated and written down before code is generated; code is a renderable view of that substrate.
• Model routing & harness: multi-provider by design — Claude Code, Codex, and local models (Ollama/OpenCode) orchestrated by role, task shape, and cost, with graceful provider fallback. Latest release ships dynamic per-runner model discovery (capabilities advertised at claim time, model/role/locality-aware lease matching) and a local-model reviewer/analyst via direct Ollama with enforced verdicts — expensive frontier models used only where the task demands it.
• Ingest pipeline: a Telegram drop-box (webhook → task → finished note, hardened with dedup, byte caps, fail-closed auth) and Whisper transcription with CUDA→CPU fallback feed external signals into the same substrate; a post-review materiality judge gates review-round convergence.

Related writing at codecot.com/notes: The Understanding Debt of AI-Generated Software, The Future Is Not Autonomous Execution, and The Scarcity of Shared Intent in the Age of AI.

Applied AI/ML in Python (current): building framework-free AI services on my own corpus — semantic search over embeddings; a hand-rolled RAG pipeline (retrieval → grounded generation with source citations, an evidence-bound abstain contract, prompt-injection defence); and an evaluation harness: deterministic metrics over a controlled synthetic corpus (ground truth by construction), an LLM-judge trusted only after calibration against a deterministic baseline, and A/B comparison of retrieval configs. Dependency-injected tests throughout; deliberately built without LangChain first, to own the mechanics end to end.

The Practice — Other Directions

Not a shelf of tools. Each of these started from a question about what should be done and how the answer holds when it has to scale — the technology underneath is replaceable, and usually gets replaced. Three concerns keep meeting: computation whose figures can be checked, media that ships on a rhythm, and measurement that decides what ships next. A daily report computed from ephemerides and a risk commentary computed from market data are the same machine with different inputs.

• Applied-AI lab on AWS — supervised autonomy for content with numbers in it: the model drafts and must declare every figure it used; a deterministic gate with no model in it recomputes each one from source and rejects anything that is not there; a human approves out of band before publication. A second module applies the same contract to risk and volatility analytics. Python, Step Functions, Bedrock, Lambda, S3.
• Scheduled analytics-to-media pipeline — periodic reports generated from a deterministic engine and delivered as text, charts, synthesised speech and rendered video, in four languages, with per-segment subscriber delivery. Speech is verified against its own source text before release. ~14.5k LOC Python, 600 tests, 70+ unattended releases. The shape transfers to any time-varying source: market data, quotes, news feeds.
• Voice media system — multi-platform (server, desktop, Android): streaming transcription on a consumer GPU, a text-to-speech router across cloud and self-hosted engines, cloned voices, an accumulating library, memory across sessions, and rooms where several models and several people share one conversation with per-participant profiles. ~9.5k LOC, running entirely on owned hardware — a working answer to what stays local when the data is personal.
• Hiring copilot with privacy as its architecture — CV and dossier analysis, interviewer support, ad-hoc tests with automatic scoring, and a designed integrity layer over a remote interview's audio and video. Two planes that never merge: candidate data lives on the operator's host and the public surface carries only candidate-independent content, so there is no route by which personal data could leave.
• Channel analytics joined to publishing control — statistics across channels on the same surface as the schedule, which is the only arrangement in which the evidence changes the rhythm.
• Applied research on multi-agent behaviour — a small lab for the questions the systems raise, run with predictions registered before the data, a chance baseline beside every number, and nulls reported as intervals. Its published study: agents with no private information voted with the crowd 78.9% of the time against a 58.4% chance baseline; four prompt-level interventions moved nothing (best −0.9 pts, p=0.73); frontier models made individual play flawless and left group conformity unchanged (+1.7 pts, p=0.62). Capability fixed the players and left the room exactly as it was.

Fuller write-ups — problem, what runs, what is designed next — at codecot.com/projects.

Skills