Derek Sean Spratt
Leavenworth, WA (Wenatchee area) · seansp@gmail.com · sean-makes-stuff.com/work · github.com/404seannotfound · linkedin.com/in/sean-spratt-4b00b94
Summary
Engineer who ships applied-AI systems end to end: multi-model evaluation harnesses, judged generation pipelines, and retrieval-and-scoring jobs that run unattended every night. Twenty years in production engineering, including 16 at Microsoft on secure Azure build infrastructure and four at NCC Group leading continuous malware-analysis engagements, where defining the problem and its success criteria was most of the job. Self-taught in modern LLM tooling; built and shipped more than 25 AI products in 2026, all of them public.
Experience
SaaSaaS LLC, Leavenworth, WA — Founder and Applied AI Engineer Feb 2026 – present
- Built Choir, a macOS app and CLI that sends one prompt to 10 providers plus local models (Ollama, any OpenAI-compatible endpoint) and has a judge model score every response; reports latency, tokens, temperature sweeps, thinking vs. non-thinking variants, and n-gram divergence between models.
- Published 23 comparison studies on that harness, including a 21-model peer-review benchmark in which models rate each other and themselves, and a 109-run study across 22 endpoints.
- Built Series Notes, a multi-agent pipeline that turns a whole book series into an illustrated reader's companion: 11 series and 107 novels so far, every page written by one model and verified against the source text by an adversarial fact-checker, about 3,100 painted frames.
- Built Scriptorium, a 10-stage generation pipeline: 18 candidate story arcs fanned out across three frontier models, scored by two judge models on five criteria, every chapter written twice and judged, 100+ illustrations placed by beat, and a multi-voice audiobook, in about 30 minutes end to end.
- Run two unattended nightly pipelines, Dexter's Papers and Dexter's Cyber Security, that ingest the day's AI research papers and security advisories, score each one 0–10 against a published rubric, and publish an illustrated, narrated episode.
- Shipped FestivalIMS, hosted incident management for festivals and regional burns with an offline command center, plus about two dozen other apps on Cloudflare Workers, React, and Swift.
- Use AI coding agents (Claude Code, Windsurf, MCP) as the primary build tool; wrote a three-document method for scoping work so an agent can build it in one pass.
NCC Group — Technical Project Manager, malware analysis and AI security Oct 2021 – Jan 2026
- Led multi-year engagements for enterprise accounts including Google, Meta, and Amazon, built on continuous malware analysis and threat reporting; defined the intake process, the reporting structure, and what a useful finding looked like.
- Coordinated analysts reviewing high-volume malware intake, and turned their findings into behavioral patterns and risk trends.
- Prototyped AI-assisted malware triage and report drafting for the engagement.
- Contributed to AI product security assessments: abuse cases, integration risk, and how a model's outputs reach downstream systems.
- Delivered executive-level reports and client-facing visualizations of trend data and recurring attack techniques.
Microsoft — Senior Software Engineer (SDE/T II to SDE II to Senior SDE) 2005 – Aug 2021
- Final role: built and maintained the Secure Base Image (SBI) for Azure services across public and sovereign clouds, including the ARM64 Linux images.
- Implemented FIPS compliance and CVE remediation in the image build pipelines.
- Designed CI/CD automation in Jenkins and Azure DevOps; built telemetry and visualization tools for cloud infrastructure metrics.
- Earlier, on Microsoft Network Monitor: received an Engineering Excellence Award for network protocol analysis tooling (C/C++).
TriGeo Network Security — System Architect about 2002 – 2005
- System architect at a security information and event management (SIEM) appliance vendor.
Microsoft — Software Engineer in Test, Excel 2000 – 2002
- Joined as an SDE/T intern and converted to full time on the Excel team.
Selected work
| What it shows | Evidence | Link |
|---|---|---|
| Evaluation methods and benchmarks | Choir: judge-model scoring, divergence heatmaps, temperature sweeps; 23 published studies | Choir |
| Agent systems and multi-stage pipelines | Scriptorium: 10 stages, fan-out, two judge panels, human lock-in step | Scriptorium |
| Large source corpora into verified, structured output | Series Notes: 107 novels across 11 series, every page summarized and checked against the text by an adversarial fact-checker | Series Notes |
| Unattended research tracking in production | Dexter's Papers: nightly ingest and 0–10 scoring of the day's arXiv AI papers against a rubric | Dexter's Papers |
| Deploying to production, not just prototypes | FestivalIMS: hosted, offline-capable incident management with paying users | FestivalIMS |
| Working familiarity with AI coding agents | Three-document scoping method; 25+ apps built agent-first in 2026 | Method |
Skills
- AI and ML: LLM evaluation and benchmark design, multi-model judging, RAG and agent pipelines, prompt and pipeline orchestration, diffusion and multimodal models, local inference (Ollama, Hugging Face Transformers, Stable Diffusion WebUI/Forge on local GPU), AI coding agents (Claude Code, Windsurf, MCP)
- Languages: Python, TypeScript/JavaScript, Swift, C/C++, C#, Bash
- Platforms: Azure, Cloudflare Workers/Pages/D1, iOS and macOS, CI/CD (Jenkins, Azure DevOps), headless Chrome and ffmpeg pipelines
- Security: malware analysis and threat reporting, AI product risk and abuse cases, FIPS compliance and CVE remediation
Education and training
- Nucamp, Solo AI Tech Entrepreneur bootcamp, graduated August 2026
- Washington State University, Computer Science coursework toward a BS, through 2000; Marquette University, 1993
Community
Ten years as a Black Rock City Ranger, and ten years on site build and teardown for Burning Man. In 2026, paid by Burning Man Project for IT logistics, the longest stretch on playa yet, and added surveying to the skill set along the way. Established fire performer with a long history in the fire arts community; former member of the Ignition NW board of directors and currently on its Conduct Committee.