Karpathy's Autoresearch Tool Lets AI Run Marketing Experiments Overnight
The Development
On March 8, Andrej Karpathy—former Tesla AI lead and OpenAI co-founder—released autoresearch, a 630-line MIT-licensed script on GitHub that automates the scientific method using AI agents. The system assigns an agent a task and compute budget, lets it modify variables, run experiments, measure outcomes, and self-correct without human input. In one overnight run, Karpathy's agent completed 126 experiments and cut model loss by 2.8%. Over two days, 700 autonomous changes produced an 11% efficiency gain on a benchmark he considered already well-optimized. Hyperspace AI CEO Varun Mathur scaled the concept across a peer-to-peer network using the GossipSub protocol—35 agents ran 333 experiments in one night, independently rediscovering ML techniques that took human researchers eight years to formalize. Eric Siu, founder of agency Single Grain, publicly applied the framework to marketing: replace the training script with a landing page or ad creative, measure reply or conversion rate, and let the agent iterate. His projection: 36,500 marketing experiments per year versus the current industry norm of 20–30.
Our Take
The marketing application here is more disruptive than the ML benchmarks. Karpathy's loop doesn't require a research team or a sophisticated mar-tech stack—it requires a defined success metric and an asset to test. That's a low barrier for well-resourced teams and a near-zero barrier for AI-native startups. The real competitive dynamic is that experiment history becomes a proprietary dataset. A brand that has run 36,500 audience-specific tests owns a map of what actually converts—one that no competitor can buy or replicate quickly. What the viral coverage misses is the compounding effect: the winning conditions found on day one seed the hypotheses for day two. This isn't A/B testing at scale; it's evolutionary optimization. Marketing leaders who treat this as a productivity tool are underestimating it. It's an infrastructure decision.
The Exploit
Action Item
If your team runs paid search or email campaigns, define a single conversion metric this week and deploy autoresearch against one asset type—subject lines are the lowest-friction entry point—before a competitor in your category builds six months of experiment history you can't close.