The Karpathy Playbook #3: Eureka Labs

Part 3 of 6 — tracing Karpathy's public thinking with FRE|Nxt Labs production commentary.

---

The announcement

"⚡️ Excited to share that I am starting an AI+Education company called Eureka Labs. We are Eureka Labs and we are building a new kind of school that is AI native. How can we approach an ideal experience for learning something new?"

— @karpathy, 16 July 2024

Five months after leaving OpenAI, his pitch: teachers still design course materials, but an AI teaching assistant guides each student through them. First product: LLM101n, an undergraduate course where students train their own LLM.

---

What we heard

Eureka Labs is the most concrete signal Karpathy has given about what he thinks "AI native" actually means as a product pattern — and it's the opposite of what most people build.

Most "AI-powered" products are a thin wrapper: take a human-designed experience, bolt on a chatbot. Eureka's model is the reverse. A domain expert designs the content once. The AI's job is to be the teacher — ask the student questions, diagnose what they don't understand, re-explain in their vocabulary, pace the session. The human designs the thing being taught. The AI designs the teaching.

That split — expert designs the artifact, AI personalizes the delivery — is a genuinely new product pattern. It's what separates an AI-native product from an AI-feature bolted onto a SaaS tool.

---

What we actually do with this

When we scope an AI product with a client, the first question we ask now is: which half are you building?

The artifact (built once, by a domain expert):

• Course content, legal templates, medical protocols, interview rubrics, compliance checklists
• This is deterministic, reviewable, versioned. No AI.

The teacher (built once, by us, powered by an LLM):

• Personalization, pacing, follow-up questions, diagnostic probes, re-explanations
• This is probabilistic, evaluated via samples, tuned continuously.

Clients usually show up with only half the spec. They'll have a brilliant artifact and no idea how to deliver it (just "put a chatbot on it"). Or they'll have a chatbot and nothing principled for it to teach. The Eureka frame forces both halves to exist before we write code.

---

Applied: InterviewLM's two halves

InterviewLM is structured exactly this way:

Artifact half: The interview rubrics, job-role competency frameworks, and evaluation criteria. Designed by hiring managers and psychometric experts. Stored as structured data, versioned in Git, reviewable by humans.

Teacher half: The interviewer persona agents. They read the rubric and run a conversational assessment — ask, probe, redirect, score. This half is LLM-native, runs on LangGraph, uses prompt caching for the rubric (90%+ cache hits because the rubric is stable across all candidates).

The artifact is updated monthly by human experts. The teacher half is updated weekly by us as we observe failure modes in production traces. The two halves evolve independently.

---

The one thing to steal from this

For your next AI product spec, write two docs, not one:

1. The artifact doc — what the domain expert is contributing
2. The teacher doc — what the LLM is personalizing or pacing

If you can't write the artifact doc, your product isn't ready for AI — it's ready for a designer. If you can't write the teacher doc, you don't have an AI product — you have a content library.

---

Next in this series

#4 — Vibe coding (February 2025). Karpathy's throwaway tweet that named a generation of AI-assisted development — and why "mostly works" is not a production SLA.