The 'Build it from Scratch' Discipline in the Age of Skills

Andrej Karpathy's most influential teaching product is a video titled "Let's build GPT: from scratch, in code, spelled out." It runs for two hours. It starts from an empty Python file. By the end, you have a working language model and a detailed understanding of every decision that produced it.

The Claude Code skills ecosystem is structurally opposed to this pedagogy. Skills are designed to accelerate: you install one, you get capability, you move on. The entire value proposition is "don't build this yourself." The karpathy-guidelines skill is no exception — install it, get better coding behavior, never read the internals unless you want to.

The tension here is real, worth taking seriously, and ultimately not a contradiction. But understanding where it resolves and where it doesn't is useful for anyone thinking about how to work with AI tools without losing the engineering judgment that makes them worth using.

What the Pedagogy Claims

Karpathy's "build from scratch" approach is not just an aesthetic preference. It's a specific claim about the conditions under which deep understanding forms.

The claim is: understanding that comes from building is qualitatively different from understanding that comes from use. If you've implemented a transformer from scratch, you know why the attention heads are shaped the way they are, what goes wrong when you change the normalization order, and what the training curve looks like when you've made a specific kind of error. If you've only used a transformer via an API, you have a black-box understanding that works until it doesn't.

Black-box understanding fails in specific ways. When things go wrong — and in production systems, things go wrong — you need the structural understanding to diagnose failures that the documentation doesn't cover. The user of the library finds the bug in the hours or days it takes to search Stack Overflow and open a GitHub issue. The person who built a version of the library from scratch finds the bug in minutes.

This is why nanoGPT is 300 lines, not 3,000. The implementation has to be small enough to hold in your head completely. If you can hold the whole thing, you can reason about any part of it.

What Skills Optimize For

Skills optimize for a different goal: maximum output per unit of developer time. The right skill turns a 3-hour task into a 15-minute one. For capabilities you'll use repeatedly and understand at the conceptual level, the time savings compound dramatically.

The karpathy-guidelines skill is a clean example of a skill that complements engineering judgment rather than replacing it. It doesn't make architectural decisions. It doesn't generate code on your behalf. It shapes the behavioral defaults of an AI assistant in ways that align with what a careful engineer would want — but the engineering judgment is still yours.

This is the design pattern that resolves the build-from-scratch tension most cleanly: skills that encode behavioral constraints or known-good patterns (where understanding is already present) versus skills that generate implementations (where the black-box question is live).

Where the Tension Is Real

The tension becomes live when you install a skill that does something you don't fully understand and trust the output without verification. This is vibe coding at the skill level — the skill handles the implementation, you accept the result, you don't deeply read what it produced.

For simple tasks in domains you know well, this is fine. The risk is calibrated by your understanding of the domain. You can catch obvious errors because you know what correct looks like.

The risk compounds when you're in a domain you don't understand deeply and you're using skills to accelerate past the learning phase. The skills work, the output looks plausible, and you proceed. Until something fails and you don't have the structural understanding to diagnose it.

Karpathy's pedagogy is most relevant here: the time spent understanding something from scratch is an insurance premium against future debugging costs. The premium is real; so is the claim on your time.

The Practical Resolution

The practical resolution isn't "always build from scratch" or "always install and trust" — it's a classification problem.

For foundational infrastructure that will run in production and that you'll need to debug under pressure: build from scratch, or at minimum read the source until you understand it. Use skills to scaffold, not to own.

For capabilities that are auxiliary to your core work, that you'll use regularly, and that fail gracefully if they fail at all: install the skill, use it, update it when you notice it producing bad output. The karpathy-guidelines skill falls into this category — it shapes behavior at the session level, its failures are visible and recoverable, and you don't need to understand its internals to use it well.

For capabilities you're building for the first time and want to understand: Karpathy's "from scratch" approach is still the best answer. nanoGPT exists precisely because the library wrappers hide enough that users miss important concepts. The cost of the from-scratch path is time; the return is structural understanding that survives for years.

The Skills Ecosystem's Pedagogical Gap

There's a genuine gap in the skills ecosystem: it has strong tools for accelerating experienced practitioners and weak tools for building the understanding that makes practitioners experienced.

A skill that teaches you how something works — not a skill that does something for you, but one that walks you through building it — would be a different kind of artifact. Karpathy's video lectures are that artifact. They don't have direct equivalents in the Claude Code skill format.

This isn't a criticism of the ecosystem — it's a description of what it currently optimizes for. The "from scratch" pedagogy lives on YouTube and in public GitHub repositories, not in installable skill bundles. For now, the two approaches are complementary rather than integrated.

For the broader argument about how Karpathy's pedagogical approach connects to the structure of the karpathy-guidelines skill, see From nanoGPT to Claude Skills: The Pedagogy of Karpathy's Public Code.

---

Part of the Karpathy on Claude Code series. Published 2026-05-23.