71.5x Fewer Tokens: How Karpathy's LLM Wiki + Obsidian Changed My Claude Code Workflow

A step-by-step guide to the 5-layer context stack that turns your amnesiac AI into a compounding knowledge partner.

Last month I measured my Claude Code token usage: 20,000 tokens per session — before I even asked a question.

Claude was re-reading every file. Every session. From scratch.

After implementing the stack I’m about to show you, that number dropped to 280 tokens per session. A 71.5x reduction.

Here’s exactly what I did, with every command.

---

The Problem Nobody Talks About

Every time you open a new chat with Claude, GPT, or any AI coding agent, it starts from absolute zero. It doesn’t know your codebase. It doesn’t know your conventions. It doesn’t remember what it decided last Thursday.

$ new session started
→ reading file 1/37...
→ reading file 2/37...
→ 20,000 tokens gone. haven't asked anything yet.

You paste 50 files “for context.” Half are irrelevant. The context window fills up. The critical instructions get buried. And when Claude doesn’t have context, it doesn’t say “I don’t know” — it invents a confident, plausible, wrong answer.

This isn’t a capability failure. It’s a memory architecture failure.

The numbers back this up:

• ~40% of AI coding errors are context failures, not capability failures

• O(n²) attention cost — longer context does NOT mean better reasoning. Every extra token makes the important tokens harder to attend to

• Zero knowledge persists between sessions by default

---

Karpathy’s Mental Model: Your AI Is an Operating System

Andrej Karpathy — the Stanford PhD who trained GPT-2, built Tesla Autopilot, and coined the term “vibe coding” — explained this perfectly.

The LLM is not a chatbot. It’s an operating system

.
Just as a 1970s computer was constrained by physical RAM and had to page to disk, your AI agent is constrained by context size and must intelligently page knowledge in and out.

The workflow IS the memory management strategy.

“The LLM is a powerful but amnesiac reasoner operating in a finite, expensive, attention-weighted window. Your job is not to write better prompts — it is to be a better memory architect.” — Andrej Karpathy

The Caveman Compression Principle

Karpathy’s key insight: don’t give the AI everything. Give it the minimum that carries the maximum signal.

Like a military intelligence officer briefing a general — not the full intelligence file, just the decision-relevant extract. One example teaches more than a paragraph of instructions.

---

4 Types of AI Memory (You’re Only Using One)

Karpathy identified a taxonomy that explains 80% of AI frustrations:

1. In-Context Memory (Working RAM) — VOLATILE

Tokens currently loaded in the conversation. Fast, expensive, completely wiped at session end. 95% of people think this is all there is.

In Claude Code: your current conversation + any files you’ve loaded.

2. External Storage (Hard Drive) — PERSISTENT

Files, vector databases, notes. Persistent, but must be explicitly loaded each session. This is where Obsidian, Acontext, and your docs live.

In Claude Code: summaries/, docs/, .claude/features/, and Acontext skill files.

3. In-Weights Memory (Instinct) — GENERAL ONLY

Baked into the model at training time. Claude “knows” Python and design patterns — but knows nothing about YOUR project. Can’t be changed without retraining.

4. KV Cache (Short-term Recall) — FREE PREFIX

If your CLAUDE.md file never changes, its attention states get cached and reused for free. Keep it lean and stable — it’s literally free context.

The insight: Most agent failures are memory architecture failures, not capability failures. The fix is rarely a better prompt. It’s loading the right memory layer.

---

Token Efficiency: The Main Skill in LLM Engineering

Before we get to the tools, internalize these five principles. They work regardless of what stack you use:

Applying just these rules cuts token usage by ~60% immediately. But we can go much further.

---

Karpathy’s LLM Wiki: The Architecture That Replaces RAG

On April 3, 2026, Karpathy posted a GitHub Gist with the full architecture of his personal LLM Wiki. 5,000+ stars in a week. It’s elegant in its simplicity.

Most people’s experience with LLMs and documents looks like RAG: upload files, retrieve chunks at query time, generate an answer. It works, but the LLM is rediscovering knowledge from scratch on every question. There’s no accumulation. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

Karpathy’s idea is different. Instead of retrieving from raw documents at query time, the LLM incrementally builds and maintains a persistent wiki — a structured, interlinked collection of markdown files that sits between you and the raw sources. The knowledge is compiled once and then kept current, not re-derived on every query.

The wiki is a persistent, compounding artifact. The cross-references are already there. The contradictions have already been flagged. The synthesis already reflects everything you’ve read.

The Three-Layer Architecture

Layer 1: Raw Sources — your curated collection of source documents. Articles, papers, images, data files. These are immutable — the LLM reads from them but never modifies them. This is your source of truth.

Layer 2: The Wiki — a directory of LLM-generated markdown files. Summaries, entity pages, concept pages, comparisons, an overview, a synthesis. The LLM owns this layer entirely. It creates pages, updates them when new sources arrive, maintains cross-references, and keeps everything consistent. You read it; the LLM writes it.

Layer 3: The Schema — a configuration document (CLAUDE.md for Claude Code, AGENTS.md for Codex) that tells the LLM how the wiki is structured, what conventions to follow, and what workflows to run when ingesting sources, answering questions, or maintaining the wiki. This is what makes the LLM a disciplined wiki maintainer rather than a generic chatbot. You and the LLM co-evolve this over time.

$ ls karpathy-wiki/
→ raw/      ← PDFs, notes, screenshots (never modified by LLM)
→ wiki/     ← LLM-compiled articles with backlinks
→ schema    ← CLAUDE.md / AGENTS.md (the config file)

The 3-Operation Cycle

The wiki runs on three operations that form a continuous loop:

1. INGEST — Drop a new file in raw/. The LLM reads it, discusses key takeaways with you, writes a summary page, updates the index, updates relevant entity and concept pages across the wiki, and appends an entry to the log. A single source might touch 10-15 wiki pages.

2. QUERY — Read the wiki index. Navigate to relevant pages. Synthesize an answer with citations. The LLM acts as a librarian, not a search engine. No vector search. No embeddings. The critical insight from Karpathy: good answers can be filed back into the wiki as new pages. A comparison you asked for, an analysis, a connection you discovered — these shouldn’t disappear into chat history. Your explorations compound just like ingested sources do.

3. LINT — Periodically scan for: contradictions between pages, stale claims that newer sources have superseded, orphan pages with no inbound links, important concepts mentioned but lacking their own page, missing cross-references, data gaps that could be filled with a web search. The wiki self-heals.

Two Navigation Files

Two special files help the LLM navigate the wiki as it grows:

index.md is content-oriented. A catalog of every page with a link, a one-line summary, and metadata. Organized by category. The LLM updates it on every ingest. When answering a query, the LLM reads the index first, then drills into relevant pages. This works surprisingly well at moderate scale (~100 sources, ~hundreds of pages) and avoids the need for embedding infrastructure.

log.md is chronological. An append-only record of what happened and when — ingests, queries, lint passes. A useful tip from the gist: if each entry starts with a consistent prefix (e.g. ## [2026-04-02] ingest | Article Title), the log becomes parseable with grep. The log gives you a timeline of the wiki’s evolution.

Why This Beats RAG

RAG re-discovers knowledge on every query. You embed documents, search for similar chunks, stuff them into context. Every time.

The wiki compounds. Every question makes the knowledge base richer. And the numbers are dramatic:

71.5x fewer tokens. No vector database. No embeddings. Just markdown + LLM.

---

Where This Applies (Not Just Code)

Karpathy’s gist emphasizes that this pattern is domain-agnostic:

• Personal: tracking goals, health, psychology — filing journal entries, articles, podcast notes, building a structured picture of yourself over time

• Research: going deep on a topic over weeks — reading papers and incrementally building a wiki with an evolving thesis

• Reading a book: filing each chapter, building pages for characters, themes, plot threads. Think Tolkien Gateway — thousands of interlinked pages. You could build that personally as you read, with the LLM doing the cross-referencing

• Business/team: an internal wiki fed by Slack threads, meeting transcripts, project documents, customer calls. The wiki stays current because the LLM does the maintenance no one on the team wants to do

• Competitive analysis, due diligence, trip planning, course notes — anything where you’re accumulating knowledge over time

In practice, Karpathy says: “I have the LLM agent open on one side and Obsidian open on the other. The LLM makes edits based on our conversation, and I browse the results in real time. Obsidian is the IDE; the LLM is the programmer; the wiki is the codebase.“

---

Step-by-Step Setup: The Complete 5-Layer Stack

Everything below is free and open-source.

Step 1: Install Graphify

Graphify automates Karpathy’s wiki architecture for codebases. It scans your files once and builds a semantic knowledge graph. From then on, Claude reads the graph — not every single file.

pip install graphifyy && graphify install

What it supports:

• 20 programming languages via tree-sitter AST parsing

• Multimodal: code, PDFs, images, video

• Works with Claude Code, Cursor, Codex, Gemini CLI

Step 2: Build Your Map

In Claude Code, run one command:

/graphify

This scans all your files and produces:

• graph.json — machine-readable semantic graph

• graph.html — interactive visual explorer

• Clustered communities with labels

• A markdown summary report

Your entire codebase, compressed into one navigable artifact.

Step 3: Tell Claude to Use the Map

Add these three rules to your CLAUDE.md:

## Context Navigation
1. ALWAYS query the knowledge graph first
2. Only read raw files if I explicitly say so
3. Use graphify-out/wiki/index.md as entry point

This single change transforms behavior. Claude stops re-reading 37 files and starts navigating a semantic map instead.

Result:

$ claude --usage
→ before: 20,000 tokens/session
→ after:  280 tokens/session
→ ~95% reduction vs loading all docs
✓ 71.5x reduction

Step 4: Set Up Obsidian (Your Visual IDE for Knowledge)

Obsidian is a free note-taking app built on local Markdown files. Connect it to Claude via MCP, and it becomes your AI’s persistent, visual, searchable external memory.

Setup:

1. Download Obsidian (free) from obsidian.md

2. Create a vault pointing to your raw/ and wiki/ folders

3. Import graph.json — graph view lights up

4. Install the 3D Graph plugin (via BRAT community plugin)

5. Connect to Claude via MCP: npx obsidian-mcp-server

Pro tips from Karpathy’s gist:

• Obsidian Web Clipper — browser extension that converts web articles to markdown. Drop them straight into your raw/ folder.

• Graph view is the best way to see the shape of your wiki — what’s connected, which pages are hubs, which are orphans.

• Dataview plugin — if your LLM adds YAML frontmatter to wiki pages (tags, dates, source counts), Dataview can generate dynamic tables and lists.

• Marp plugin — markdown-based slide decks, generated directly from wiki content.
  
  Here’s what a real Graphify knowledge graph looks like on a production monorepo:
  

  

  
  

Step 5: Deploy Acontext (Cross-Session Memory)

Acontext is the persistence layer. It auto-captures what Claude learns each session and stores it as Markdown skill files. Session 1: you explain everything. Session 10: Claude already knows.

git clone github.com/memodb-io/Acontext
docker compose up

What happens:

• After each session, Acontext distills learnings into SKILL.md files

• Next session, Claude loads relevant skills via get_skill()

• Knowledge compounds automatically — no manual work

The genius insight: Acontext uses the same file format for both “what the agent can do” (skills) and “what the agent has learned” (memories). Knowledge = capability. One format, two uses.

No embeddings. No vector databases. No vendor lock-in. Just plain Markdown files that are git-versionable, grep-able, and readable by any AI.

Bonus: the whole wiki is just a git repo of markdown files. You get version history, branching, and collaboration for free. At scale, add qmd — a local search engine for markdown with hybrid BM25/vector search and LLM re-ranking, all on-device. It has both a CLI and an MCP server.

---

The Complete 5-Layer Stack

Each layer solves one problem. Together they turn an amnesiac AI into a compounding knowledge partner:

All free. All open-source.

---

3 Things You Can Do This Week

You don’t need to implement the full stack at once. Each step independently improves your workflow.

Today: Apply Caveman Compression

Before your next Claude session: strip the preamble. Put the task first. Show an example instead of describing what you want. Never paste a whole file when you need one function.

You’ll immediately use 60% fewer tokens with better results.

This week: Run Graphify on your most complex repo

/graphify

Wait 3-5 minutes. You’ll have a semantic knowledge graph of your entire codebase. Import it into Obsidian for visual navigation. Claude will never need you to explain your architecture again.

This weekend: Write one “memory file”

At the end of your next Claude session, write a short Markdown file:

• What the task was

• What decision you made

• What you rejected and why

Save it as memory/decisions.md. Paste it at the start of next session. You’ve just built your first external memory layer — manually. Acontext automates this.

---

Why This Actually Works

The tedious part of maintaining a knowledge base is not the reading or the thinking — it’s the bookkeeping. Updating cross-references, keeping summaries current, noting when new data contradicts old claims, maintaining consistency across dozens of pages.

Humans abandon wikis because the maintenance burden grows faster than the value. LLMs don’t get bored, don’t forget to update a cross-reference, and can touch 15 files in one pass. The wiki stays maintained because the cost of maintenance is near zero.

The human’s job is to curate sources, direct the analysis, ask good questions, and think about what it all means. The LLM’s job is everything else.

The idea is related in spirit to Vannevar Bush’s Memex (1945) — a personal, curated knowledge store with associative trails between documents. Bush’s vision was closer to this than to what the web became: private, actively curated, with the connections between documents as valuable as the documents themselves. The part he couldn’t solve was who does the maintenance. The LLM handles that.

Your competitors using AI are not smarter than you. They just figured out memory architecture first. The gap compounds every week they run and you don’t.

Build the memory. Own the output.

---

Resources & Links

• Karpathy’s LLM Wiki Gist: gist.github.com/karpathy/442a6bf...

• Karpathy’s “Introduction to LLMs”: YouTube

• Graphify: /graphify in Claude Code (free skill)

• Obsidian: obsidian.md (free)

• Acontext: github.com/memodb-io/Acontext

• qmd search: github.com/tobi/qmd (local markdown search)

• Anthropic Skills Repo: github.com/anthropics/skills (114K+ stars)

• Awesome Agent Skills: VoltAgent/awesome-agent-skills (1,060+ skills)

---

If this was useful, follow me on LinkedIn for more AI engineering breakdowns. I post weekly about context engineering, agent architecture, and the tools that actually move the needle.

Have questions about the setup? Drop a comment — I read all of them.