Visualizing Next Word Prediction - How to LLMs Work?

 https://bbycroft.net/llm

What AI does before it writes code for you.

Before your AI writes a single line of Python, it takes 15 hidden mental steps.

Researchers just mapped the entire "thought process"—and it's wild.

Here's the complete breakdown 🧠👇

🗂️ PHASE 1: REQUIREMENTS GATHERING
The AI isn't just reading your prompt. It's:

TSK - Identifying the core task
CTX - Understanding code context (variables, functions, types)
CST - Spotting constraints (performance, recursion, input limits)

🧩 PHASE 2: SOLUTION PLANNING
Now it strategizes:

KRL - Recalls libraries/patterns from training data
CFL - Constructs control flow (loops, branches, logic)
CMP - Compares alternative approaches
AMB - Flags ambiguous/missing info

This is where smart prompts = better code.

⚙️ PHASE 3: IMPLEMENTATION
Two substeps:

SCG - Scaffold Code Generation (rough draft/pseudocode)
CCG - Complete Code Generation (final output)

Fun fact: 30% of AI responses skip this phase entirely in the reasoning trace.

🔍 PHASE 4: REFLECTION
The AI reviews its work:

UTC - Creates unit tests
ALT - Explores post-hoc alternatives
EGC - Identifies edge cases
FLW - Spots logical flaws
STY - Checks code style
SFA - Self-asserts "this is correct"

Here's the kicker:
Not all 15 steps happen every time.
The study found 5 common "reasoning patterns" (combos of steps).
The MOST successful pattern (FP1)?
TSK→CTX→CST→KRL→CFL→CMP→AMB→SCG→CCG→ALT→EGC→SFA
It's a complete human-like workflow.

But simpler tasks use simpler patterns.

Example: Self-contained functions skip:
❌ Ambiguity recognition (AMB)
❌ Alternative exploration (ALT)
❌ Edge case checks (EGC)

The AI adapts its reasoning depth based on task complexity.

Which step matters MOST for correct code?

📊 Analysis of 1,150 traces shows:

🥇 UTC (Unit Test Creation) - Strongest positive correlation
🥈 CCG (Complete Code) - Necessary for success
🥉 SCG (Scaffold) - Helps catch logic errors early

Which steps HURT performance?

🔻 CST (Constraint ID) - Negative correlation
🔻 AMB (Ambiguity Recognition) - Negative correlation
🔻 CMP (Solution Comparison) - Negative correlation

Why? They signal unclear prompts → bad assumptions → wrong code.

Real-world example:
When tasked with validating IP addresses, Qwen3-14B:

Identified task (TSK)
Recalled regex patterns (KRL)
Planned validation logic (CFL)
Generated test cases (UTC)
Wrote final code (CCG)
Self-asserted correctness (SFA)

Result? ✅ Passed all tests.

Understanding these 15 steps lets you:
✅ Write prompts that trigger the RIGHT reasoning
✅ Spot when AI is stuck in bad patterns
✅ Improve code quality by 10-15%

Carlos E Perez on X

If we already have automation, what's the need for Agents?

“Automation” and “agent” sound similar — but they solve very different classes of problems.

Automation = Fixed Instruction → Fixed Outcome

• Like Zapier, IFTTT, Jenkins pipelines, cron jobs.

• You pre-define exact triggers, actions, rules.

• Great when:

  • Context is stable.

  • No judgment / interpretation is needed.

  • The world doesn’t change mid-execution.

Example:

“Every day at 5pm, send me a sales report.”
✅ Perfect automation — zero thinking needed.

Agent = Goal → Autonomous Decision-Making

• Given a goal, not just rules.

• Perceives, plans, adapts, self-corrects, retries, negotiates ambiguity.

• Can operate even when instructions are incomplete or circumstances change.

• Doesn’t need babysitting.

Example:

“Grow my revenue 15% next quarter — find the best channels, experiment, and adjust.”

✅ That’s NOT automatable. Needs strategy, improvisation, learning, resource orchestration. 

Understanding token size

 

LLM - where are the parameters stored, and the file system

 

What is an LLM? Is it a set of files? Does it sit as an .exe? A folder? A single binary? What does it LOOK LIKE if I download it?”

Answer: YES — an LLM is literally a set of files.
A big model file — like .bin, .pth, .safetensors, etc. — usually 2GB to 400GB+.

Parameters live inside the model — not in vector DB.

✅ Vector DB only stores embeddings of user/business knowledge for retrieval.