Tuesday, December 09, 2025

What AI does before it writes code for you.

Before your AI writes a single line of Python, it takes 15 hidden mental steps.

Researchers just mapped the entire "thought process"—and it's wild.

Here's the complete breakdown 🧠👇

🗂️ PHASE 1: REQUIREMENTS GATHERING
The AI isn't just reading your prompt. It's:

TSK - Identifying the core task
CTX - Understanding code context (variables, functions, types)
CST - Spotting constraints (performance, recursion, input limits)

🧩 PHASE 2: SOLUTION PLANNING
Now it strategizes:

KRL - Recalls libraries/patterns from training data
CFL - Constructs control flow (loops, branches, logic)
CMP - Compares alternative approaches
AMB - Flags ambiguous/missing info

This is where smart prompts = better code.

⚙️ PHASE 3: IMPLEMENTATION
Two substeps:

SCG - Scaffold Code Generation (rough draft/pseudocode)
CCG - Complete Code Generation (final output)

Fun fact: 30% of AI responses skip this phase entirely in the reasoning trace.

🔍 PHASE 4: REFLECTION
The AI reviews its work:

UTC - Creates unit tests
ALT - Explores post-hoc alternatives
EGC - Identifies edge cases
FLW - Spots logical flaws
STY - Checks code style
SFA - Self-asserts "this is correct"

Here's the kicker:
Not all 15 steps happen every time.
The study found 5 common "reasoning patterns" (combos of steps).
The MOST successful pattern (FP1)?
TSK→CTX→CST→KRL→CFL→CMP→AMB→SCG→CCG→ALT→EGC→SFA
It's a complete human-like workflow.

But simpler tasks use simpler patterns.

Example: Self-contained functions skip:
❌ Ambiguity recognition (AMB)
❌ Alternative exploration (ALT)
❌ Edge case checks (EGC)

The AI adapts its reasoning depth based on task complexity.

Which step matters MOST for correct code?

📊 Analysis of 1,150 traces shows:

🥇 UTC (Unit Test Creation) - Strongest positive correlation
🥈 CCG (Complete Code) - Necessary for success
🥉 SCG (Scaffold) - Helps catch logic errors early

Which steps HURT performance?

🔻 CST (Constraint ID) - Negative correlation
🔻 AMB (Ambiguity Recognition) - Negative correlation
🔻 CMP (Solution Comparison) - Negative correlation

Why? They signal unclear prompts → bad assumptions → wrong code.

Real-world example:
When tasked with validating IP addresses, Qwen3-14B:

Identified task (TSK)
Recalled regex patterns (KRL)
Planned validation logic (CFL)
Generated test cases (UTC)
Wrote final code (CCG)
Self-asserted correctness (SFA)

Result? ✅ Passed all tests.

Understanding these 15 steps lets you:
✅ Write prompts that trigger the RIGHT reasoning
✅ Spot when AI is stuck in bad patterns
✅ Improve code quality by 10-15%

Carlos E Perez on X

No comments:

Post a Comment

Visualizing Next Word Prediction - How to LLMs Work?

 https://bbycroft.net/llm