Visualizing Next Word Prediction - How to LLMs Work?

 https://bbycroft.net/llm

What AI does before it writes code for you.

Before your AI writes a single line of Python, it takes 15 hidden mental steps.

Researchers just mapped the entire "thought process"—and it's wild.

Here's the complete breakdown 🧠👇

🗂️ PHASE 1: REQUIREMENTS GATHERING
The AI isn't just reading your prompt. It's:

TSK - Identifying the core task
CTX - Understanding code context (variables, functions, types)
CST - Spotting constraints (performance, recursion, input limits)

🧩 PHASE 2: SOLUTION PLANNING
Now it strategizes:

KRL - Recalls libraries/patterns from training data
CFL - Constructs control flow (loops, branches, logic)
CMP - Compares alternative approaches
AMB - Flags ambiguous/missing info

This is where smart prompts = better code.

⚙️ PHASE 3: IMPLEMENTATION
Two substeps:

SCG - Scaffold Code Generation (rough draft/pseudocode)
CCG - Complete Code Generation (final output)

Fun fact: 30% of AI responses skip this phase entirely in the reasoning trace.

🔍 PHASE 4: REFLECTION
The AI reviews its work:

UTC - Creates unit tests
ALT - Explores post-hoc alternatives
EGC - Identifies edge cases
FLW - Spots logical flaws
STY - Checks code style
SFA - Self-asserts "this is correct"

Here's the kicker:
Not all 15 steps happen every time.
The study found 5 common "reasoning patterns" (combos of steps).
The MOST successful pattern (FP1)?
TSK→CTX→CST→KRL→CFL→CMP→AMB→SCG→CCG→ALT→EGC→SFA
It's a complete human-like workflow.

But simpler tasks use simpler patterns.

Example: Self-contained functions skip:
❌ Ambiguity recognition (AMB)
❌ Alternative exploration (ALT)
❌ Edge case checks (EGC)

The AI adapts its reasoning depth based on task complexity.

Which step matters MOST for correct code?

📊 Analysis of 1,150 traces shows:

🥇 UTC (Unit Test Creation) - Strongest positive correlation
🥈 CCG (Complete Code) - Necessary for success
🥉 SCG (Scaffold) - Helps catch logic errors early

Which steps HURT performance?

🔻 CST (Constraint ID) - Negative correlation
🔻 AMB (Ambiguity Recognition) - Negative correlation
🔻 CMP (Solution Comparison) - Negative correlation

Why? They signal unclear prompts → bad assumptions → wrong code.

Real-world example:
When tasked with validating IP addresses, Qwen3-14B:

Identified task (TSK)
Recalled regex patterns (KRL)
Planned validation logic (CFL)
Generated test cases (UTC)
Wrote final code (CCG)
Self-asserted correctness (SFA)

Result? ✅ Passed all tests.

Understanding these 15 steps lets you:
✅ Write prompts that trigger the RIGHT reasoning
✅ Spot when AI is stuck in bad patterns
✅ Improve code quality by 10-15%

Carlos E Perez on X

If we already have automation, what's the need for Agents?

“Automation” and “agent” sound similar — but they solve very different classes of problems.

Automation = Fixed Instruction → Fixed Outcome

• Like Zapier, IFTTT, Jenkins pipelines, cron jobs.

• You pre-define exact triggers, actions, rules.

• Great when:

  • Context is stable.

  • No judgment / interpretation is needed.

  • The world doesn’t change mid-execution.

Example:

“Every day at 5pm, send me a sales report.”
✅ Perfect automation — zero thinking needed.

Agent = Goal → Autonomous Decision-Making

• Given a goal, not just rules.

• Perceives, plans, adapts, self-corrects, retries, negotiates ambiguity.

• Can operate even when instructions are incomplete or circumstances change.

• Doesn’t need babysitting.

Example:

“Grow my revenue 15% next quarter — find the best channels, experiment, and adjust.”

✅ That’s NOT automatable. Needs strategy, improvisation, learning, resource orchestration. 

Understanding token size

 

LLM - where are the parameters stored, and the file system

 

What is an LLM? Is it a set of files? Does it sit as an .exe? A folder? A single binary? What does it LOOK LIKE if I download it?”

Answer: YES — an LLM is literally a set of files.
A big model file — like .bin, .pth, .safetensors, etc. — usually 2GB to 400GB+.

Parameters live inside the model — not in vector DB.

✅ Vector DB only stores embeddings of user/business knowledge for retrieval.

Automation vs AI Workflow vs AI Agents

 

Amazon Bedrock Guardrails

Amazon Bedrock Guardrails provides safeguards that you can configure for your generative AI applications based on your use cases and responsible AI policies. You can create multiple guardrails tailored to different use cases and apply them across multiple foundation models (FMs), providing a consistent user experience and standardizing safety and privacy controls across generative AI applications. You can use guardrails for both model prompts and responses with natural language.

You can use Amazon Bedrock Guardrails in multiple ways to help safeguard your generative AI applications. For example:

• A chatbot application can use guardrails to help filter harmful user inputs and toxic model responses.

• A banking application can use guardrails to help block user queries or model responses associated with seeking or providing investment advice.

• A call center application to summarize conversation transcripts between users and agents can use guardrails to redact users’ personally identifiable information (PII) to protect user privacy.

Amazon Bedrock Guardrails provides the following safeguards (also known as policies) to detect and filter harmful content:

• Content filters – Detect and filter harmful text or image content in input prompts or model responses. Filtering is done based on detection of certain predefined harmful content categories: Hate, Insults, Sexual, Violence, Misconduct and Prompt Attack. You also can adjust the filter strength for each of these categories.

• Denied topics – Define a set of topics that are undesirable in the context of your application. The filter will help block them if detected in user queries or model responses.

• Word filters – Configure filters to help block undesirable words, phrases, and profanity (exact match). Such words can include offensive terms, competitor names, etc.

• Sensitive information filters – Configure filters to help block or mask sensitive information, such as personally identifiable information (PII), or custom regex in user inputs and model responses. Blocking or masking is done based on probabilistic detection of sensitive information in standard formats in entities such as SSN number, Date of Birth, address, etc. This also allows configuring regular expression based detection of patterns for identifiers.

• Contextual grounding checks – Help detect and filter hallucinations in model responses based on grounding in a source and relevance to the user query.

• Automated Reasoning checks – Can help you validate the accuracy of foundation model responses against a set of logical rules. You can use Automated Reasoning checks to detect hallucinations, suggest corrections, and highlight unstated assumptions in model responses.

x

Generative AI Lifecycle

The generative AI lifecycle provides a structured framework for developing and deploying AI solutions. It consists of five key stages: defining a use case, selecting a foundation model, improving performance, evaluating results, and deploying the application.

This iterative process begins with clearly articulating the business problem and requirements, then choosing an appropriate pre-trained model as a starting point.

Throughout the lifecycle, there's a focus on continuous refinement to ensure the AI solution remains effective and aligned with business objectives.

While generative AI has numerous applications, it's equally important to recognize situations where it might not be the most appropriate solution. These include situations with high accuracy and reliability requirements, ill-defined or constantly changing problems, insufficient data quality, the need for explainability and transparency, cost-benefit considerations, and ethical concerns.

Sometimes, other methods work better than AI. This includes simple tasks that are solvable with rule-based solutions, or when model costs outweigh business benefits.

AI Use Cases

Amazon Sagemaker

 

Amazon SageMaker is used by hundreds of thousands of AWS customers to build, train, and deploy machine learning models. Now, we've taken the machine learning service and added AWS analytics capabilities - creating one unified platform for data, analytics, and AI.

The next generation of Amazon SageMaker includes virtually all of the components you need for fast SQL analytics, big data processing, search, data preparation and integration, AI model development and training, and generative AI - along with a single view into all of your enterprise data. You get a single data and AI development environment with the SageMaker unified studio, a lakehouse architecture that unifies access to all your data - on S3, in Redshift, in SaaS applications, on-premises, or in other clouds - through the open Apache Iceberg standard interface and with the SageMaker Catalog built into Unified Studio, you get end-to-end governance for your data and AI workflows.

Amazon SageMaker AIThe service previously known as Amazon SageMaker has been renamed Amazon SageMaker AI. It is integrated within the next generation of SageMaker and is also available as a standalone service for those who wish to focus specifically on building, training, and deploying AI and ML models at scale.

Amazon SageMaker AI is a fully managed service to build, train, and deploy ML models - including foundation models - for any use case by bringing together a broad set of tools to enable high-performance, low-cost machine learning. It is available as a standalone service in the AWS console, or via APIs. Model development capabilities from SageMaker AI are available in the next generation of Amazon SageMaker.

1/Amazon SageMaker AI provides access to high-performance, cost-effective, scalable, and fully managed infrastructure and tools for each step of the ML lifecycle. Using Amazon SageMaker AI tools, you can easily build, train, test, troubleshoot, deploy, and manage FMs models at scale and boost productivity of data scientists and ML engineers while maintaining model performance in production.

2/You can explore Amazon SageMaker JumpStart, which is a ML hub offering models, algorithms, and prebuilt ML solutions. SageMaker JumpStart offers hundreds of ready-to-use FMs from various model providers, including a growing list of best performing publicly available FMs such as Falcon-40B, Stable Diffusion, OpenLLaMA, and Flan-T5/UL2.

3/Amazon SageMaker machine learning operations (MLOps) capabilities help you create repeatable workflows across the ML lifecycle to experiment, train, deploy, and govern ML models at scale while maintaining model performance in production.

4/Amazon SageMaker AI provides purpose-built governance tools to help you implement ML responsibly. Amazon SageMaker Model Cards makes it easier to capture, retrieve, and share essential model information. Once the models are deployed, SageMaker Model Dashboard gives you unified monitoring across all your models by providing deviations from expected behavior, automated alerts, and troubleshooting to improve model performance. Amazon SageMaker Clarify detects and measures potential bias using a variety of metrics to help you address potential bias and explain model predictions.

5/With Amazon SageMaker Ground Truth, you can use human feedback to customize models on company- or domain-specific data for your unique use case to improve model output and task performance.

AI, ML, DL, Gen AI

Artificial intelligence (AI): The overarching field of AI, which creates intelligent systems that perform human-like tasks

• Example: Siri and Alexa are examples of AI systems that can perform human-like tasks such as answering questions, setting reminders, and controlling smart home devices.

• Machine learning (ML): A subset of AI that uses statistical techniques for prediction based on patterns

• Example: Spam filters that learn to identify and block unwanted emails are an example of ML, where the system analyzes patterns in email data to make predictions about future messages.

• Deep learning (DL): A type of ML based on neural networks that are capable of learning complex patterns from large datasets

• Example: Facial recognition systems used in smartphones and social media platforms are powered by deep learning, which can learn complex patterns in large datasets of facial images.

• Generative AI: A subset of DL that creates new data based on learned patterns, often without retraining

• Example: Text-generating models like Amazon Nova Lite and image-generating models like Amazon Nova Canvas are examples of generative AI, which can create new content (such as articles, stories, or images) based on the patterns they've learned from their training data.

In generative AI, a model is the result of applying a machine learning algorithm to training data. Models encapsulate the patterns, relationships, and rules learned from the data, so that the AI system can generate new content or make predictions when given new inputs.

The quality of a generative AI model is critically dependent on both the training data and the ML algorithm that you use. High-quality, diverse training data helps the model learn a wide range of patterns and nuances, while an appropriate algorithm ensures effective learning from this data.

Model development is often iterative. Initial models might have limitations or biases. You can address these issues by refining training data, adjusting algorithms, or fine-tuning model parameters. AWS services such as SageMaker AI help with this iterative process by providing tools for model training, evaluation, and deployment.

Be aware that a model is only as good as the information it was trained on. It is important to carefully curate data and continuously monitor and update models to make sure they remain accurate and relevant over time.

Generative AI Essentials on AWS

 https://www.bespoketraining.com/resources/webinars/unlocking-azure-ai-essentials-generative-solutions/

Generative AI - New Workflows and Opportunities

 

Re engineering of workflows above

Automation opportunities

 

RAG vs Pre-Train vs. Fine-tune

 

Agents

 

LLM Tool Use

 

Instruction Tuning and Reinforcement Learning from Human Feedback (RLHF)

 

RLHF - rewards to generate higher reinforcements

Choosing model - closed source or open source

 

How to choose an Model?

 

LLM Pre-training

Definition of Pre-training

Pre-training is the process of training a model on vast, diverse, and largely unlabeled data to learn general representations and patterns of a domain — such as language, vision, audio, or sensor data — before it is specialized for specific tasks.

It is a self-supervised learning stage where the model develops an internal “world model” by predicting or reconstructing parts of its input (e.g., the next token, masked pixel, next audio frame, or next action).
The goal is not to perform a narrow task, but to build a foundation of understanding that later fine-tuning, prompting, or reinforcement can adapt to many downstream objectives.

---

🧠 Core Idea

Pre-training = learning how the world looks, sounds, or reads —
before learning how to do something with that knowledge.

General Formulation

Aspect	Description
Input	Large, diverse, unlabeled data (text, images, audio, code, trajectories, etc.)
Objective	Predict missing or future parts of data (self-supervised task)
Outcome	Dense, structured representations (embeddings) capturing meaning and relationships
Purpose	Build transferable understanding to accelerate later adaptation

Why It Matters

Pre-training converts raw data → reusable intelligence.
Once the base model is pretrained, it can be:

• Fine-tuned for specialized tasks,

• Aligned for human intent (via RLHF),

• Connected to live knowledge (via RAG).

It’s the difference between:

teaching a brain how to think and perceive,
versus teaching it what to think or do.

When You Should Pre-train

You should pre-train from scratch only when:

• You need a new base model (new architecture, tokenizer, or modality);

• Existing models don’t cover your language or data type (e.g., low-resource languages, medical imaging, genomic data);

• You want full control over knowledge, bias, and compliance;

• You’re performing foundational research into architectures or training dynamics.

Otherwise — reuse and fine-tune an existing pre-trained foundation.