Why is ChatGPT doing what it is doing?

https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/

GPUs

https://jax-ml.github.io/scaling-book/gpus/

AI Agents Memory

𝗔𝗜 𝗔𝗴𝗲𝗻𝘁’𝘀 𝗠𝗲𝗺𝗼𝗿𝘆 is the most important piece of 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴, this is how we define it 

In general, the memory for an agent is something that we provide via context in the prompt passed to LLM that helps the agent to better plan and react given past interactions or data not immediately available.

It is useful to group the memory into four types:

𝟭. 𝗘𝗽𝗶𝘀𝗼𝗱𝗶𝗰 - This type of memory contains past interactions and actions performed by the agent. After an action is taken, the application controlling the agent would store the action in some kind of persistent storage so that it can be retrieved later if needed. A good example would be using a vector Database to store semantic meaning of the interactions.

𝟮. 𝗦𝗲𝗺𝗮𝗻𝘁𝗶𝗰 - Any external information that is available to the agent and any knowledge the agent should have about itself. You can think of this as a context similar to one used in RAG applications. It can be internal knowledge only available to the agent or a grounding context to isolate part of the internet scale data for more accurate answers.

𝟯. 𝗣𝗿𝗼𝗰𝗲𝗱𝘂𝗿𝗮𝗹 - This is systemic information like the structure of the System Prompt, available tools, guardrails etc. It will usually be stored in Git, Prompt and Tool Registries.

𝟰. Occasionally, the agent application would pull information from long-term memory and store it locally if it is needed for the task at hand.

𝟱. All of the information pulled together from the long-term or stored in local memory is called short-term or working memory. Compiling all of it into a prompt will produce the prompt to be passed to the LLM and it will provide further actions to be taken by the system.

We usually label 1. - 3. as Long-Term memory and 5. as Short-Term memory.

#LLM #AI #ContextEngineering

Google Genie 3 & where's it leading us to

1. Advancing “world models” for AI

AI agents need realistic, interactive environments to learn decision-making (e.g., how to navigate, manipulate objects, or plan actions).

Traditional simulators (like game engines) are hand-coded and slow to build. Genie 3 generates new, physics-aware environments instantly from text prompts.

This makes it useful for training AI at scale without needing human-designed levels.

---

2. Democratizing content creation

Currently, building a game or simulation requires coding, asset design, and engines.

Genie 3 removes that barrier by letting anyone type a prompt (“a forest at sunset with floating islands”) and get an explorable world in seconds.

This could lead to personalized games, educational tools, or VR simulations without technical skills.

---

3. Testing AI memory and reasoning

Genie 3 introduces visual memory (the AI remembers object placement for ~1 minute).

This allows researchers to study how AI handles continuity—a step toward agents that can remember and interact in more complex ways.

---

4. Faster experimentation for researchers and developers

Instead of waiting weeks for artists and engineers to design levels, researchers can spin up thousands of unique worlds for experiments, robotics planning, or reinforcement learning.

Potential applications: autonomous driving, robotics training, creative prototyping.

---

5. Laying groundwork for AI-generated entertainment

While not a finished product, Genie 3 hints at a future where games “write themselves” based on what you imagine.

Think: a Minecraft-like game that reshapes itself dynamically rather than relying on blocks or mods.

---

In short: Genie 3 solves the problem of rapidly generating rich, interactive worlds without manual effort, which is crucial for AI development and creative prototyping, not just gaming

Framework for AI Workflow

Source

https://www.arxiv.org/pdf/2508.01186

Modern large language models (LLMs) are increasingly used as autonomous agents—capable of planning tasks, invoking tools, collaborating with other agents, and adapting to changing environments. However, as these systems grow more complex, ad hoc approaches to building and coordinating them are breaking down. Current challenges include:

1. Lack of standardized structures for how agents should coordinate, plan, and execute tasks.

2. Fragmentation of frameworks—academic and industrial systems vary widely in architecture, terminology, and capabilities, making comparison difficult.

3. Scalability and reliability issues—dynamic environments demand flexible workflows, but existing designs are often brittle or inefficient.

4. Security and trust concerns—multi-agent workflows introduce risks like tool poisoning, memory corruption, and collusion.

5. Absence of clear evaluation metrics—it’s unclear how to measure success or optimize these workflows systematically.

In other words, there’s no unified understanding of how to design, manage, and improve agent workflows. The paper proposes to address this by surveying current approaches, identifying their strengths and weaknesses, and outlining future research directions.

Train yourself in QA - Roadmap

Train yourself in QA

AI Industrial complex

The AI Industrial Complex is a critical term used to describe the growing network of companies, governments, research institutions, and military or security organizations that are driving the rapid development and deployment of artificial intelligence—often prioritizing power, profit, or control over ethical considerations.

It’s modeled on terms like the “Military-Industrial Complex,” which warned about entrenched systems where industries and governments reinforce each other’s interests, making oversight and reform difficult.

Core Features:

1. Concentration of Power

A few tech giants (e.g., OpenAI, Google, Anthropic, Microsoft) dominate AI research, infrastructure, and data access.

These companies influence policy and public narratives around AI risks and benefits.

2. State-Industry Alliances

Governments fund AI development for economic competition, surveillance, and defense.

In return, companies gain contracts, regulatory advantages, or subsidies.

3. Hype and Speculation

Fear of “falling behind” drives massive investment, often inflating promises of what AI can deliver.

Narratives about “AI safety” or “AI for good” can mask underlying motives (e.g., market control or militarization).

4. Ethical and Social Trade-offs

Labor displacement, surveillance, bias, and environmental costs are sidelined.

Smaller players and public interests struggle to influence the trajectory.

Why the term matters:

Critics use “AI Industrial Complex” to suggest that AI development isn’t purely about innovation but about consolidating power and shaping society around the interests of a few.

Do you want me to explain who coined it, examples of how it operates today, or why some argue it’s dangerous but others think it’s necessary?

Prover-Verifier Games and GPT-5

https://arxiv.org/html/2407.13692v2

How to use AI

https://www.oneusefulthing.org/p/using-ai-right-now-a-quick-guide?s=09

Persona vectors

Rohan Paul

Anthropic just showed that an AI's “personality” can be traced to specific directions in its brain ("Persona vectors"), and shows what might make it act in evil or unsafe ways.

Sometimes when you're chatting with a model, it suddenly starts behaving oddly—overly flattering, factually wrong, or even outright malicious. This paper is about understanding why that happens, and how to stop it.

🧠 What's going on inside these models?

AI models don’t actually have personalities like humans do, but they sometimes act like they do—especially when prompted a certain way or trained on particular data. 

Anthropic’s team found that specific behaviors, like being “evil,” “sycophantic,” or prone to “hallucination,” show up as linear directions inside the model's activation space. 

They call these persona vectors.

Think of it like this: if you observe how the model responds in different situations, you can map those behaviors to certain regions inside the model’s brain. And if you spot where these traits live, you can monitor and even control them.

---

The diagram shows a simple pipeline that turns a plain description of a trait such as evil into a single “persona vector”, which is just a pattern of activity inside the model that tracks that trait.

Once this vector exists, engineers can watch the model’s activations and see in real time if the model is drifting toward the unwanted personality while it runs or while it is being finetuned.

The very same vector works like a control knob. 

Subtracting it during inference tones the trait down, and sprinkling a small amount of it during training teaches the model to resist picking that trait up in the first place, so regular skills stay intact.

Because each piece of training text can also be projected onto the vector, any snippet that would push the model toward the trait lights up early, letting teams filter or fix that data before it causes trouble.

Al that means, you can control the following of a model

- Watch how a model’s personality evolves, either while chatting or during training

- Control or reduce unwanted personality changes as the model is being developed or trained

- Figure out what training data is pushing those changes

🔬 How to make sense of this persona vector?

Think of a large language model as a machine that turns every word it reads into a long list of numbers. That list is called the activation vector for that word, and it might be 4096 numbers long in a model the size of Llama-3.

A persona vector is another list of the same length, but it is not baked into the model’s weights. The team creates it after the model is already trained:

They run the model twice with the same user question, once under a “be evil” system prompt and once under a “be helpful” prompt.

They grab the hidden activations from each run and average them, so they now have two mean activation vectors.

They subtract the helpful average from the evil average. The result is a single direction in that 4096-dimensional space. That direction is the persona vector for “evil.”

Because the vector lives outside the model, you can store it in a tiny file and load it only when you need to check or steer the personality. During inference you add (or subtract) a scaled copy of the vector to the activations at one or more layers. Pushing along the vector makes the bot lean into the trait, pulling against it tones the trait down. During fine-tuning you can sprinkle a bit of the vector in every step to “vaccinate” the model so later data will not push it toward that trait.

So, under the hood, a persona vector is simply a 1-dimensional direction inside the model’s huge activation space, not a chunk of the weight matrix. It is computed once, saved like any other small tensor, and then used as a plug-in dial for personality control.

---

The pipeline is automated, so any new trait just needs a plain-language description and a handful of trigger prompts. 

They validate the result by injecting the vector and watching the bot slip instantly into the matching personality.

AI Context

https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html

LLM vs Gen AI vs AI Agents vs Agentic AI

 Brij Kishore Pandey

LLM ≠ Generative AI ≠ AI Agents ≠ Agentic AI

We need to stop grouping them together.

Each serves a different purpose, operates at a different level of complexity, and solves a different class of problems.

Here’s the breakdown:

🔹 LLM
Predicts tokens based on patterns in data.
No memory. No intent. No task execution. Just input → output.

🔹 Generative AI
Builds on LLMs to create text, code, images, etc.
It understands latent space and can generate novel content—but it still waits for instructions.

🔹 AI Agents
Execute predefined tasks.
They detect intent, call tools or APIs, and handle responses. They’re modular and functional—but not autonomous.

🔹 Agentic AI
Operates with goals, plans, context, and memory.
It reasons, adapts, calls sub-agents, monitors progress, and decides what to do next—without human instruction.

This isn’t just a progression of features.

It’s a shift in system design—from prediction to orchestration, from commands to autonomy.

If you're building with AI, clarity on where your system fits in this stack determines everything: architecture, tooling, risk, and value.

For GIF, look here

https://www.linkedin.com/posts/brijpandeyji_llm-generative-ai-ai-agents-agentic-activity-7355965770012491776-jAH9?utm_source=share&utm_medium=member_desktop&rcm=ACoAAAFvb4sBDNrkvuRa0AxL4xsDk4H1TYEYH30