LLM Quality vs Size small language models

 Courtesy: Microsoft.com 

Hadoop Ecosystem

 Courtesy: data-flair.training, bizety

• Hadoop provides a distributed storage and processing framework.
• Hadoop is a framework for distributed storage and processing of large datasets across clusters of computers. 
• It includes two main components: 
• Hadoop Distributed File System (HDFS) for storage and 
• MapReduce for processing. 
• Hadoop is designed to store and process massive amounts of data in a fault-tolerant and scalable manner.
• Hive is a data warehouse infrastructure built on top of Hadoop. It provides a SQL-like interface for querying and analyzing data stored in Hadoop. Hive is suitable for data warehousing and analytics use cases.
• PySpark enables data processing using the Spark framework with Python.
• YARN manages cluster resources to support various processing frameworks.
• HBase provides a scalable NoSQL database solution for real-time access to large datasets.
  

LLM Architecture

AI and LLMs - why GPUs, what happened to CPUs?

This has been written with the help of Copilot.

Central Processing Units (CPUs)

• CPUs are versatile and handle a wide range of tasks (execution of instructions, general purpose computing, coordinating various components within the computer system (memory, I/O, peripheral devices, etc.).
• They are the “brains” of a computer, executing instructions from software programs.
• CPUs excel at sequential processing, where tasks are executed one after the other. They also support parallel processing to some extent through techniques like multi-core architectures and pipelining.
• They manage tasks like operating system functions, application execution, and handling I/O operations.
• CPUs are essential for general-purpose computing, including running applications, managing memory, and coordinating system resources.

Graphics Processing Units (GPUs), 

GPUs were initially associated primarily with gaming, and other applications like scientific simulations and data processing, however they now have transcended their original purpose as GPUs have evolved beyond just gaming. Let’s explore how this transformation occurred and why GPUs are now indispensable for various computing tasks beyond gaming.

• GPUs are specialized hardware components designed for parallel processing.
• Their architecture consists of thousands of cores, each capable of handling computations simultaneously.
• Originally developed for graphics rendering (such as gaming), GPUs evolved to handle complex mathematical operations efficiently.
• GPUs excel at tasks like matrix operations, image processing, and parallel algorithms.
• In recent years, GPUs have become crucial for AI, machine learning, scientific simulations, and data-intensive workloads.

1. Evolution of GPUs:

   • Early Days: GPUs were initially designed for rendering lifelike graphics in video games. Their primary role was to alleviate the burden on the Central Processing Unit (CPU) by handling graphical computations.
   • Parallel Processing: Over time, GPUs evolved significantly. Modern GPUs are equipped with thousands of cores optimized for parallel processing. Unlike CPUs, which focus on sequential tasks, GPUs excel at performing numerous calculations simultaneously.
   • Massive Computational Power: Today’s GPUs can achieve teraflops (trillions of floating-point operations per second), a capability that was inconceivable just 15 years ago1.

2. Beyond Gaming: Diverse Applications:

   • Artificial Intelligence (AI) and Machine Learning:
     • GPUs play a pivotal role in training neural networks for AI and machine learning.
     • Their parallel architecture accelerates tasks like natural language processing and computer vision.
   • Data Science and Analytics:
     • GPUs handle massive datasets efficiently, reducing computation times for tasks like data preprocessing and statistical analysis.
   • High-Performance Computing (HPC):
     • Scientific research, weather forecasting, and simulations rely heavily on GPUs.
     • They excel in solving complex mathematical models with remarkable accuracy.
   • Medical Imaging and Research:
     • GPUs process large amounts of visual data in medical imaging techniques like MRI and CT scans, aiding diagnosis and research1.

3. The Trajectory of GPUs:

   • As technology advances, the demand for high-performance computing, especially in AI and machine learning, is expected to soar.
   • GPUs will continue to evolve, integrating more efficiently with different technologies and expanding their role beyond traditional applications.
   • Optimizing GPU usage involves understanding their architecture, using optimized libraries (like CUDA or OpenCL), and parallelizing tasks to maximize potential.

LLMs - words vs tokens

https://kelvin.legal/understanding-large-language-models-words-versus-tokens/#:~:text=The%20size%20of%20text%20an,the%20cost%20and%20vice%20versa.

Tokens can be thought of as pieces of words. Before the model processes the prompts, the input is broken down into tokens. These tokens are not cut up exactly where the words start or end - tokens can include trailing spaces and even sub-words. -- Llama.

The size of text an LLM can process and generate is measured in tokens. Additionally, the operational expense of LLMs is directly proportional to the number of tokens it processes - the fewer the tokens, the lower the cost and vice versa. 

Tokenizing language translates it into numbers – the format that computers can actually process. Using tokens instead of words enables LLMs to handle larger amounts of data and more complex language. By breaking words into smaller parts (tokens), LLMs can better handle new or unusual words by understanding their building blocks.