A Comprehensive Study Guide
A detailed glossary of essential terminology for understanding large language models and their applications.
Glossary of Key Terms
- Foundation Model: A large language model (LLM) pre-trained on a massive dataset, capable of understanding and generating human-like text across a wide range of tasks.
- Fine-Tuning: The process of taking a pre-trained LLM and further training it on a smaller, task-specific dataset to adapt its weights and improve its performance on that particular task or domain.
- RAG (Retrieval-Augmented Generation): A framework that enhances LLM responses by retrieving relevant information from an external Knowledge Base and incorporating it into the generation process.
- Knowledge Base (KB): A collection of documents or data from which relevant information is retrieved in a RAG system.
- Vector Database: A specialized database that stores vector representations (embeddings) of data, optimized for efficient similarity searches used in RAG for retrieving relevant information.
- Prompting: The act of providing carefully crafted input text to an LLM to guide its output generation and elicit desired responses.
- Zero-Shot Learning: The ability of an LLM to perform a task based solely on the task instructions, without any prior examples.
- Few-Shot Learning: The ability of an LLM to learn and perform a task given only a very small number of examples in the prompt.
- Instruction Tuning: A fine-tuning technique where the training data includes specific instructions paired with desired outputs, improving the LLM’s ability to follow instructions effectively.
- Hallucination: The tendency of LLMs to generate incorrect, nonsensical, or factually inconsistent information that is not supported by the input context or their training data.
- Context Length: The maximum number of input tokens or words that an LLM can process and consider when generating an output.
- Transformer: A popular neural network architecture widely used in LLMs, known for its attention mechanism that allows it to weigh the importance of different parts of the input sequence and its parallel processing capabilities.
- In-Context Learning: The ability of an LLM to learn a new task by being provided with examples directly within the prompt, without requiring explicit fine-tuning.
- Quantization: A technique used to reduce the computational resources and memory footprint of an LLM by decreasing the precision of its parameters.
- Freeze Tuning: A fine-tuning method where most of the LLM’s parameters are kept frozen, and only a small subset of layers or parameters are updated during training.
- Contrastive Learning: A fine-tuning approach that trains LLMs to understand the similarity and differences between data points, often used for improving the quality of embeddings.
- RLHF (Reinforcement Learning from Human Feedback): A technique used to align LLM behavior with human preferences by using human feedback as a reward signal to train the model.
- Reward Modeling: A component of RLHF where a separate model is trained to predict human preference scores for different LLM outputs, serving as the reward signal for reinforcement learning.
- Pruning: A technique used to reduce the size and computational cost of LLMs by removing redundant or less important connections or parameters.
- LoRA (Low-Rank Adaption): A Parameter-Efficient Fine-Tuning (PEFT) method that inserts a smaller set of new weight matrices into the LLM and trains only these new parameters, significantly reducing the number of trainable parameters.
- SFT (Supervised Fine-Tuning): The process of updating a pre-trained LLM with labeled data (input-output pairs) to make it perform a specific task.
- Transfer Learning: A machine learning technique where knowledge gained from training on a large dataset is applied to improve the performance on a smaller, related task.
- PEFT (Parameter-Efficient Fine-Tuning): Techniques that update only a small fraction of an LLM’s parameters during fine-tuning, making the process more computationally efficient and cost-effective.
- Agent Planning: A module in LLM applications that breaks down complex tasks into smaller, manageable steps to fulfill user requests.
- LLM Agent: An application that combines the capabilities of an LLM with other modules like planning, memory, and tool use to execute complex tasks.
- Agent Memory: A module that allows an LLM agent to store and recall past interactions and experiences, enabling more coherent and context-aware behavior.
- Function Calling: The ability of LLM agents to interact with external tools and APIs to gather information or perform actions required to complete a task.
- Vector Search: The process of finding the most relevant vector representations in a vector database based on similarity to a query vector.
- Indexing: The process of organizing and structuring data in a Knowledge Base (KB) to enable efficient retrieval. In the context of RAG, it often involves converting KB chunks into vector embeddings and storing them in a vector database.
- Embedding Model: An LLM or a specialized model that converts text or other data into numerical vector representations (embeddings).
- AGIRetrieval: An approach used to rank and fetch Knowledge Base (KB) chunks from the vector search results, which will then be used as additional context for the LLM in RAG.
- Chunking: The process of dividing large documents or the Knowledge Base into smaller, more manageable pieces (chunks) for efficient storage and retrieval in RAG.
- Artificial General Intelligence (AGI): The theoretical ability of a machine to perform any intellectual task that a human being can, across a wide range of domains.
- LLM Bias: Systematic and unfair prejudices present in an LLM’s predictions, often originating from biases in the training data.
- Responsible AI: An overarching framework encompassing principles and practices aimed at ensuring the ethical, fair, and transparent development and deployment of AI systems.
- GDPR Compliance: Ensuring that the development and deployment of AI systems adhere to the regulations outlined in the General Data Protection Regulation, which protects individuals’ privacy rights in the European Union.
- AI Governance: The set of rules, policies, and frameworks that regulate the development and deployment of AI systems.
- XAI (Explainable AI): Techniques and methods used to make the outputs and decision-making processes of AI models understandable and transparent to humans.
- LLMOps: A set of practices and tools for managing and optimizing the entire lifecycle of LLM deployment, including development, training, deployment, monitoring, and maintenance.
- Alignment: The process of ensuring that the behavior and outputs of an LLM are consistent with human values, intentions, and ethical principles.
- Model Ethics: Principles and guidelines that promote ethical behavior (transparency, fairness, accountability, etc.) when deploying AI models, especially those that are publicly facing.
- PII (Personally Identifiable Information): Any information that can be used to identify an individual. Handling PII requires careful processes and user consent.
- Privacy-preserving AI: Techniques and methods used to train and utilize LLMs while safeguarding the privacy of sensitive data.
- Adversarial Defense: Methods and techniques designed to protect LLMs against malicious attempts to manipulate their behavior or exploit vulnerabilities.
- Prompt Injection: A type of adversarial attack where carefully crafted inputs are used to trick an LLM into deviating from its intended purpose or revealing sensitive information.
- Adversarial Attacks: Deliberate attempts to manipulate LLMs through crafted inputs, causing them to produce incorrect, unexpected, or harmful outputs.
- Jailbreaking: A type of adversarial attack that attempts to bypass the safety measures and constraints of an LLM to make it generate unsafe or prohibited content.
- Red-Teaming: A security assessment process involving simulated adversarial attacks to identify vulnerabilities and weaknesses in LLM systems.
- Prompt Leaking: An adversarial technique that tricks an LLM into revealing parts of its original prompt or internal workings.
- Robustness: The ability of an LLM to maintain its performance and accuracy even when encountering noisy, unexpected, or adversarial inputs.
- Black-Box Attacks: Adversarial attacks where the attacker has no knowledge of the LLM’s internal architecture or parameters and can only interact with it through its input and output.
- White-Box Attacks: Adversarial attacks where the attacker has full knowledge of the LLM’s internal architecture, parameters, and training data.
- Vulnerability: A weakness or flaw in an LLM system that can be exploited for malicious purposes, such as adversarial attacks or data breaches.
- Deep-fakes: Synthetic media (images, videos, audio) generated by AI models, often used to create realistic but fake content.
- Watermarking: Embedding hidden, detectable markers into LLM-generated content to identify its origin and potentially combat the spread of misinformation.
- Unsupervised Learning: A machine learning paradigm where models learn patterns and structures from unlabeled data without explicit guidance or correct answers.
- Supervised Learning: A machine learning paradigm where models learn from labeled data, associating inputs with their corresponding correct outputs.
- Reinforcement Learning: A machine learning paradigm where an agent learns through trial and error by interacting with an environment and receiving rewards or penalties based on its actions.
- Federated Learning: A decentralized machine learning approach where models are trained across multiple devices or organizations without sharing the raw data.
- Online Learning: A learning paradigm where a model continuously learns from a stream of incoming data, updating its knowledge in real-time.
- Continual Learning: A learning paradigm focused on enabling models to learn from a sequence of tasks or data without forgetting previously learned knowledge.
- Multi-task Learning: A learning approach where a single model is trained to perform multiple different tasks, often leveraging shared knowledge between related tasks to improve performance.
- Adversarial Learning: A learning paradigm that involves training models against adversarial examples or competing models to improve their robustness and ability to generalize.
- Active Learning: A learning approach where the model strategically selects the most informative data points for human labeling to improve learning efficiency.
- Meta-Learning: Also known as “learning to learn,” this paradigm focuses on training models to acquire general knowledge and learning skills that can be quickly applied to new, unseen tasks with minimal data.
Quiz and Answer Key
Explain the core functionality of a Foundation Model and provide a key characteristic that distinguishes this type of LLM.
A Foundation Model is an LLM designed to generate and understand human-like text across a wide range of use-cases. A key characteristic is its broad pre-training on massive datasets, enabling it to perform diverse tasks with minimal or no task-specific fine-tuning.
Describe the process of Fine-Tuning an LLM. What is the primary goal of this process?
Fine-tuning is the process of adapting a pre-trained LLM to a specific task or domain by further training it on task-specific data. The primary goal is to improve the LLM’s performance and accuracy on the targeted application.
What is Retrieval-Augmented Generation (RAG)? Briefly outline the roles of the Knowledge Base and Vector Database in this process.
Retrieval-Augmented Generation (RAG) is a framework that enhances LLM responses by retrieving relevant information from an external Knowledge Base and appending it to the prompt. The Knowledge Base is a collection of documents, while the Vector Database stores vector representations of this KB to enable efficient similarity-based retrieval.
Differentiate between Zero-Shot Learning and Few-Shot Learning in the context of prompting LLMs for specific tasks.
In Zero-Shot Learning, an LLM is given only task instructions and must rely solely on its pre-existing knowledge to perform the task. In contrast, Few-Shot Learning provides the LLM with a very small number of examples alongside the task instructions to guide its output generation.
Define Instruction Tuning and explain how it aims to improve the behavior of an LLM.
Instruction Tuning involves adjusting an LLM’s behavior during fine-tuning by providing specific instructions along with the training data. This process aims to improve the LLM’s ability to follow instructions and generate more accurate and relevant responses based on those instructions.
What is Hallucination in the context of LLMs? Provide a brief example of what this might look like.
Hallucination in LLMs refers to the tendency of these models to sometimes generate incorrect, nonsensical, or factually inconsistent information that is not grounded in the provided context or their training data. An example could be an LLM generating a fictitious historical event or attributing a quote to the wrong person.
Explain the concept of Context Length and why it is a significant factor in LLM performance.
Context Length is the maximum number of input words or tokens that an LLM can consider when generating an output. It is significant because it limits the amount of information the LLM can process at once, impacting its ability to understand long documents or maintain context over extended conversations.
Describe In-Context Learning and how it differs from traditional fine-tuning methods.
In-Context Learning involves integrating task examples directly into the prompts provided to an LLM, enabling it to understand and handle new tasks without requiring explicit fine-tuning of its weights. This approach leverages the LLM’s pre-existing knowledge and its ability to learn from the provided examples within the prompt itself.
What is Reinforcement Learning from Human Feedback (RLHF)? Briefly explain the role of Reward Modeling in this process.
Reinforcement Learning from Human Feedback (RLHF) is a technique that uses human feedback as a reward or penalty signal to further train an LLM and align its behavior with human preferences. Reward Modeling is a key component where a separate model is trained to predict the human preference score for different LLM outputs, which then serves as the reward signal during reinforcement learning.
Explain the concept of Prompt Injection and why it is considered a security vulnerability for LLMs.
Prompt Injection refers to deliberate attempts to trick LLMs with carefully crafted inputs that manipulate the model’s original instructions and cause it to perform unintended or malicious tasks. This is a vulnerability because it can be exploited to bypass safety measures, extract sensitive information, or generate harmful content.