Generative AI: AI Agents Using RAG and LangChain

Wednesday, April 22, 2026
A+ A- Print Email


  • RAG is an AI framework that helps optimize the output of large language models or LLMs.

  • RAG combines retrieved information and generates natural language to create responses.

  • RAG consists of two main components: the retriever, the core of RAG, and the generator, which functions as a chatbot.

  • In RAG process: 

    • The retriever encodes user-provided prompts and relevant documents into vectors, stores them in a vector database, and retrieves relevant context vectors based on the distance between the encoded prompt and documents. 

    • The generator then combines the retrieved context with the original prompt to produce a response.  

  • The Dense Passage Retrieval (or DPR) Context Encoder and its tokenizer focus on encoding potential answer passages or documents. This encoder creates embeddings from extensive texts, allowing the system to compare these with question embeddings to find the best match.

  • Facebook AI Similarity Search, also known as Faiss, is a library developed by Facebook AI Research that offers efficient algorithms for searching through large collections of high-dimensional vectors.

  • Faiss is essentially a tool to calculate the distance between the question embedding and the vector database of context vector embeddings.

  • The DPR question encoder and its tokenizer focus on encoding the input questions into fixed-dimensional vector representations, grasping their meaning and context to facilitate answering them.

  • The chatbot generates responses based on the questions; however, it is challenging to generate responses for specific domains, such as the company’s mobile policy.

  • RAG process involves encoding prompts into vectors, storing them, and retrieving the relevant ones to produce a response.

  • The DPR Context Encoder and its tokenizer encode the potential answer passages or documents.

  • FAISS is a library developed by Facebook AI Research for searching through large collections of high-dimensional vectors.

  • In-context learning is a technique that incorporates task-specific examples into the prompt to boost model performance.

  • Prompt engineering enhances the effectiveness and accuracy of LLMs by designing prompts that ensure relevant responses without continual fine-tuning.

  • Advanced prompt engineering methods like zero-shot, few-shot, Chain-of-Thought prompting, and self-consistency enhance LLM interactions.

  • Tools like LangChain and agents can facilitate effective prompt creation and enable complex, multi-domain tasks.

  • LangChain is an open-source interface that simplifies the application development process using LLMs.

  • The ‘Document object’ in LangChain serves as a container for data information, including two key attributes, such as page_content and metadata.

  • The LangChain document loader handles various document types, such as HTML, PDF, and code from various locations.

  • Chains in LangChain enable sequential processing, where the output from one step becomes the input for the next, streamlining the prompt generation and processing workflow.

  • Agents in LangChain dynamically sequence actions, integrating with external tools like search engines and databases to fulfill complex user requests.

  • LangChain provides an environment for building and integrating large language model (LLM) applications into external data sets and workflow.

  • LangChain simplifies the integration of language models like GPT-4 and makes it accessible for developers to build natural language processing or NLP applications.

  • The components of LangChain are:

    • Chains, agents, and retriever

    • LangChain-Core

    • LangChain-Community

  • Generative models understand and capture the underlying patterns and data distribution to resemble the given data sets. Generative models are applicable in generating images, text, and music, augmenting data, discovering drugs, and detecting anomalies.

  • Types of generative models are:

    • Gaussian mixture models (GMMs)

    • Hidden Markov models (HMMs)

    • Restricted Boltzmann machines (RBMs)

    • Variational autoencoders (VAEs)

    • Generative adversarial networks (GANs)

    • Diffusion models

  • In-context learning is a method of prompt engineering where task demonstrations are provided to the model as part of the prompt.

  • Prompts are inputs given to an LLM to guide it toward performing a specific task. They consist of instructions and context.

  • Prompt engineering is a process where you design and refine the prompts to get relevant and accurate responses from AI.

  • Prompt engineering has several advantages:

    • It boosts the effectiveness and accuracy of LLMs.

    • It ensures relevant responses.

    • It facilitates meeting user expectations.

    • It eliminates the need for continual fine-tuning.

  • A prompt consists of four key elements: instructions, context, input data, and output indicator.

  • Advanced methods for prompt engineering are: zero-shot prompting, few-shot prompting, chain-of-thought prompting, and self-consistency.

  • Prompt engineering tools facilitate interactions with LLMs. 

  • LangChain uses “prompt templates,” which are predefined recipes for generating effective prompts for LLMs 

  • An agent is a key component in prompt applications that can perform complex tasks across various domains using different prompts.

  • The language models in LangChain use text input to generate text output. 

  • The chat model understands the questions or prompts and responds like a human.

  • The chat model handles various chat messages, such as:

    • HumanMessage

    • AIMessage

    • SystemMessage

    • FunctionMessage

    • ToolMessage

  • The prompt templates in LangChain translate the questions or messages into clear instructions.

  • An example selector instructs the model for the inserted context and guides the LLM to generate the desired output. 

  • Output parsers transform the output from an LLM into a suitable format.

  • LangChain facilitates comprehensive tools for retrieval-augmented generation (RAG) applications, focusing on the retrieval step to ensure sufficient data fetching. 

  • The “Document object” in LangChain serves as a container for data information, including two key attributes, such as page_content and metadata.

  • The LangChain document loader handles various document types, such as HTML, PDF, and code, from various locations.

  • LangChain in document retrieves relevant isolated sections from the documents by splitting them into manageable pieces. 

  • LangChain embeds documents and facilitates various retrievers. 

  • LangChain is a platform that embeds APIs for developing applications. 

  • Chains in the LangChain is a sequence of calls. In chains, the output from one step becomes the input for the next step. 

  • In LangChain, chains first define the template string for the prompt, then create a PromptTemplate using the defined template and create an LLMChain object name.  

  • In LangChain, memory storage is important for reading and writing historical data. 

  • Agents in LangChain are dynamic systems where a language model determines and sequences actions, such as predefined chains. 

  • Agents integrate with tools such as search engines, databases, and websites to fulfill user requests.


Standard architecture of a Retrieval-Augmented Generation (RAG) system, the correct pipeline is:

Chunk → Embed → Index → Retrieve → Generate

Why this is the correct pipeline

This sequence represents the complete lifecycle of data in a RAG system, from raw text to a final AI answer.

  1. Chunk (Segmentation): Large knowledge documents (like PDFs or manuals) are split into smaller, manageable pieces called "chunks" (e.g., 300–500 words). This ensures that the context passed to the AI is precise and fits within its processing limits.

  2. Embed (Vectorization): These text chunks are passed through an embedding model, which converts them into numerical vectors (lists of numbers) that represent the semantic meaning of the text.

    • Note: This specific step is the "converting into a vectorized form" mentioned in your question.

  3. Index (Storage): The resulting vectors are stored in a vector database (or index) to allow for ultra-fast similarity searching later.

  4. Retrieve: When a user asks a question, the system searches the index to find the vectors most similar to the user's query.

  5. Generate: The retrieved text chunks are combined with the user's question and sent to the Large Language Model (LLM) to generate a final, accurate response.

































































Glossary: Fundamentals of Building AI Agents using RAG and LangChain:

TermDefinition
Bidirectional and Auto-Regressive Transformers (BART)Sequence-to-sequence large language model (LLM) that follows an encoder-decoder architecture. It leverages encoding for contextual understanding and decoding to generate text.
Bidirectional Representation of Transformers (BERT)An open-source, deeply bidirectional, unsupervised language representation pretrained using a plain text corpus.
Bradley-Terry modelA probability model for the outcome of pairwise comparisons between items, teams, or objects.
Chain-of-thought (CoT)An AI technique that simulates human-like reasoning by breaking down complex tasks into logical steps.
Chat modelA model designed for efficient conversations. It means that it understands the questions or prompts and responds to them like a human.
Context encoderA neural network architecture used for image inpainting.
Contextual embeddingsA type of embedding that aptly describes how the transformer processes the input word embeddings by accounting for the context in which each word occurs within the sequence.
Data leakageAn organization faces challenges in exposing sensitive information.
Dense Passage Retrieval (DPR)A set of tools that fetches relevant passages with respect to the question asked based on the similarity between the high-quality, low-dimensional continuous representation of passages and questions.
Facebook AI Similarity Search (Faiss)It is a library developed by Facebook AI Research that offers efficient algorithms for searching through large collections of high-dimensional vectors.
Faiss indexA data structure that facilitates efficient similarities between vector searches.
Few-shot promptA technique where the model provides a small number of examples, usually between two and five, to adapt new examples from the previous objects.
Fine-tuningA supervised process that optimizes the initially trained GPT model for specific tasks, like QA classification.
Generative pre-trained transformer (GPT)A self-supervised model that involves training a decoder to predict the subsequent token or word in a sequence.
GitHubA developer platform to create, store, manage, and share codes.
Graphic processing unit (GPU)A process that helps to render graphic smoothly.
Hugging FacePlatform that offers an open-source library with pretrained models and tools to streamline the process of training and fine-tuning generative AI models.
In-Context learningA technique in which task demonstrations are integrated into the prompt in a natural language format.
LangChainAn open-source interface that simplifies the application development process using LLMs. It facilitates a structured way to integrate language models into various use cases, including natural language processing or NLP and data retrieval.
LangChain-CoreA LangChain Expression Language and is the base for abstractions.
LangChain chainsSequences of calls
Language modelA model that predicts words by analyzing the previous text, where context length acts as a hyperparameter.
Large language models (LLMs)Foundation models that use AI and deep learning with vast data sets to generate text, translate languages, and create various types of content. They are called large language models due to the size of the training data set and the number of parameters.
Machine learningMachine learning is a data analysis method for automating analytical model building.
Model inferenceIn machine learning, model inference refers to the operationalization of a trained ML model.
Natural language processing (NLP)The subfield of artificial intelligence (AI) that deals with the interaction of computers and humans in human language. It involves creating algorithms and models that will help computers understand and comprehend human language and generate contextually relevant text in human language.
Prompt engineeringA process of creating effective prompts to enable AI models to generate responses based on the given inputs.
Prompt templateA predefined structure or a format that can be filled with specific content to generate prompts.
PythonA programming language.
PyTorchA software-based open-source deep learning framework used to build neural networks, combining Torch's machine learning library with a Python-based high-level API.
PyTorch tensorsA fundamental data structure that is useful to represent a multi-dimensional array.
Retrieval-augmented generation (RAG)RAG is an AI framework that helps optimize the output of large language models or LLMs. RAG uses the capabilities of LLMs in specific domains or the internal database of an organization without retraining the model.
Scoring functionMeasures the summary for the evaluation of the point prediction. It means it predicts a property or a function.
Self-consistencyA technique for enhancing the reliability and accuracy of outputs.
TokenizationThe process of converting the words in the prompt into tokens.
Text classifierA machine learning technique that assigns a set of predefined categories to open-ended text.
Vector averagingProcess of calculating mean vector from a set of vectors.
watsonx.aiA platform that allows developers to leverage a wide range of large language models (LLMs) under IBM's own series.
WatsonxLLMA wrapper of IBM watsonx.ai foundation models.
Zero-shot promptA prompt in natural language processing (NLP) where a model can generate results for tasks that have not been trained explicitly.

 

Cheat Sheet: Fundamentals of Building AI Agents using RAG and LangChain

Package/MethodDescriptionCode example
Generate textThis code snippet generates text sequences based on the input and doesn't compute the gradient to generate output.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  1. # Generate text
  2. output_ids = model.generate(
  3.     inputs.input_ids, 
  4.     attention_mask=inputs.attention_mask,
  5.     pad_token_id=tokenizer.eos_token_id,
  6.     max_length=50, 
  7.     num_return_sequences=1
  8. )
  9. output_ids
  10. or
  11. with torch.no_grad():
  12.     outputs = model(**inputs) 
  13. outputs
formatting_prompts_func_no_response functionThe prompt function generates formatted text prompts from a data set by using the instructions from the data set. It creates strings that include only the instruction and a placeholder for the response.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  1. def formatting_prompts_func(mydataset):
  2.     output_texts = []
  3.     for i in range(len(mydataset['instruction'])):
  4.         text = (
  5.             f"### Instruction:\n{mydataset['instruction'][i]}"
  6.             f"\n\n### Response:\n{mydataset['output'][i]}"
  7.         )
  8.         output_texts.append(text)
  9.     return output_texts
  10. def formatting_prompts_func_no_response(mydataset):
  11.     output_texts = []
  12.     for i in range(len(mydataset['instruction'])):
  13.         text = (
  14.             f"### Instruction:\n{mydataset['instruction'][i]}"
  15.             f"\n\n### Response:\n"
  16.         )
  17.         output_texts.append(text)
  18.     return output_texts
torch.no_grad()This code snippet helps generate text sequences from the pipeline function. It ensures that the gradient computations are disabled and optimizes the performance and memory usage.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  1. with torch.no_grad():
  2.     # Due to resource limitation, only apply the function on 3 records using "instructions_torch[:10]"
  3.     pipeline_iterator= gen_pipeline(instructions_torch[:3],
  4.                                 max_length=50, # this is set to 50 due to resource constraint, using a GPU, you can increase it to the length of your choice
  5.                                 num_beams=5,
  6.                                 early_stopping=True,)
  7. generated_outputs_lora = []
  8. for text in pipeline_iterator:
  9.     generated_outputs_lora.append(text[0]["generated_text"])
mixtral-8x7b-instruct-v01 watsonx.ai inference model objectAdjusts the parameters to push the limits of creativity and response length.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  1. model_id = 'mistralai/mixtral-8x7b-instruct-v01'
  2. parameters = {
  3.     GenParams.MAX_NEW_TOKENS: 256,  # this controls the maximum number of tokens in the generated output
  4.     GenParams.TEMPERATURE: 0.5, # this randomness or creativity of the model's responses
  5. }
  6. credentials = {
  7.     "url": "https://us-south.ml.cloud.ibm.com"
  8. }
  9. project_id = "skills-network"
  10. model = ModelInference(
  11.     model_id=model_id,
  12.     params=parameters,
  13.     credentials=credentials,
  14.     project_id=project_id
  15. )
String prompt templatesUsed to format a single string and are generally used for simpler inputs.
  1. 1
  2. 2
  3. 3
  4. 4
  1. from langchain_core.prompts import PromptTemplate
  2. prompt = PromptTemplate.from_template("Tell me one {adjective} joke about {topic}")
  3. input_ = {"adjective": "funny", "topic": "cats"}  # create a dictionary to store the corresponding input to placeholders in prompt template
  4. prompt.invoke(input_)
Chat prompt templatesUsed to format a list of messages. These "templates" consist of a list of templates themselves.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  1. from langchain_core.prompts import ChatPromptTemplate
  2. prompt = ChatPromptTemplate.from_messages([
  3.     ("system", "You are a helpful assistant"),
  4.     ("user", "Tell me a joke about {topic}")
  5. ])
  6. input_ = {"topic": "cats"}
  7. prompt.invoke(input_)
Messages place holderThis prompt template is responsible for adding a list of messages in a particular place. But if you want the user to pass in a list of messages that you would slot into a particular spot, the given code snippet is helpful.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  1. from langchain_core.prompts import MessagesPlaceholder
  2. from langchain_core.messages import HumanMessage
  3. prompt = ChatPromptTemplate.from_messages([
  4.     ("system", "You are a helpful assistant"),
  5.     MessagesPlaceholder("msgs")
  6. ])
  7. input_ = {"msgs": [HumanMessage(content="What is the day after Tuesday?")]}
  8. prompt.invoke(input_)
Example selectorIf you have many examples, you may need to select which ones to include in the prompt. The Example Selector is the class responsible for doing so.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  1. from langchain_core.example_selectors import LengthBasedExampleSelector
  2. from langchain_core.prompts import FewShotPromptTemplate, PromptTemplate
  3. # Examples of a pretend task of creating antonyms.
  4. examples = [
  5.     {"input": "happy", "output": "sad"},
  6.     {"input": "tall", "output": "short"},
  7.     {"input": "energetic", "output": "lethargic"},
  8.     {"input": "sunny", "output": "gloomy"},
  9.     {"input": "windy", "output": "calm"},
  10. ]
  11. example_prompt = PromptTemplate(
  12.     input_variables=["input", "output"],
  13.     template="Input: {input}\nOutput: {output}",
  14. )
  15. example_selector = LengthBasedExampleSelector(
  16.     examples=examples,
  17.     example_prompt=example_prompt,
  18.     max_length=25,  # The maximum length that the formatted examples should be.
  19. )
  20. dynamic_prompt = FewShotPromptTemplate(
  21.     example_selector=example_selector,
  22.     example_prompt=example_prompt,
  23.     prefix="Give the antonym of every input",
  24.     suffix="Input: {adjective}\nOutput:",
  25.     input_variables=["adjective"],
  26. )
JSON parserThis output parser allows users to specify an arbitrary JSON schema and query LLMs for outputs that conform to that schema.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  1. from langchain_core.output_parsers import JsonOutputParser
  2. from langchain_core.pydantic_v1 import BaseModel, Field
  3. # Define your desired data structure.
  4. class Joke(BaseModel):
  5.     setup: str = Field(description="question to set up a joke")
  6.     punchline: str = Field(description="answer to resolve the joke")
  7. # And a query intented to prompt a language model to populate the data structure.
  8. joke_query = "Tell me a joke."
  9. # Set up a parser + inject instructions into the prompt template.
  10. output_parser = JsonOutputParser(pydantic_object=Joke)
  11. format_instructions = output_parser.get_format_instructions()
  12. prompt = PromptTemplate(
  13.     template="Answer the user query.\n{format_instructions}\n{query}\n",
  14.     input_variables=["query"],
  15.     partial_variables={"format_instructions": format_instructions},
  16. )
  17. chain = prompt | mixtral_llm | output_parser
  18. chain.invoke({"query": joke_query})
Comma separated list parserThis output parser can be used when you want to return a list of comma-separated items.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  1. from langchain.output_parsers import CommaSeparatedListOutputParser
  2. output_parser = CommaSeparatedListOutputParser()
  3. format_instructions = output_parser.get_format_instructions()
  4. prompt = PromptTemplate(
  5.     template="Answer the user query. {format_instructions}\nList five {subject}.",
  6.     input_variables=["subject"],
  7.     partial_variables={"format_instructions": format_instructions},
  8. )
  9. chain = prompt | mixtral_llm | output_parser
Document objectContains information about some data in LangChain. It has two attributes: page_content: str: This attribute holds the content of the document.
metadata: dict: This attribute contains arbitrary metadata associated with the document. It can be used to track various details such as the document id, file name, and so on.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  1. from langchain_core.documents import Document
  2. Document(page_content="""Python is an interpreted high-level general-purpose programming language. 
  3.                         Python's design philosophy emphasizes code readability with its notable use of significant indentation.""",
  4.          metadata={
  5.              'my_document_id' : 234234,
  6.              'my_document_source' : "About Python",
  7.              'my_document_create_time' : 1680013019
  8.          })
text_splitterAt a high level, text splitters work as follows:
• Split the text into small, semantically meaningful chunks (often sentences).
• Start combining these small chunks into a larger chunk until you reach a certain size (as measured by some function).
• Once you reach that size, make that chunk its own piece of text and start creating a new chunk with some overlap (to keep context between chunks).
  1. 1
  2. 2
  3. 3
  1. text_splitter = CharacterTextSplitter(chunk_size=200, chunk_overlap=20, separator="\n")  # define chunk_size which is length of characters, and also separator.
  2. chunks = text_splitter.split_documents(document)
  3. print(len(chunks))
Embedding modelsEmbedding models are specifically designed to interface with text embeddings. Embeddings generate a vector representation for a given piece of text. This is advantageous as it allows you to conceptualize text within a vector space. Consequently, you can perform operations such as semantic search, where you identify pieces of text that are most similar within the vector space.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  1. from ibm_watsonx_ai.metanames import EmbedTextParamsMetaNames
  2. embed_params = {
  3.     EmbedTextParamsMetaNames.TRUNCATE_INPUT_TOKENS: 3,
  4.     EmbedTextParamsMetaNames.RETURN_OPTIONS: {"input_text": True},
  5. }
  6. from langchain_ibm import WatsonxEmbeddings
  7. watsonx_embedding = WatsonxEmbeddings(
  8.     model_id="ibm/slate-125m-english-rtrvr",
  9.     url="https://us-south.ml.cloud.ibm.com",
  10.     project_id="skills-network",
  11.     params=embed_params,
  12. )
Vector store-backed retrieverA retriever that uses a vector store to retrieve documents. It is a lightweight wrapper around the vector store class to make it conform to the retriever interface. It uses the search methods implemented by a vector store, like similarity search and MMR (maximum marginal relevance), to query the texts in the vector store. Since we've constructed a vector store docsearch, it's very easy to construct a retriever.
  1. 1
  2. 2
  3. 3
  1. retriever = docsearch.as_retriever()
  2. docs = retriever.invoke("Langchain")A vector store retriever is a retriever that uses a vector store to retrieve documents. It is a lightweight wrapper around the vector store class to make it conform to the retriever interface. It uses the search methods implemented by a vector store, like similarity search and MMR (Maximum marginal relevance), to query the texts in the vector store.
  3. Since we've constructed a vector store docsearch, it's very easy to construct a retriever.
ChatMessageHistory classOne of the core utility classes underpinning most (if not all) memory modules is the ChatMessageHistory class. This super lightweight wrapper provides convenient methods for saving HumanMessages, AIMessages, and then fetching them all.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  1. from langchain.memory import ChatMessageHistory
  2. chat = mixtral_llm
  3. history = ChatMessageHistory()
  4. history.add_ai_message("hi!")
  5. history.add_user_message("what is the capital of France?")
langchain.chainsThis code snippet uses a LangChain, library for building language model applications, creating a chain to generate popular dish recommendations based on the specified locations. It also configures model inference settings for further processing.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  1. from langchain.chains import LLMChain
  2. template = """Your job is to come up with a classic dish from the area that the users suggests.
  3.                 {location}
  4.                 YOUR RESPONSE:
  5. """
  6. prompt_template = PromptTemplate(template=template, input_variables=['location'])
  7. # chain 1
  8. location_chain = LLMChain(llm=mixtral_llm, prompt=prompt_template, output_key='meal')
  9. location_chain.invoke(input={'location':'China'})
Simple sequential chainSequential chains allow the output of one LLM to be used as the input for another. This approach is beneficial for dividing tasks and maintaining the focus of your LLM.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  1. from langchain.chains import SequentialChain
  2. template = """Given a meal {meal}, give a short and simple recipe on how to make that dish at home.
  3.                 YOUR RESPONSE:
  4. """
  5. prompt_template = PromptTemplate(template=template, input_variables=['meal'])
  6. # chain 2
  7. dish_chain = LLMChain(llm=mixtral_llm, prompt=prompt_template, output_key='recipe')
  8. template = """Given the recipe {recipe}, estimate how much time I need to cook it.
  9.                 YOUR RESPONSE:
  10. """
  11. prompt_template = PromptTemplate(template=template, input_variables=['recipe'])
  12. # chain 3
  13. recipe_chain = LLMChain(llm=mixtral_llm, prompt=prompt_template, output_key='time')
  14. # overall chain
  15. overall_chain = SequentialChain(chains=[location_chain, dish_chain, recipe_chain],
  16.                                       input_variables=['location'],
  17.                                       output_variables=['meal', 'recipe', 'time'],
  18.                                       verbose= True)
load_summarize_chainThis code snippet uses LangChain library for loading and using a summarization chain with a specific language model and chain type. This chain type will be applied to web data to print a resulting summary.
  1. 1
  2. 2
  3. 3
  4. 4
  1. from langchain.chains.summarize import load_summarize_chain
  2. chain = load_summarize_chain(llm=mixtral_llm, chain_type="stuff", verbose=False)
  3. response = chain.invoke(web_data)
  4. print(response['output_text'])n
TextClassifierRepresents a simple text classifier that uses an embedding layer, a hidden linear layer with a ReLU avtivation, and an output linear layer. The constructor takes the following arguments: num_class: The number of classes to classify. freeze: Whether to freeze the embedding layer.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  1. from torch import nn
  2. class TextClassifier(nn.Module):
  3.     def __init__(self, num_classes,freeze=False):
  4.         super(TextClassifier, self).__init__()
  5.         self.embedding = nn.Embedding.from_pretrained(glove_embedding.vectors.to(device),freeze=freeze)
  6.         # An example of adding additional layers: A linear layer and a ReLU activation
  7.         self.fc1 = nn.Linear(in_features=100, out_features=128)
  8.         self.relu = nn.ReLU()
  9.         # The output layer that gives the final probabilities for the classes
  10.         self.fc2 = nn.Linear(in_features=128, out_features=num_classes)
  11.     def forward(self, x):
  12.         # Pass the input through the embedding layer
  13.         x = self.embedding(x)
  14.         # Here you can use a simple mean pooling
  15.         x = torch.mean(x, dim=1)
  16.         # Pass the pooled embeddings through the additional layers
  17.         x = self.fc1(x)
  18.         x = self.relu(x)
  19.         return self.fc2(x)
Train the modelThis code snippet outlines the function to train a machine learning model using PyTorch. This function trains the model over a specified number of epochs, tracks them, and evaluates the performance on the data set.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  1. def train_model(model, optimizer, criterion, train_dataloader, valid_dataloader, epochs=100, model_name="my_modeldrop"):
  2.     cum_loss_list = []
  3.     acc_epoch = []
  4.     best_acc = 0
  5.     file_name = model_name
  6.     for epoch in tqdm(range(1, epochs + 1)):
  7.         model.train()
  8.         cum_loss = 0
  9.         for _, (label, text) in enumerate(train_dataloader):            
  10.             optimizer.zero_grad()
  11.             predicted_label = model(text)
  12.             loss = criterion(predicted_label, label)
  13.             loss.backward()
  14.             torch.nn.utils.clip_grad_norm_(model.parameters(), 0.1)
  15.             optimizer.step()
  16.             cum_loss += loss.item()
  17.         #print("Loss:", cum_loss)
  18.         cum_loss_list.append(cum_loss)
  19.         acc_val = evaluate(valid_dataloader, model, device)
  20.         acc_epoch.append(acc_val)
  21.         if acc_val > best_acc:
  22.             best_acc = acc_val
  23.             print(f"New best accuracy: {acc_val:.4f}")
  24.             #torch.save(model.state_dict(), f"{model_name}.pth")
  25.     #save_list_to_file(cum_loss_list, f"{model_name}_loss.pkl")
  26.     #save_list_to_file(acc_epoch, f"{model_name}_acc.pkl")
llm_modelThis code snippet defines function 'llm_model' for generating text using the language model from the mistral.ai platform, specifically the 'mitral-8x7b-instruct-v01' model. The function helps in customizing generating parameters and interacts with IBM Watson's machine learning services.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  1. def llm_model(prompt_txt, params=None):
  2.     model_id = 'mistralai/mixtral-8x7b-instruct-v01'
  3.     default_params = {
  4.         "max_new_tokens": 256,
  5.         "min_new_tokens": 0,
  6.         "temperature": 0.5,
  7.         "top_p": 0.2,
  8.         "top_k": 1
  9.     }
  10.     if params:
  11.         default_params.update(params)
  12.     parameters = {
  13.         GenParams.MAX_NEW_TOKENS: default_params["max_new_tokens"],  # this controls the maximum number of tokens in the generated output
  14.         GenParams.MIN_NEW_TOKENS: default_params["min_new_tokens"], # this controls the minimum number of tokens in the generated output
  15.         GenParams.TEMPERATURE: default_params["temperature"], # this randomness or creativity of the model's responses
  16.         GenParams.TOP_P: default_params["top_p"],
  17.         GenParams.TOP_K: default_params["top_k"]
  18.     }
  19.     credentials = {
  20.         "url": "https://us-south.ml.cloud.ibm.com"
  21.     }
  22.     project_id = "skills-network"
  23.     model = Model(
  24.         model_id=model_id,
  25.         params=parameters,
  26.         credentials=credentials,
  27.         project_id=project_id
  28.     )
  29.     mixtral_llm = WatsonxLLM(model=model)
  30.     response  = mixtral_llm.invoke(prompt_txt)
  31.     return response
Zero-shot promptZero-shot learning is crucial for testing a model's ability to apply its pre-trained knowledge to new, unseen tasks without additional training. This capability is valuable for gauging the model's generalization skills.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  1. prompt = """Classify the following statement as true or false: 
  2.             'The Eiffel Tower is located in Berlin.'
  3.             Answer:
  4. """
  5. response = llm_model(prompt, params)
  6. print(f"prompt: {prompt}\n")
  7. print(f"response : {response}\n")
One-shot promptOne-shot learning example where the model is given a single example to help guide its translation from English to French. The prompt provides a sample translation pairing, "How is the weather today?" translated to "Comment est le temps aujourd'hui?" This example serves as a guide for the model to understand the task context and desired format. The model is then tasked with translating a new sentence, "Where is the nearest supermarket?" without further guidance.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  1. params = {
  2.     "max_new_tokens": 20,
  3.     "temperature": 0.1,
  4. }
  5. prompt = """Here is an example of translating a sentence from English to French:
  6.             English: “How is the weather today?”
  7.             French: “Comment est le temps aujourd'hui?”
  8.             Now, translate the following sentence from English to French:
  9.             English: “Where is the nearest supermarket?”
  10. """
  11. response = llm_model(prompt, params)
  12. print(f"prompt: {prompt}\n")
  13. print(f"response : {response}\n")
Few-shot promptThis code snippet classifies emotions using a few-shot learning approach. The prompt includes various examples where statements are associated with their respective emotions.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  1. #parameters  `max_new_tokens` to 10, which constrains the model to generate brief responses
  2. params = {
  3.     "max_new_tokens": 10,
  4. }
  5. prompt = """Here are few examples of classifying emotions in statements:
  6.             Statement: 'I just won my first marathon!'
  7.             Emotion: Joy
  8.             Statement: 'I can't believe I lost my keys again.'
  9.             Emotion: Frustration
  10.             Statement: 'My best friend is moving to another country.'
  11.             Emotion: Sadness
  12.             Now, classify the emotion in the following statement:
  13.             Statement: 'That movie was so scary I had to cover my eyes.’
  14. """
  15. response = llm_model(prompt, params)
  16. print(f"prompt: {prompt}\n")
  17. print(f"response : {response}\n")
Chain-of-thought (CoT) promptingThe Chain-of-Thought (CoT) prompting technique, designed to guide the model through a sequence of reasoning steps to solve a problem.

The CoT technique involves structuring the prompt by instructing the model to "Break down each step of your calculation." This encourages the model to include explicit reasoning steps, mimicking human-like problem-solving processes.

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  1. params = {
  2.     "max_new_tokens": 512,
  3.     "temperature": 0.5,
  4. }
  5. prompt = """Consider the problem: 'A store had 22 apples. They sold 15 apples today and got a new delivery of 8 apples. 
  6.             How many apples are there now?’
  7.             Break down each step of your calculation
  8. """
  9. response = llm_model(prompt, params)
  10. print(f"prompt: {prompt}\n")
  11. print(f"response : {response}\n")
Self-consistencyThis code snippet determines the consistent result for age-related problems and generates multiple responses. The 'params' dictionary specifies the maximum number of tokens to generate responses.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  1. params = {
  2.     "max_new_tokens": 512,
  3. }
  4. prompt = """When I was 6, my sister was half of my age. Now I am 70, what age is my sister?
  5.             Provide three independent calculations and explanations, then determine the most consistent result.
  6. """
  7. response = llm_model(prompt, params)
  8. print(f"prompt: {prompt}\n")
  9. print(f"response : {response}\n")
Prompt templateA key concept in LangChain, it helps to translate user input and parameters into instructions for a language model. This can be used to guide a model's response, helping it understand the context and generate relevant and coherent language-based output.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  1. model_id = 'mistralai/mixtral-8x7b-instruct-v01'
  2. parameters = {
  3.     GenParams.MAX_NEW_TOKENS: 256,  # this controls the maximum number of tokens in the generated output
  4.     GenParams.TEMPERATURE: 0.5, # this randomness or creativity of the model's responses
  5. }
  6. credentials = {
  7.     "url": "https://us-south.ml.cloud.ibm.com"
  8. }
  9. project_id = "skills-network"
  10. model = Model(
  11.     model_id=model_id,
  12.     params=parameters,
  13.     credentials=credentials,
  14.     project_id=project_id
  15. )
  16. mixtral_llm = WatsonxLLM(model=model)
  17. mixtral_llm
Text summarizationText summarization agent designed to help summarize the content you provide to the LLM. You can store the content to be summarized in a variable, allowing for repeated use of the prompt.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  1. content = """
  2.         The rapid advancement of technology in the 21st century has transformed various industries, including healthcare, education, and transportation. 
  3.         Innovations such as artificial intelligence, machine learning, and the Internet of Things have revolutionized how we approach everyday tasks and complex problems. 
  4.         For instance, AI-powered diagnostic tools are improving the accuracy and speed of medical diagnoses, while smart transportation systems are making cities more efficient and reducing traffic congestion. 
  5.         Moreover, online learning platforms are making education more accessible to people around the world, breaking down geographical and financial barriers. 
  6.         These technological developments are not only enhancing productivity but also contributing to a more interconnected and informed society.
  7. """
  8. template = """Summarize the {content} in one sentence.
  9. """
  10. prompt = PromptTemplate.from_template(template)
  11. llm_chain = LLMChain(prompt=prompt, llm=mixtral_llm)
  12. response = llm_chain.invoke(input = {"content": content})
  13. print(response["text"])
Question answeringAn agent that enables the LLM to learn from the provided content and answer questions based on what it has learned. Occasionally, if the LLM does not have sufficient information, it might generate a speculative answer. To manage this, you'll specifically instruct it to respond with "Unsure about the answer" if it is uncertain about the correct response.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  1. content = """
  2.         The solar system consists of the Sun, eight planets, their moons, dwarf planets, and smaller objects like asteroids and comets. 
  3.         The inner planets—Mercury, Venus, Earth, and Mars—are rocky and solid. 
  4.         The outer planets—Jupiter, Saturn, Uranus, and Neptune—are much larger and gaseous.
  5. """
  6. question = "Which planets in the solar system are rocky and solid?"
  7. template = """
  8.             Answer the {question} based on the {content}.
  9.             Respond "Unsure about answer" if not sure about the answer.
  10.             Answer:
  11. """
  12. prompt = PromptTemplate.from_template(template)
  13. output_key = "answer"
  14. llm_chain = LLMChain(prompt=prompt, llm=mixtral_llm, output_key=output_key)
  15. response = llm_chain.invoke(input = {"question":question ,"content": content})
  16. print(response["answer"])
Code generationAn agent that is designed to generate SQL queries based on given descriptions. It interprets the requirements from your input and translates them into executable SQL code.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  1. description = """
  2.         Retrieve the names and email addresses of all customers from the 'customers' table who have made a purchase in the last 30 days. 
  3.         The table 'purchases' contains a column 'purchase_date'
  4. """
  5. template = """
  6.             Generate an SQL query based on the {description}
  7.             SQL Query:
  8. """
  9. prompt = PromptTemplate.from_template(template)
  10. output_key = "query"
  11. llm_chain = LLMChain(prompt=prompt, llm=mixtral_llm, output_key=output_key)
  12. response = llm_chain.invoke(input = {"description":description})
  13. print(response["query"])
Role playingConfigures the LLM to assume specific roles as defined by us, enabling it to follow predetermined rules and behave like a task-oriented chatbot.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  1. role = """
  2.         game master
  3. """
  4. tone = "engaging and immersive"
  5. template = """
  6.             You are an expert {role}. I have this question {question}. I would like our conversation to be {tone}.
  7.             Answer:
  8. """
  9. prompt = PromptTemplate.from_template(template)
  10. output_key = "answer"
  11. llm_chain = LLMChain(prompt=prompt, llm=mixtral_llm, output_key=output_key)
class_namesThis code snippet maps numerical labels to their corresponding textual descriptions to classify tasks. This code helps in machine learning to interpret the output model, where the model's predictions are numerical and should be presented in a more human-readable format.
  1. 1
  2. 2
  1. class_names = {0: "negative", 1: "positive"}
  2. class_names
read_and_split_textInvolves opening the file, reading its contents, and splitting the text into individual paragraphs. Each paragraph represents a section of the company policies. You can also filter out any empty paragraphs to clean your data set.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  1. def read_and_split_text(filename):
  2.     with open(filename, 'r', encoding='utf-8') as file:
  3.         text = file.read()
  4.     # Split the text into paragraphs (simple split by newline characters)
  5.     paragraphs = text.split('\n')
  6.     # Filter out any empty paragraphs or undesired entries
  7.     paragraphs = [para.strip() for para in paragraphs if len(para.strip()) > 0]
  8.     return paragraphs
  9. # Read the text file and split it into paragraphs
  10. paragraphs = read_and_split_text('companyPolicies.txt')
  11. paragraphs[0:10]
encode_contextsThis code snippet encodes a list of texts into embeddings using content_tokenizer and context_encoder. This code helps iterate through each text in the input list, tokenizes and encodes it, and then appends the pooler_output to the embeddings list. The resulting embeddings get stored in the context_embeddings variables and generate embeddings from text data for various natural language processing (NLP) applications.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  1. def encode_contexts(text_list):
  2.     # Encode a list of texts into embeddings
  3.     embeddings = []
  4.     for text in text_list:
  5.         inputs = context_tokenizer(text, return_tensors='pt', padding=True, truncation=True, max_length=256)
  6.         outputs = context_encoder(**inputs)
  7.         embeddings.append(outputs.pooler_output)
  8.     return torch.cat(embeddings).detach().numpy()
  9. # you would now encode these paragraphs to create embeddings.
  10. context_embeddings = encode_contexts(paragraphs)
import faissFAISS (Facebook AI Similarity Search) is an efficient library developed by Facebook for similarity search and clustering of dense vectors. FAISS is designed for fast similarity search, which is particularly valuable when dealing with large data sets. It is highly suitable for tasks in natural language processing where retrieval speed is critical. It effectively handles large volumes of data, maintaining performance even as data set sizes increase.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  1. import faiss
  2. # Convert list of numpy arrays into a single numpy array
  3. embedding_dim = 768  # This should match the dimension of your embeddings
  4. context_embeddings_np = np.array(context_embeddings).astype('float32')
  5. # Create a FAISS index for the embeddings
  6. index = faiss.IndexFlatL2(embedding_dim)
  7. index.add(context_embeddings_np)  # Add the context embeddings to the index
search_relevant_contextsThis code snippet is useful in searching relevant contexts for a given question. It tokenizes the question using the question_tokenizer, encodes the question using question_encoder, and searches an index for retrieving the relevant context based on question embedding.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  1. def search_relevant_contexts(question, question_tokenizer, question_encoder, index, k=5):
  2.     """
  3.     Searches for the most relevant contexts to a given question.
  4.     Returns:
  5.     tuple: Distances and indices of the top k relevant contexts.
  6.     """
  7.     # Tokenize the question
  8.     question_inputs = question_tokenizer(question, return_tensors='pt')
  9.     # Encode the question to get the embedding
  10.     question_embedding = question_encoder(**question_inputs).pooler_output.detach().numpy()
  11.     # Search the index to retrieve top k relevant contexts
  12.     D, I = index.search(question_embedding, k)
  13.     return D, I
generate_answer_without_contextThis code snippet generates responses using the entered prompt without requiring additional context. It tokenizes the input questions using the tokenizer, generates the output text using the model, and decodes the generated text to obtain the answer.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  1. def generate_answer_without_context(question):
  2.     # Tokenize the input question
  3.     inputs = tokenizer(question, return_tensors='pt', max_length=1024, truncation=True)
  4.     # Generate output directly from the question without additional context
  5.     summary_ids = model.generate(inputs['input_ids'], max_length=150, min_length=40, length_penalty=2.0, num_beams=4, early_stopping=True)
  6.     # Decode and return the generated text
  7.     answer = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
  8.     return answer
Generating answers with DPR contextsAnswers are generated when the model utilizes contexts retrieved via DPR, which are expected to enhance the answer's relevance and depth:
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  1. def generate_answer(contexts):
  2.     # Concatenate the retrieved contexts to form the input to BART
  3.     input_text = ' '.join(contexts)
  4.     inputs = tokenizer(input_text, return_tensors='pt', max_length=1024, truncation=True)
  5.     # Generate output using BART
  6.     summary_ids = model.generate(inputs['input_ids'], max_length=150, min_length=40, length_penalty=2.0, num_beams=4, early_stopping=True)
  7.     return tokenizer.decode(summary_ids[0], skip_special_tokens=True)
aggregate_embeddings functionThe function aggregate_embeddings takes token indices and their corresponding attention masks, and uses a BERT model to convert these tokens into word embeddings. It then filters out the embeddings for zero-padded tokens and computes the mean embedding for each sequence. This helps in reducing the dimensionality of the data while retaining the most important information from the embeddings.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  1. def aggregate_embeddings(input_ids, attention_masks, bert_model=bert_model):
  2.     """
  3.     Converts token indices and masks to word embeddings, filters out zero-padded embeddings,
  4.     and aggregates them by computing the mean embedding for each input sequence.
  5.     """
  6.     mean_embeddings = []
  7.     # Process each sequence in the batch
  8.     print('number of inputs',len(input_ids))
  9.     for input_id, mask in tqdm(zip(input_ids, attention_masks)):
  10.         input_ids_tensor = torch.tensor([input_id]).to(DEVICE)
  11.         mask_tensor = torch.tensor([mask]).to(DEVICE)
  12.         with torch.no_grad():
  13.             # Obtain the word embeddings from the BERT model
  14.             word_embeddings = bert_model(input_ids_tensor, attention_mask=mask_tensor)[0].squeeze(0)
  15.             # Filter out the embeddings at positions where the mask is zero 
  16.             valid_embeddings_mask=mask_tensor[0] != 0 
  17.             valid_embeddings = word_embeddings[valid_embeddings_mask,:]
  18.             # Compute the mean of the filtered embeddings
  19.             mean_embedding = valid_embeddings.mean(dim=0)           mean_embeddings.append(mean_embedding.unsqueeze(0))
  20.     # Concatenate the mean embeddings from all sequences in the batch
  21.     aggregated_mean_embeddings = torch.cat(mean_embeddings)
  22.     return aggregated_mean_embeddings
text_to_embDesigned to convert a list of text strings into their corresponding embeddings using a pre-defined tokenizer.
  1. 1
  2. 2
  3. 3
  1. def text_to_emb(list_of_text,max_input=512):
  2.     data_token_index  = tokenizer.batch_encode_plus(list_of_text, add_special_tokens=True,padding=True,truncation=True,max_length=max_input) question_embeddings=aggregate_embeddings(data_token_index['input_ids'], data_token_index['attention_mask'])
  3.     return question_embeddings
process_songConvert both the predefined appropriateness questions and the song lyrics into "RAG embeddings" and measure the similarity between them to determine the appropriateness.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  1. import re
  2. def process_song(song):
  3.     # Remove line breaks from the song
  4.     song_new = re.sub(r'[\n]', ' ', song)
  5.     # Remove single quotes from the song
  6.     song_new = [song_new.replace("\'", "")]
  7.     return song_new
RAG_QAThis code snippet performs question-answering using question embeddings and provides embeddings. It helps reshape the results for processing, sorting the indices in descending order, and printing the top 'n-responses' based on the highest dot product values.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  1. def RAG_QA(embeddings_questions, embeddings, n_responses=3):
  2.     # Calculate the dot product between the question embeddings and the provided embeddings (transpose of the second matrix for proper alignment).
  3.     dot_product = embeddings_questions @ embeddings.T
  4.     # Reshape the dot product results to a 1D tensor for easier processing.
  5.     dot_product = dot_product.reshape(-1)
  6.     # Sort the indices of the dot product results in descending order (setting descending to False should be True for typical similarity tasks).
  7.     sorted_indices = torch.argsort(dot_product, descending=True)
  8.     # Convert sorted indices to a list for easier iteration.
  9.     sorted_indices = sorted_indices.tolist()
  10.     # Print the top 'n_responses' responses from the sorted list, which correspond to the highest dot product values.
  11.     for index in sorted_indices[:n_responses]:
  12.         print(yes_responses[index])
model_name_or_pathThis code snippet defines the model name to 'gpt2' and initializes the token and model using the GPT-2 model. In this code, add special tokens for padding by keeping the maximum sequence length to 1024.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  1. # Define the model name or path
  2. model_name_or_path = "gpt2"
  3. # Initialize tokenizer and model
  4. tokenizer = GPT2Tokenizer.from_pretrained(model_name_or_path, use_fast=True)
  5. model = GPT2ForSequenceClassification.from_pretrained(model_name_or_path, num_labels=1)
  6. # Add special tokens if necessary
  7. tokenizer.pad_token = tokenizer.eos_token
  8. model.config.pad_token_id = model.config.eos_token_id
  9. # Define the maximum length
  10. max_length = 1024
add_combined_columnsThis code snippet combines the prompt with chosen and rejected responses in a data set example. It combines with the 'Human:' and 'Assistant:' for clarity. This function modifies each example in the 'train' split the data set by creating new columns 'prompt_chosen' and 'prompt_rejected' with the combined text.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  1. # Define a function to combine 'prompt' with 'chosen' and 'rejected' responses
  2. def add_combined_columns(example):
  3.     # Combine 'prompt' with 'chosen' response, formatting it with "Human:" and "Assistant:" labels
  4.     example['prompt_chosen'] = "\n\nHuman: " + example["prompt"] + "\n\nAssistant: " + example["chosen"]
  5.     # Combine 'prompt' with 'rejected' response, formatting it with "Human:" and "Assistant:" labels
  6.     example['prompt_rejected'] = "\n\nHuman: " + example["prompt"] + "\n\nAssistant: " + example["rejected"]
  7.     # Return the modified example
  8.     return example
  9. # Apply the function to each example in the 'train' split of the dataset
  10. dataset['train'] = dataset['train'].map(add_combined_columns)
RetrievalQAThis code snippet creates an example for 'RetrievalQA' using a language model and document retriever.
  1. 1
  2. 2
  3. 3
  4. 4
  1. qa = RetrievalQA.from_chain_type(llm=flan_ul2_llm, 
  2.                                  chain_type="stuff",                         retriever=docsearch.as_retriever(),                                 return_source_documents=False)
  3. query = "what is mobile policy?"
  4. qa.invoke(query)
https://pytorch.org/ https://www.ibm.com/topics/natural-language-processing Skills Network Labs Skills Network Labs
Skills Network Labs Skills Network Labs
Skills Network Labs

Contact Form

Name

Email *

Message *

lifehack.guide © 2015. All Rights Reserved.
Powered by Blogger | Shared by JOJOThemes
Powered by Blogger.