Add files for midterm project

=0.2.0

@@ -0,0 +1,29 @@
+Requirement already satisfied: langchain_core in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (0.3.1)
+Requirement already satisfied: langchain_openai in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (0.2.0)
+Requirement already satisfied: PyYAML>=5.3 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from langchain_core) (6.0)
+Requirement already satisfied: jsonpatch<2.0,>=1.33 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from langchain_core) (1.33)
+Requirement already satisfied: langsmith<0.2.0,>=0.1.117 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from langchain_core) (0.1.122)
+Requirement already satisfied: packaging<25,>=23.2 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from langchain_core) (23.2)
+Requirement already satisfied: pydantic<3.0.0,>=2.5.2 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from langchain_core) (2.8.2)
+Requirement already satisfied: tenacity!=8.4.0,<9.0.0,>=8.1.0 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from langchain_core) (8.2.2)
+Requirement already satisfied: typing-extensions>=4.7 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from langchain_core) (4.12.2)
+Requirement already satisfied: openai<2.0.0,>=1.40.0 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from langchain_openai) (1.46.0)
+Requirement already satisfied: tiktoken<1,>=0.7 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from langchain_openai) (0.7.0)
+Requirement already satisfied: jsonpointer>=1.9 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from jsonpatch<2.0,>=1.33->langchain_core) (2.1)
+Requirement already satisfied: httpx<1,>=0.23.0 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from langsmith<0.2.0,>=0.1.117->langchain_core) (0.24.1)
+Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from langsmith<0.2.0,>=0.1.117->langchain_core) (3.10.7)
+Requirement already satisfied: requests<3,>=2 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from langsmith<0.2.0,>=0.1.117->langchain_core) (2.31.0)
+Requirement already satisfied: anyio<5,>=3.5.0 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from openai<2.0.0,>=1.40.0->langchain_openai) (3.5.0)
+Requirement already satisfied: distro<2,>=1.7.0 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from openai<2.0.0,>=1.40.0->langchain_openai) (1.9.0)
+Requirement already satisfied: jiter<1,>=0.4.0 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from openai<2.0.0,>=1.40.0->langchain_openai) (0.5.0)
+Requirement already satisfied: sniffio in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from openai<2.0.0,>=1.40.0->langchain_openai) (1.2.0)
+Requirement already satisfied: tqdm>4 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from openai<2.0.0,>=1.40.0->langchain_openai) (4.65.0)
+Requirement already satisfied: annotated-types>=0.4.0 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from pydantic<3.0.0,>=2.5.2->langchain_core) (0.7.0)
+Requirement already satisfied: pydantic-core==2.20.1 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from pydantic<3.0.0,>=2.5.2->langchain_core) (2.20.1)
+Requirement already satisfied: regex>=2022.1.18 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from tiktoken<1,>=0.7->langchain_openai) (2022.7.9)
+Requirement already satisfied: idna>=2.8 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from anyio<5,>=3.5.0->openai<2.0.0,>=1.40.0->langchain_openai) (3.4)
+Requirement already satisfied: certifi in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from httpx<1,>=0.23.0->langsmith<0.2.0,>=0.1.117->langchain_core) (2023.7.22)
+Requirement already satisfied: httpcore<0.18.0,>=0.15.0 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from httpx<1,>=0.23.0->langsmith<0.2.0,>=0.1.117->langchain_core) (0.17.3)
+Requirement already satisfied: charset-normalizer<4,>=2 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from requests<3,>=2->langsmith<0.2.0,>=0.1.117->langchain_core) (2.0.4)
+Requirement already satisfied: urllib3<3,>=1.21.1 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from requests<3,>=2->langsmith<0.2.0,>=0.1.117->langchain_core) (1.26.16)
+Requirement already satisfied: h11<0.15,>=0.13 in /home/rchrdgwr/anaconda3/lib/python3.11/site-packages (from httpcore<0.18.0,>=0.15.0->httpx<1,>=0.23.0->langsmith<0.2.0,>=0.1.117->langchain_core) (0.14.0)

BuildingAChainlitApp.md

@@ -0,0 +1,312 @@
+# Building a Chainlit App
+
+What if we want to take our Week 1 Day 2 assignment - [Pythonic RAG](https://github.com/AI-Maker-Space/AIE4/tree/main/Week%201/Day%202) - and bring it out of the notebook?
+
+Well - we'll cover exactly that here!
+
+## Anatomy of a Chainlit Application
+
+[Chainlit](https://docs.chainlit.io/get-started/overview) is a Python package similar to Streamlit that lets users write a backend and a front end in a single (or multiple) Python file(s). It is mainly used for prototyping LLM-based Chat Style Applications - though it is used in production in some settings with 1,000,000s of MAUs (Monthly Active Users).
+
+The primary method of customizing and interacting with the Chainlit UI is through a few critical [decorators](https://blog.hubspot.com/website/decorators-in-python).
+
+> NOTE: Simply put, the decorators (in Chainlit) are just ways we can "plug-in" to the functionality in Chainlit. 
+
+We'll be concerning ourselves with three main scopes:
+
+1. On application start - when we start the Chainlit application with a command like `chainlit run app.py`
+2. On chat start - when a chat session starts (a user opens the web browser to the address hosting the application)
+3. On message - when the users sends a message through the input text box in the Chainlit UI
+
+Let's dig into each scope and see what we're doing!
+
+## On Application Start:
+
+The first thing you'll notice is that we have the traditional "wall of imports" this is to ensure we have everything we need to run our application. 
+
+```python
+import os
+from typing import List
+from chainlit.types import AskFileResponse
+from utilities_2.text_utils import CharacterTextSplitter, TextFileLoader
+from utilities_2.openai_utils.prompts import (
+    UserRolePrompt,
+    SystemRolePrompt,
+    AssistantRolePrompt,
+)
+from utilities_2.openai_utils.embedding import EmbeddingModel
+from utilities_2.vectordatabase import VectorDatabase
+from utilities_2.openai_utils.chatmodel import ChatOpenAI
+import chainlit as cl
+```
+
+Next up, we have some prompt templates. As all sessions will use the same prompt templates without modification, and we don't need these templates to be specific per template - we can set them up here - at the application scope. 
+
+```python
+system_template = """\
+Use the following context to answer a users question. If you cannot find the answer in the context, say you don't know the answer."""
+system_role_prompt = SystemRolePrompt(system_template)
+
+user_prompt_template = """\
+Context:
+{context}
+
+Question:
+{question}
+"""
+user_role_prompt = UserRolePrompt(user_prompt_template)
+```
+
+> NOTE: You'll notice that these are the exact same prompt templates we used from the Pythonic RAG Notebook in Week 1 Day 2!
+
+Following that - we can create the Python Class definition for our RAG pipeline - or *chain*, as we'll refer to it in the rest of this walkthrough. 
+
+Let's look at the definition first:
+
+```python
+class RetrievalAugmentedQAPipeline:
+    def __init__(self, llm: ChatOpenAI(), vector_db_retriever: VectorDatabase) -> None:
+        self.llm = llm
+        self.vector_db_retriever = vector_db_retriever
+
+    async def arun_pipeline(self, user_query: str):
+        ### RETRIEVAL
+        context_list = self.vector_db_retriever.search_by_text(user_query, k=4)
+
+        context_prompt = ""
+        for context in context_list:
+            context_prompt += context[0] + "\n"
+
+        ### AUGMENTED
+        formatted_system_prompt = system_role_prompt.create_message()
+
+        formatted_user_prompt = user_role_prompt.create_message(question=user_query, context=context_prompt)
+
+
+        ### GENERATION
+        async def generate_response():
+            async for chunk in self.llm.astream([formatted_system_prompt, formatted_user_prompt]):
+                yield chunk
+
+        return {"response": generate_response(), "context": context_list}
+```
+
+Notice a few things:
+
+1. We have modified this `RetrievalAugmentedQAPipeline` from the initial notebook to support streaming. 
+2. In essence, our pipeline is *chaining* a few events together:
+    1. We take our user query, and chain it into our Vector Database to collect related chunks
+    2. We take those contexts and our user's questions and chain them into the prompt templates
+    3. We take that prompt template and chain it into our LLM call
+    4. We chain the response of the LLM call to the user
+3. We are using a lot of `async` again!
+
+Now, we're going to create a helper function for processing uploaded text files.
+
+First, we'll instantiate a shared `CharacterTextSplitter`.
+
+```python
+text_splitter = CharacterTextSplitter()
+```
+
+Now we can define our helper.
+
+```python
+def process_text_file(file: AskFileResponse):
+    import tempfile
+
+    with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".txt") as temp_file:
+        temp_file_path = temp_file.name
+
+    with open(temp_file_path, "wb") as f:
+        f.write(file.content)
+
+    text_loader = TextFileLoader(temp_file_path)
+    documents = text_loader.load_documents()
+    texts = text_splitter.split_text(documents)
+    return texts
+```
+
+Simply put, this downloads the file as a temp file, we load it in with `TextFileLoader` and then split it with our `TextSplitter`, and returns that list of strings!
+
+<div style="border: 2px solid white; padding: 10px; border-radius: 5px; background-color: black; padding: 10px;">
+QUESTION #1:
+
+Why do we want to support streaming? What about streaming is important, or useful?
+
+### ANSWER #1:
+
+Streaming is the continuous transmission of the data from the model to the UI. Instead of waiting and batching up the response into a single
+large message, the response is sent in pieces (streams) as it is created.
+
+The advantages of streaming:
+- quicker initial response - the user sees the first part of the answer sooner
+- it is easier to identify the results are incorrect and terminate the request
+- it is a more natural mode of communication for humans
+- better handling of large data, not requiring complex caching
+- essential for real time processing
+- humans can only read so fast so its an advantage to get some of the data earlier
+
+</div>
+
+## On Chat Start:
+
+The next scope is where "the magic happens". On Chat Start is when a user begins a chat session. This will happen whenever a user opens a new chat window, or refreshes an existing chat window.
+
+You'll see that our code is set-up to immediately show the user a chat box requesting them to upload a file. 
+
+```python
+while files == None:
+        files = await cl.AskFileMessage(
+            content="Please upload a Text File file to begin!",
+            accept=["text/plain"],
+            max_size_mb=2,
+            timeout=180,
+        ).send()
+```
+
+Once we've obtained the text file - we'll use our processing helper function to process our text!
+
+After we have processed our text file - we'll need to create a `VectorDatabase` and populate it with our processed chunks and their related embeddings!
+
+```python
+vector_db = VectorDatabase()
+vector_db = await vector_db.abuild_from_list(texts)
+```
+
+Once we have that piece completed - we can create the chain we'll be using to respond to user queries!
+
+```python
+retrieval_augmented_qa_pipeline = RetrievalAugmentedQAPipeline(
+        vector_db_retriever=vector_db,
+        llm=chat_openai
+    )
+```
+
+Now, we'll save that into our user session!
+
+> NOTE: Chainlit has some great documentation about [User Session](https://docs.chainlit.io/concepts/user-session). 
+
+<div style="border: 2px solid white; padding: 10px; border-radius: 5px; background-color: black; padding: 10px;">
+
+### QUESTION #2: 
+
+Why are we using User Session here? What about Python makes us need to use this? Why not just store everything in a global variable?
+
+### ANSWER #2:
+The application hopefully will be run by many people, at the same time. If the data was stored in a global variable
+this would be accessed by everyone using the application. So everytime someone started a new session, the information 
+would be overwritten, meaning everyone would basically get the same results. Unless only one person used the system 
+at a time.
+
+So the goal is to keep each users session information separate from all the other users. The ChainLit User session
+provides the capability of storing each users data separately.
+</div>
+
+## On Message
+
+First, we load our chain from the user session:
+
+```python
+chain = cl.user_session.get("chain")
+```
+
+Then, we run the chain on the content of the message - and stream it to the front end - that's it!
+
+```python
+msg = cl.Message(content="")
+result = await chain.arun_pipeline(message.content)
+
+async for stream_resp in result["response"]:
+    await msg.stream_token(stream_resp)
+```
+
+## 🎉
+
+With that - you've created a Chainlit application that moves our Pythonic RAG notebook to a Chainlit application!
+
+## 🚧 CHALLENGE MODE 🚧
+
+For an extra challenge - modify the behaviour of your applciation by integrating changes you made to your Pythonic RAG notebook (using new retrieval methods, etc.)
+
+If you're still looking for a challenge, or didn't make any modifications to your Pythonic RAG notebook:
+
+1) Allow users to upload PDFs (this will require you to build a PDF parser as well)
+2) Modify the VectorStore to leverage [Qdrant](https://python-client.qdrant.tech/)
+
+> NOTE: The motivation for these challenges is simple - the beginning of the course is extremely information dense, and people come from all kinds of different technical backgrounds. In order to ensure that all learners are able to engage with the content confidently and comfortably, we want to focus on the basic units of technical competency required. This leads to a situation where some learners, who came in with more robust technical skills, find the introductory material to be too simple - and these open-ended challenges help us do this! 
+
+## Support pdf documents
+
+Code was modified to support pdf documents in the following areas:
+
+1) Change to the request for documents in on_chat_start:
+
+- changed the message to ask for .txt or .pdf file
+- changed the acceptable file formats so that the pdf documents are included in the select pop up
+
+```python
+    while not files:
+        files = await cl.AskFileMessage(
+            content="Please upload a .txt or .pdf file to begin processing!",
+            accept=["text/plain", "application/pdf"],
+            max_size_mb=2,
+            timeout=180,
+        ).send()
+```
+
+2) change process_text_file() function to handle .pdf files
+
+- refactor the code to do all file handling in utilities.text_utils
+- app calls process_file, optionally passing in the text splitter function
+    - default text splitter function is CharacterTextSplitter
+    ```python
+    texts = process_file(file)
+    ```
+- load_file() function does the following
+    - read the uploaded document into a temporary file
+    - identify the file extension
+    - process a .txt file as before resulting in the texts list
+    - if the file is .pdf use the PyMuPDF library to read each page and extract the text and add it to texts list
+    - use the passed in text splitter function to split the documents
+
+```python
+    def load_file(self, file, text_splitter=CharacterTextSplitter()):
+        file_extension = os.path.splitext(file.name)[1].lower()
+        with tempfile.NamedTemporaryFile(mode="wb", delete=False, suffix=file_extension) as temp_file:
+            self.temp_file_path = temp_file.name
+            temp_file.write(file.content)
+
+        if os.path.isfile(self.temp_file_path):
+            if self.temp_file_path.endswith(".txt"):
+                self.load_text_file()
+            elif self.temp_file_path.endswith(".pdf"):
+                self.load_pdf_file()
+            else:
+                raise ValueError(
+                    f"Unsupported file type: {self.temp_file_path}"
+                )
+            return text_splitter.split_text(self.documents)
+        else:
+            raise ValueError(
+                    "Not a file"
+                )
+
+    def load_text_file(self):
+        with open(self.temp_file_path, "r", encoding=self.encoding) as f:
+            self.documents.append(f.read())
+
+    def load_pdf_file(self):
+
+        pdf_document = fitz.open(self.temp_file_path)
+        for page_num in range(len(pdf_document)):
+            page = pdf_document.load_page(page_num)
+            text = page.get_text()
+            self.documents.append(text)
+```
+
+3) Test the handling of .pdf and .txt files
+
+Several different .pdf and .txt files were successfully uploaded and processed by the app
+
+

Dockerfile

@@ -0,0 +1,11 @@
+FROM python:3.11.9
+RUN useradd -m -u 1000 user
+USER user
+ENV HOME=/home/user \
+    PATH=/home/user/.local/bin:$PATH
+WORKDIR $HOME/app
+COPY --chown=user . $HOME/app
+COPY ./requirements.txt ~/app/requirements.txt
+RUN pip install -r requirements.txt
+COPY . .
+CMD ["chainlit", "run", "app.py", "--port", "7860"]

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Richard Gower
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,11 +1,118 @@
----
-title: AI4Midterm
-emoji: 🐨
-colorFrom: yellow
-colorTo: red
-sdk: docker
-pinned: false
-license: mit
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+---
+title: DeployPythonicRAG
+emoji: 📉
+colorFrom: blue
+colorTo: purple
+sdk: docker
+pinned: false
+license: apache-2.0
+---
+
+# Deploying Pythonic Chat With Your Text File Application
+
+In today's breakout rooms, we will be following the processed that you saw during the challenge - for reference, the instructions for that are available [here](https://github.com/AI-Maker-Space/Beyond-ChatGPT/tree/main).
+
+Today, we will repeat the same process - but powered by our Pythonic RAG implementation we created last week. 
+
+You'll notice a few differences in the `app.py` logic - as well as a few changes to the `utilities_2` package to get things working smoothly with Chainlit.
+
+## Reference Diagram (It's Busy, but it works)
+
+![image](https://i.imgur.com/IaEVZG2.png)
+
+## Deploying the Application to Hugging Face Space
+
+Due to the way the repository is created - it should be straightforward to deploy this to a Hugging Face Space!
+
+> NOTE: If you wish to go through the local deployments using `chainlit run app.py` and Docker - please feel free to do so!
+
+<details>
+    <summary>Creating a Hugging Face Space</summary>
+
+1.  Navigate to the `Spaces` tab.
+
+![image](https://i.imgur.com/aSMlX2T.png)
+
+2. Click on `Create new Space`
+
+![image](https://i.imgur.com/YaSSy5p.png)
+
+3. Create the Space by providing values in the form. Make sure you've selected "Docker" as your Space SDK.
+
+![image](https://i.imgur.com/6h9CgH6.png)
+
+</details>
+
+<details>
+    <summary>Adding this Repository to the Newly Created Space</summary>
+
+1. Collect the SSH address from the newly created Space. 
+
+![image](https://i.imgur.com/Oag0m8E.png)
+
+> NOTE: The address is the component that starts with `[email protected]:spaces/`.
+
+2. Use the command:
+
+```bash
+git remote add hf HF_SPACE_SSH_ADDRESS_HERE
+```
+
+3. Use the command:
+
+```bash
+git pull hf main --no-rebase --allow-unrelated-histories -X ours
+```
+
+4. Use the command: 
+
+```bash 
+git add .
+```
+
+5. Use the command:
+
+```bash
+git commit -m "Deploying Pythonic RAG"
+```
+
+6. Use the command: 
+
+```bash
+git push hf main
+```
+
+7. The Space should automatically build as soon as the push is completed!
+
+> NOTE: The build will fail before you complete the following steps!
+
+</details>
+
+<details>
+    <summary>Adding OpenAI Secrets to the Space</summary>
+
+1. Navigate to your Space settings.
+
+![image](https://i.imgur.com/zh0a2By.png)
+
+2. Navigate to `Variables and secrets` on the Settings page and click `New secret`: 
+
+![image](https://i.imgur.com/g2KlZdz.png)
+
+3. In the `Name` field - input `OPENAI_API_KEY` in the `Value (private)` field, put your OpenAI API Key.
+
+![image](https://i.imgur.com/eFcZ8U3.png)
+
+4. The Space will begin rebuilding!
+
+</details>
+
+## 🎉
+
+You just deployed Pythonic RAG!
+
+Try uploading a text file and asking some questions!
+
+## 🚧CHALLENGE MODE 🚧
+
+For more of a challenge, please reference [Building a Chainlit App](./BuildingAChainlitApp.md)!

app.py

@@ -0,0 +1,112 @@
+import os
+from dotenv import load_dotenv
+import chainlit as cl
+from langchain_openai import ChatOpenAI
+from langchain.prompts import PromptTemplate
+from utilities.rag_utilities import create_vector_store
+from langchain_core.prompts import ChatPromptTemplate
+from operator import itemgetter
+from langchain.schema.output_parser import StrOutputParser
+from langchain.schema.runnable import RunnablePassthrough
+from classes.app_state import AppState
+
+document_urls = [
+    "https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf",
+     "https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf",
+]
+
+# Load environment variables from .env file
+load_dotenv()
+
+# Get the OpenAI API key from environment variables
+openai_api_key = os.getenv("OPENAI_API_KEY")
+
+# Setup our state
+state = AppState()
+state.set_document_urls(document_urls)
+state.set_llm_model("gpt-3.5-turbo")
+state.set_embedding_model("text-embedding-3-small")
+
+
+# Initialize the OpenAI LLM using LangChain
+llm = ChatOpenAI(model=state.llm_model, openai_api_key=openai_api_key)
+
+
+
+
+qdrant_retriever = create_vector_store(state)
+
+system_template = """
+    You are an expert at explaining technical documents to people.
+    You are provided context below to answer the question.
+    Only use the information provided below.
+    If they do not ask a question, have a conversation with them and ask them if they have any questions
+    If you cannot answer the question with the content below say 'I don't have enough information, sorry'
+    The two documents are 'Blueprint for an AI Bill of Rights' and 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile'
+"""
+human_template = """ 
+===
+question:
+{question}
+
+===
+context:
+{context}
+===
+"""
+chat_prompt = ChatPromptTemplate.from_messages([
+    ("system", system_template),
+    ("human", human_template)
+])
+# create the chain
+openai_chat_model = ChatOpenAI(model="gpt-4o")
+
+
+
+retrieval_augmented_qa_chain = (
+    {"context": itemgetter("question") | qdrant_retriever, "question": itemgetter("question")}
+    | RunnablePassthrough.assign(context=itemgetter("context"))
+
+
+    | {"response": chat_prompt | openai_chat_model, "context": itemgetter("context")}
+)
+
+opening_content = """
+Welcome! I can answer your questions on AI based on the following 2 documents:
+- Blueprint for an AI Bill of Rights
+- Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile
+
+What questions do you have for me?
+"""
+
+@cl.on_chat_start
+async def on_chat_start():
+
+    await cl.Message(content=opening_content).send()
+
+
+
+@cl.on_message
+async def main(message):
+
+    # formatted_prompt = prompt.format(question=message.content)
+
+    # Call the LLM with the formatted prompt
+    # response = llm.invoke(formatted_prompt)
+    # 
+    response = retrieval_augmented_qa_chain.invoke({"question" : message.content })
+    answer_content = response["response"].content
+    msg = cl.Message(content="")    
+    # print(response["response"].content)
+    # print(f"Number of found context: {len(response['context'])}")
+    for i in range(0, len(answer_content), 50):  # Adjust chunk size (e.g., 50 characters)
+        chunk = answer_content[i:i+50]
+        await msg.stream_token(chunk)
+
+    # Send the response back to the user
+    await msg.send()
+
+    context_documents = response["context"]
+    num_contexts = len(context_documents)
+    context_msg = f"Number of found context: {num_contexts}"
+    await cl.Message(content=context_msg).send()

chainlit.md

@@ -0,0 +1,3 @@
+# Welcome to Chat with Your Text File
+
+With this application, you can chat with an uploaded text file that is smaller than 2MB!

classes/app_state.py

@@ -0,0 +1,81 @@
+class AppState:
+    def __init__(self):
+        self.debug = False
+        self.llm_model = "gpt-3.5-turbo"
+        self.embedding_model = "text-embedding-3-small"
+        self.chunk_size = 1000
+        self.chunk_overlap = 100
+        self.document_urls = []
+        self.download_folder = "data/"
+        self.loaded_documents = []
+        self.single_text_documents = []
+        self.metadata = []
+        self.titles = []
+        self.documents = []
+        self.combined_document_objects = []
+        self.retriever = None
+
+        self.system_template = "You are a helpful assistant"
+        #
+        self.user_input = None
+        self.retrieved_documents = []
+        self.chat_history = []
+        self.current_question = None
+    
+    def set_document_urls(self, document_urls):
+        self.document_urls = document_urls
+
+    def set_llm_model(self, llm_model):
+        self.llm_model = llm_model
+
+    def set_embedding_model(self, embedding_model):
+        self.embedding_model = embedding_model
+
+    def set_chunk_size(self, chunk_size):
+        self.chunk_size = chunk_size
+
+    def set_chunk_overlap(self, chunk_overlap):
+        self.chunk_overlap = chunk_overlap
+
+    def set_system_template(self, system_template):
+        self.system_template = system_template
+
+    def add_loaded_document(self, loaded_document):
+        self.loaded_documents.append(loaded_document)
+
+    def add_single_text_documents(self, single_text_document):
+        self.single_text_documents.append(single_text_document)
+    def add_metadata(self, metadata):
+        self.metadata = metadata
+
+    def add_title(self, title):
+        self.titles.append(title)
+    def add_document(self, document):
+        self.documents.append(document)
+    def add_combined_document_objects(self, combined_document_objects):
+        self.combined_document_objects = combined_document_objects
+    def set_retriever(self, retriever):
+        self.retriever = retriever
+    #
+    # Method to update the user input
+    def set_user_input(self, input_text):
+        self.user_input = input_text
+
+    # Method to add a retrieved document
+    # def add_document(self, document):
+    #     print("adding document")
+    #     print(self)
+    #     self.retrieved_documents.append(document)
+
+    # Method to update chat history
+    def update_chat_history(self, message):
+        self.chat_history.append(message)
+
+    # Method to get the current state
+    def get_state(self):
+        return {
+            "user_input": self.user_input,
+            "retrieved_documents": self.retrieved_documents,
+            "chat_history": self.chat_history,
+            "current_question": self.current_question
+        }

old_app.py

@@ -0,0 +1,145 @@
+import os
+from chainlit.types import AskFileResponse
+
+from utilities_2.openai_utils.prompts import (
+    UserRolePrompt,
+    SystemRolePrompt,
+    AssistantRolePrompt,
+)
+from utilities_2.openai_utils.embedding import EmbeddingModel
+from utilities_2.vectordatabase import VectorDatabase
+from utilities_2.openai_utils.chatmodel import ChatOpenAI
+import chainlit as cl
+from utilities.text_utils import FileLoader
+from utilities.pipeline import RetrievalAugmentedQAPipeline
+# from utilities.vector_database import QdrantDatabase
+
+
+def process_file(file, use_rct): 
+    fileLoader = FileLoader()
+    return fileLoader.load_file(file, use_rct)
+
+system_template = """\
+Use the following context to answer a users question. 
+If you cannot find the answer in the context, say you don't know the answer. 
+The context contains the text from a document. Refer to it as the document not the context.
+"""
+system_role_prompt = SystemRolePrompt(system_template)
+
+user_prompt_template = """\
+Context:
+{context}
+
+Question:
+{question}
+"""
+user_role_prompt = UserRolePrompt(user_prompt_template)
+
+@cl.on_chat_start
+async def on_chat_start():
+    # get user inputs
+    res = await cl.AskActionMessage(
+        content="Do you want to use Qdrant?",
+        actions=[
+            cl.Action(name="yes", value="yes", label="✅ Yes"),
+            cl.Action(name="no", value="no", label="❌ No"),
+        ],
+    ).send()
+    use_qdrant = False
+    use_qdrant_type = "Local"
+    if res and res.get("value") == "yes":
+        use_qdrant = True
+        local_res = await cl.AskActionMessage(
+                content="Do you want to use local or cloud?",
+                actions=[
+                    cl.Action(name="Local", value="Local", label="✅ Local"),
+                    cl.Action(name="Cloud", value="Cloud", label="❌ Cloud"),
+                ],
+            ).send()
+        if local_res and local_res.get("value") == "Cloud":
+            use_qdrant_type = "Cloud"
+    use_rct = False
+    res = await cl.AskActionMessage(
+        content="Do you want to use RecursiveCharacterTextSplitter?",
+        actions=[
+            cl.Action(name="yes", value="yes", label="✅ Yes"),
+            cl.Action(name="no", value="no", label="❌ No"),
+        ],
+    ).send()
+    if res and res.get("value") == "yes":
+        use_rct = True
+    
+    files = None
+    # Wait for the user to upload a file
+    while not files:
+        files = await cl.AskFileMessage(
+            content="Please upload a .txt or .pdf file to begin processing!",
+            accept=["text/plain", "application/pdf"],
+            max_size_mb=2,
+            timeout=180,
+        ).send()
+
+    file = files[0]
+
+    msg = cl.Message(
+        content=f"Processing `{file.name}`...", disable_human_feedback=True
+    )
+    await msg.send()
+
+    texts = process_file(file, use_rct)
+
+    msg = cl.Message(
+        content=f"Resulted in {len(texts)} chunks", disable_human_feedback=True
+    )
+    await msg.send()
+
+    # decide if to use the dict vector store of the Qdrant vector store
+    
+    # Create a dict vector store
+    if use_qdrant == False:
+        vector_db = VectorDatabase()
+        vector_db = await vector_db.abuild_from_list(texts)
+    else:
+        embedding_model = EmbeddingModel(embeddings_model_name= "text-embedding-3-small", dimensions=1000)
+        if use_qdrant_type == "Local":
+            from utilities.vector_database import QdrantDatabase
+            vector_db = QdrantDatabase(
+                embedding_model=embedding_model 
+            )
+
+            vector_db = await vector_db.abuild_from_list(texts)
+        
+    msg = cl.Message(
+        content=f"The Vector store has been created", disable_human_feedback=True
+    )
+    await msg.send()
+
+    chat_openai = ChatOpenAI()
+
+    # Create a chain
+    retrieval_augmented_qa_pipeline = RetrievalAugmentedQAPipeline(
+        vector_db_retriever=vector_db,
+        llm=chat_openai,
+        system_role_prompt=system_role_prompt,
+        user_role_prompt=user_role_prompt
+    )
+    
+    # Let the user know that the system is ready
+    msg.content = f"Processing `{file.name}` is complete."
+    await msg.update()
+    msg.content = f"You can now ask questions about `{file.name}`."
+    await msg.update()
+    cl.user_session.set("chain", retrieval_augmented_qa_pipeline)
+
+
+@cl.on_message
+async def main(message):
+    chain = cl.user_session.get("chain")
+
+    msg = cl.Message(content="")
+    result = await chain.arun_pipeline(message.content)
+
+    async for stream_resp in result["response"]:
+        await msg.stream_token(stream_resp)
+
+    await msg.send()

public/custom_styles.css

@@ -0,0 +1,129 @@
+/* General Styling */
+body {
+    background-color: #f0f4f8;
+    font-family: 'Poppins', sans-serif;
+    color: #333;
+    margin: 0;
+    padding: 0;
+}
+
+/* Container for the main content */
+.container {
+    max-width: 1200px;
+    margin: 40px auto;
+    padding: 20px;
+    background-color: #fff;
+    box-shadow: 0px 4px 12px rgba(0, 0, 0, 0.1);
+    border-radius: 12px;
+}
+
+/* Header */
+header {
+    background-color: #4b6584;
+    color: white;
+    text-align: center;
+    padding: 20px 0;
+    border-radius: 12px 12px 0 0;
+}
+
+header h1 {
+    margin: 0;
+    font-size: 2.5rem;
+    font-weight: bold;
+}
+
+/* Buttons */
+button {
+    background-color: #20bf6b;
+    color: white;
+    padding: 12px 20px;
+    font-size: 1rem;
+    font-weight: bold;
+    border: none;
+    border-radius: 6px;
+    cursor: pointer;
+    transition: background-color 0.3s ease;
+}
+
+button:hover {
+    background-color: #26de81;
+}
+
+/* Input Fields */
+input, textarea {
+    width: 100%;
+    padding: 12px;
+    margin: 8px 0;
+    font-size: 1rem;
+    border: 1px solid #ced6e0;
+    border-radius: 6px;
+    transition: border-color 0.3s ease;
+}
+
+input:focus, textarea:focus {
+    border-color: #4b7bec;
+    outline: none;
+}
+
+/* Custom Card Design */
+.card {
+    background-color: white;
+    border: 1px solid #dcdde1;
+    border-radius: 12px;
+    padding: 20px;
+    margin: 20px 0;
+    box-shadow: 0px 4px 8px rgba(0, 0, 0, 0.1);
+    transition: transform 0.3s ease, box-shadow 0.3s ease;
+}
+
+.card:hover {
+    transform: translateY(-5px);
+    box-shadow: 0px 6px 16px rgba(0, 0, 0, 0.15);
+}
+
+/* Headings inside cards */
+.card h2 {
+    font-size: 1.8rem;
+    font-weight: bold;
+    margin-bottom: 12px;
+    color: #34495e;
+}
+
+/* Custom Chat Bubbles */
+.chat-bubble {
+    background-color: #eaf0f8;
+    border-radius: 20px;
+    padding: 15px;
+    margin: 10px 0;
+    max-width: 75%;
+    box-shadow: 0px 2px 10px rgba(0, 0, 0, 0.05);
+}
+
+.chat-bubble.user {
+    background-color: #20bf6b;
+    color: white;
+    align-self: flex-end;
+}
+
+.chat-bubble.bot {
+    background-color: #4b6584;
+    color: white;
+}
+
+/* Footer */
+footer {
+    text-align: center;
+    padding: 20px 0;
+    font-size: 0.9rem;
+    color: #7f8c8d;
+}
+
+footer a {
+    color: #4b7bec;
+    text-decoration: none;
+    transition: color 0.3s ease;
+}
+
+footer a:hover {
+    color: #3867d6;
+}

rag_chain.ipynb

@@ -0,0 +1,757 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "AI Engineering Bootcamp Cohort 4 Midterm"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Install our key components for RAG etc"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Requirement already satisfied: langchain-core==0.2.27 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (0.2.27)\n",
+      "Requirement already satisfied: langchain-community==0.2.10 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (0.2.10)\n",
+      "Requirement already satisfied: PyYAML>=5.3 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core==0.2.27) (6.0.1)\n",
+      "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core==0.2.27) (1.33)\n",
+      "Requirement already satisfied: langsmith<0.2.0,>=0.1.75 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core==0.2.27) (0.1.122)\n",
+      "Requirement already satisfied: packaging<25,>=23.2 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core==0.2.27) (24.1)\n",
+      "Requirement already satisfied: pydantic<3,>=1 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core==0.2.27) (2.8.2)\n",
+      "Requirement already satisfied: tenacity!=8.4.0,<9.0.0,>=8.1.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core==0.2.27) (8.5.0)\n",
+      "Requirement already satisfied: typing-extensions>=4.7 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core==0.2.27) (4.11.0)\n",
+      "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community==0.2.10) (2.0.32)\n",
+      "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community==0.2.10) (3.10.3)\n",
+      "Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community==0.2.10) (0.5.14)\n",
+      "Requirement already satisfied: langchain<0.3.0,>=0.2.9 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community==0.2.10) (0.2.12)\n",
+      "Requirement already satisfied: numpy<2,>=1 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community==0.2.10) (1.26.4)\n",
+      "Requirement already satisfied: requests<3,>=2 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community==0.2.10) (2.32.3)\n",
+      "Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community==0.2.10) (2.3.5)\n",
+      "Requirement already satisfied: aiosignal>=1.1.2 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community==0.2.10) (1.3.1)\n",
+      "Requirement already satisfied: attrs>=17.3.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community==0.2.10) (24.2.0)\n",
+      "Requirement already satisfied: frozenlist>=1.1.1 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community==0.2.10) (1.4.1)\n",
+      "Requirement already satisfied: multidict<7.0,>=4.5 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community==0.2.10) (6.0.5)\n",
+      "Requirement already satisfied: yarl<2.0,>=1.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community==0.2.10) (1.9.4)\n",
+      "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain-community==0.2.10) (3.21.3)\n",
+      "Requirement already satisfied: typing-inspect<1,>=0.4.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain-community==0.2.10) (0.9.0)\n",
+      "Requirement already satisfied: jsonpointer>=1.9 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from jsonpatch<2.0,>=1.33->langchain-core==0.2.27) (3.0.0)\n",
+      "Requirement already satisfied: langchain-text-splitters<0.3.0,>=0.2.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain<0.3.0,>=0.2.9->langchain-community==0.2.10) (0.2.2)\n",
+      "Requirement already satisfied: httpx<1,>=0.23.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langsmith<0.2.0,>=0.1.75->langchain-core==0.2.27) (0.27.0)\n",
+      "Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langsmith<0.2.0,>=0.1.75->langchain-core==0.2.27) (3.10.7)\n",
+      "Requirement already satisfied: annotated-types>=0.4.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from pydantic<3,>=1->langchain-core==0.2.27) (0.7.0)\n",
+      "Requirement already satisfied: pydantic-core==2.20.1 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from pydantic<3,>=1->langchain-core==0.2.27) (2.20.1)\n",
+      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from requests<3,>=2->langchain-community==0.2.10) (3.3.2)\n",
+      "Requirement already satisfied: idna<4,>=2.5 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from requests<3,>=2->langchain-community==0.2.10) (3.7)\n",
+      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from requests<3,>=2->langchain-community==0.2.10) (2.2.2)\n",
+      "Requirement already satisfied: certifi>=2017.4.17 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from requests<3,>=2->langchain-community==0.2.10) (2024.8.30)\n",
+      "Requirement already satisfied: greenlet!=0.4.17 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from SQLAlchemy<3,>=1.4->langchain-community==0.2.10) (3.0.3)\n",
+      "Requirement already satisfied: anyio in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpx<1,>=0.23.0->langsmith<0.2.0,>=0.1.75->langchain-core==0.2.27) (3.7.1)\n",
+      "Requirement already satisfied: httpcore==1.* in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpx<1,>=0.23.0->langsmith<0.2.0,>=0.1.75->langchain-core==0.2.27) (1.0.5)\n",
+      "Requirement already satisfied: sniffio in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpx<1,>=0.23.0->langsmith<0.2.0,>=0.1.75->langchain-core==0.2.27) (1.3.0)\n",
+      "Requirement already satisfied: h11<0.15,>=0.13 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpcore==1.*->httpx<1,>=0.23.0->langsmith<0.2.0,>=0.1.75->langchain-core==0.2.27) (0.14.0)\n",
+      "Requirement already satisfied: mypy-extensions>=0.3.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain-community==0.2.10) (1.0.0)\n",
+      "Requirement already satisfied: langchain-experimental==0.0.64 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (0.0.64)\n",
+      "Requirement already satisfied: langgraph-checkpoint==1.0.6 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (1.0.6)\n",
+      "Requirement already satisfied: langgraph==0.2.16 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (0.2.16)\n",
+      "Requirement already satisfied: langchain-qdrant==0.1.3 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (0.1.3)\n",
+      "Requirement already satisfied: langchain-community<0.3.0,>=0.2.10 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-experimental==0.0.64) (0.2.10)\n",
+      "Requirement already satisfied: langchain-core<0.3.0,>=0.2.27 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-experimental==0.0.64) (0.2.27)\n",
+      "Requirement already satisfied: pydantic<3.0.0,>=2.7.4 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-qdrant==0.1.3) (2.8.2)\n",
+      "Requirement already satisfied: qdrant-client<2.0.0,>=1.10.1 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-qdrant==0.1.3) (1.11.2)\n",
+      "Requirement already satisfied: PyYAML>=5.3 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (6.0.1)\n",
+      "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (2.0.32)\n",
+      "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (3.10.3)\n",
+      "Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (0.5.14)\n",
+      "Requirement already satisfied: langchain<0.3.0,>=0.2.9 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (0.2.12)\n",
+      "Requirement already satisfied: langsmith<0.2.0,>=0.1.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (0.1.122)\n",
+      "Requirement already satisfied: numpy<2,>=1 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (1.26.4)\n",
+      "Requirement already satisfied: requests<3,>=2 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (2.32.3)\n",
+      "Requirement already satisfied: tenacity!=8.4.0,<9.0.0,>=8.1.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (8.5.0)\n",
+      "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core<0.3.0,>=0.2.27->langchain-experimental==0.0.64) (1.33)\n",
+      "Requirement already satisfied: packaging<25,>=23.2 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core<0.3.0,>=0.2.27->langchain-experimental==0.0.64) (24.1)\n",
+      "Requirement already satisfied: typing-extensions>=4.7 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core<0.3.0,>=0.2.27->langchain-experimental==0.0.64) (4.11.0)\n",
+      "Requirement already satisfied: annotated-types>=0.4.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from pydantic<3.0.0,>=2.7.4->langchain-qdrant==0.1.3) (0.7.0)\n",
+      "Requirement already satisfied: pydantic-core==2.20.1 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from pydantic<3.0.0,>=2.7.4->langchain-qdrant==0.1.3) (2.20.1)\n",
+      "Requirement already satisfied: grpcio>=1.41.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (1.66.1)\n",
+      "Requirement already satisfied: grpcio-tools>=1.41.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (1.66.1)\n",
+      "Requirement already satisfied: httpx>=0.20.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpx[http2]>=0.20.0->qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (0.27.0)\n",
+      "Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (2.10.1)\n",
+      "Requirement already satisfied: urllib3<3,>=1.26.14 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (2.2.2)\n",
+      "Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (2.3.5)\n",
+      "Requirement already satisfied: aiosignal>=1.1.2 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (1.3.1)\n",
+      "Requirement already satisfied: attrs>=17.3.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (24.2.0)\n",
+      "Requirement already satisfied: frozenlist>=1.1.1 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (1.4.1)\n",
+      "Requirement already satisfied: multidict<7.0,>=4.5 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (6.0.5)\n",
+      "Requirement already satisfied: yarl<2.0,>=1.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (1.9.4)\n",
+      "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (3.21.3)\n",
+      "Requirement already satisfied: typing-inspect<1,>=0.4.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (0.9.0)\n",
+      "Requirement already satisfied: protobuf<6.0dev,>=5.26.1 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from grpcio-tools>=1.41.0->qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (5.28.0)\n",
+      "Requirement already satisfied: setuptools in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from grpcio-tools>=1.41.0->qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (74.0.0)\n",
+      "Requirement already satisfied: anyio in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (3.7.1)\n",
+      "Requirement already satisfied: certifi in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (2024.8.30)\n",
+      "Requirement already satisfied: httpcore==1.* in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (1.0.5)\n",
+      "Requirement already satisfied: idna in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (3.7)\n",
+      "Requirement already satisfied: sniffio in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (1.3.0)\n",
+      "Requirement already satisfied: h11<0.15,>=0.13 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpcore==1.*->httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (0.14.0)\n",
+      "Requirement already satisfied: h2<5,>=3 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpx[http2]>=0.20.0->qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (4.1.0)\n",
+      "Requirement already satisfied: jsonpointer>=1.9 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from jsonpatch<2.0,>=1.33->langchain-core<0.3.0,>=0.2.27->langchain-experimental==0.0.64) (3.0.0)\n",
+      "Requirement already satisfied: langchain-text-splitters<0.3.0,>=0.2.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain<0.3.0,>=0.2.9->langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (0.2.2)\n",
+      "Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langsmith<0.2.0,>=0.1.0->langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (3.10.7)\n",
+      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from requests<3,>=2->langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (3.3.2)\n",
+      "Requirement already satisfied: greenlet!=0.4.17 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from SQLAlchemy<3,>=1.4->langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (3.0.3)\n",
+      "Requirement already satisfied: hyperframe<7,>=6.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (6.0.1)\n",
+      "Requirement already satisfied: hpack<5,>=4.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client<2.0.0,>=1.10.1->langchain-qdrant==0.1.3) (4.0.0)\n",
+      "Requirement already satisfied: mypy-extensions>=0.3.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain-community<0.3.0,>=0.2.10->langchain-experimental==0.0.64) (1.0.0)\n",
+      "Requirement already satisfied: langchain-openai==0.1.9 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (0.1.9)\n",
+      "Requirement already satisfied: langchain-core<0.3,>=0.2.2 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-openai==0.1.9) (0.2.27)\n",
+      "Requirement already satisfied: openai<2.0.0,>=1.26.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-openai==0.1.9) (1.44.0)\n",
+      "Requirement already satisfied: tiktoken<1,>=0.7 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-openai==0.1.9) (0.7.0)\n",
+      "Requirement already satisfied: PyYAML>=5.3 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core<0.3,>=0.2.2->langchain-openai==0.1.9) (6.0.1)\n",
+      "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core<0.3,>=0.2.2->langchain-openai==0.1.9) (1.33)\n",
+      "Requirement already satisfied: langsmith<0.2.0,>=0.1.75 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core<0.3,>=0.2.2->langchain-openai==0.1.9) (0.1.122)\n",
+      "Requirement already satisfied: packaging<25,>=23.2 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core<0.3,>=0.2.2->langchain-openai==0.1.9) (24.1)\n",
+      "Requirement already satisfied: pydantic<3,>=1 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core<0.3,>=0.2.2->langchain-openai==0.1.9) (2.8.2)\n",
+      "Requirement already satisfied: tenacity!=8.4.0,<9.0.0,>=8.1.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core<0.3,>=0.2.2->langchain-openai==0.1.9) (8.5.0)\n",
+      "Requirement already satisfied: typing-extensions>=4.7 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langchain-core<0.3,>=0.2.2->langchain-openai==0.1.9) (4.11.0)\n",
+      "Requirement already satisfied: anyio<5,>=3.5.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from openai<2.0.0,>=1.26.0->langchain-openai==0.1.9) (3.7.1)\n",
+      "Requirement already satisfied: distro<2,>=1.7.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from openai<2.0.0,>=1.26.0->langchain-openai==0.1.9) (1.9.0)\n",
+      "Requirement already satisfied: httpx<1,>=0.23.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from openai<2.0.0,>=1.26.0->langchain-openai==0.1.9) (0.27.0)\n",
+      "Requirement already satisfied: jiter<1,>=0.4.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from openai<2.0.0,>=1.26.0->langchain-openai==0.1.9) (0.5.0)\n",
+      "Requirement already satisfied: sniffio in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from openai<2.0.0,>=1.26.0->langchain-openai==0.1.9) (1.3.0)\n",
+      "Requirement already satisfied: tqdm>4 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from openai<2.0.0,>=1.26.0->langchain-openai==0.1.9) (4.66.5)\n",
+      "Requirement already satisfied: regex>=2022.1.18 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from tiktoken<1,>=0.7->langchain-openai==0.1.9) (2024.7.24)\n",
+      "Requirement already satisfied: requests>=2.26.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from tiktoken<1,>=0.7->langchain-openai==0.1.9) (2.32.3)\n",
+      "Requirement already satisfied: idna>=2.8 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from anyio<5,>=3.5.0->openai<2.0.0,>=1.26.0->langchain-openai==0.1.9) (3.7)\n",
+      "Requirement already satisfied: certifi in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpx<1,>=0.23.0->openai<2.0.0,>=1.26.0->langchain-openai==0.1.9) (2024.8.30)\n",
+      "Requirement already satisfied: httpcore==1.* in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpx<1,>=0.23.0->openai<2.0.0,>=1.26.0->langchain-openai==0.1.9) (1.0.5)\n",
+      "Requirement already satisfied: h11<0.15,>=0.13 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai<2.0.0,>=1.26.0->langchain-openai==0.1.9) (0.14.0)\n",
+      "Requirement already satisfied: jsonpointer>=1.9 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from jsonpatch<2.0,>=1.33->langchain-core<0.3,>=0.2.2->langchain-openai==0.1.9) (3.0.0)\n",
+      "Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from langsmith<0.2.0,>=0.1.75->langchain-core<0.3,>=0.2.2->langchain-openai==0.1.9) (3.10.7)\n",
+      "Requirement already satisfied: annotated-types>=0.4.0 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from pydantic<3,>=1->langchain-core<0.3,>=0.2.2->langchain-openai==0.1.9) (0.7.0)\n",
+      "Requirement already satisfied: pydantic-core==2.20.1 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from pydantic<3,>=1->langchain-core<0.3,>=0.2.2->langchain-openai==0.1.9) (2.20.1)\n",
+      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken<1,>=0.7->langchain-openai==0.1.9) (3.3.2)\n",
+      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken<1,>=0.7->langchain-openai==0.1.9) (2.2.2)\n"
+     ]
+    }
+   ],
+   "source": [
+    "!pip install langchain-core==0.2.27 langchain-community==0.2.10\n",
+    "!pip install langchain-experimental==0.0.64 langgraph-checkpoint==1.0.6 langgraph==0.2.16 langchain-qdrant==0.1.3\n",
+    "!pip install langchain-openai==0.1.9"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Install our vector store - Qdrant"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "",
+     "evalue": "",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[1;31mThe kernel failed to start as the Python Environment 'Python' is no longer available. Consider selecting another kernel or refreshing the list of Python Environments."
+     ]
+    }
+   ],
+   "source": [
+    "!pip install -qU qdrant-client"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Install supporting utilities"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install -qU tiktoken pymupdf\n",
+    "from langchain_community.document_loaders import PyMuPDFLoader"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Environment Variables\n",
+    "\n",
+    "- get OpenAI API Key - will use some of the OpenAI models"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "import getpass\n",
+    "\n",
+    "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"OpenAI API Key:\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 60,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Number of pages in 1: 73\n",
+      "Number of pages in 2: 64\n",
+      "Title of Document 1: Blueprint for an AI Bill of Rights\n",
+      "Title of Document 2: Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile\n",
+      "Full metadata for Document 1: {'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': \"D:20220920133035-04'00'\", 'modDate': \"D:20221003104118-04'00'\", 'trapped': '', 'encryption': None}\n",
+      "Full metadata for Document 2: {'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': \"D:20240805141702-04'00'\", 'modDate': \"D:20240805143048-04'00'\", 'trapped': '', 'encryption': None}\n",
+      "Number of chunks for Document 1: 61\n",
+      "Number of chunks for Document 2: 53\n"
+     ]
+    }
+   ],
+   "source": [
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Number of pages in 1: 73\n",
+      "Number of pages in 2: 64\n",
+      "Title of Document 1: Blueprint for an AI Bill of Rights\n",
+      "Title of Document 2: Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile\n",
+      "Full metadata for Document 1: {'format': 'PDF 1.6', 'title': 'Blueprint for an AI Bill of Rights', 'author': '', 'subject': '', 'keywords': '', 'creator': 'Adobe Illustrator 26.3 (Macintosh)', 'producer': 'iLovePDF', 'creationDate': \"D:20220920133035-04'00'\", 'modDate': \"D:20221003104118-04'00'\", 'trapped': '', 'encryption': None}\n",
+      "Full metadata for Document 2: {'format': 'PDF 1.6', 'title': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'author': 'National Institute of Standards and Technology', 'subject': '', 'keywords': '', 'creator': 'Acrobat PDFMaker 24 for Word', 'producer': 'Adobe PDF Library 24.2.159', 'creationDate': \"D:20240805141702-04'00'\", 'modDate': \"D:20240805143048-04'00'\", 'trapped': '', 'encryption': None}\n",
+      "Number of chunks for Document 1: 61\n",
+      "Number of chunks for Document 2: 53\n"
+     ]
+    }
+   ],
+   "source": [
+    "from utilities.rag_utilities import create_vector_store\n",
+    "\n",
+    "qdrant_retriever = create_vector_store()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Test it"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "You should be protected from abusive data practices via built-in \n",
+      "protections and you should have agency over how data about \n",
+      "you is used. You should be protected from violations of privacy through \n",
+      "design choices that ensure such protections are included by default, including \n",
+      "ensuring that data collection conforms to reasonable expectations and that \n",
+      "only data strictly necessary for the specific context is collected. Designers, de­\n",
+      "velopers, and deployers of automated systems should seek your permission \n",
+      "and respect your decisions regarding collection, use, access, transfer, and de­\n",
+      "letion of your data in appropriate ways and to the greatest extent possible; \n",
+      "where not possible, alternative privacy by design safeguards should be used. \n",
+      "Systems should not employ user experience and design decisions that obfus­\n",
+      "cate user choice or burden users with defaults that are privacy invasive. Con­\n",
+      "sent should only be used to justify collection of data in cases where it can be \n",
+      "appropriately and meaningfully given. Any consent requests should be brief, \n",
+      "be understandable in plain language, and give you agency over data collection \n",
+      "and the specific context of use; current hard-to-understand no­\n",
+      "tice-and-choice practices for broad uses of data should be changed. Enhanced \n",
+      "protections and restrictions for data and inferences related to sensitive do­\n",
+      "mains, including health, work, education, criminal justice, and finance, and \n",
+      "for data pertaining to youth should put you first. In sensitive domains, your \n",
+      "data and related inferences should only be used for necessary functions, and \n",
+      "you should be protected by ethical review and use prohibitions. You and your \n",
+      "communities should be free from unchecked surveillance; surveillance tech­\n",
+      "nologies should be subject to heightened oversight that includes at least \n",
+      "pre-deployment assessment of their potential harms and scope limits to pro­\n",
+      "tect privacy and civil liberties. Continuous surveillance and monitoring \n",
+      "should not be used in education, work, housing, or in other contexts where the \n",
+      "use of such surveillance technologies is likely to limit rights, opportunities, or \n",
+      "access. Whenever possible, you should have access to reporting that confirms \n",
+      "your data decisions have been respected and provides an assessment of the \n",
+      "potential impact of surveillance technologies on your rights, opportunities, or \n",
+      "access. \n",
+      "DATA PRIVACY\n",
+      "30\n",
+      "{'source': 'Blueprint for an AI Bill of Rights', 'document_id': 'doc1', '_id': 'c7bb1309f85f46c7b5d1c7511523cb10', '_collection_name': 'ffb2f8aece51430086938e04161466e4'}\n",
+      "---\n",
+      "You should be protected from abusive data practices via built-in \n",
+      "protections and you should have agency over how data about \n",
+      "you is used. You should be protected from violations of privacy through \n",
+      "design choices that ensure such protections are included by default, including \n",
+      "ensuring that data collection conforms to reasonable expectations and that \n",
+      "only data strictly necessary for the specific context is collected. Designers, de­\n",
+      "velopers, and deployers of automated systems should seek your permission \n",
+      "and respect your decisions regarding collection, use, access, transfer, and de­\n",
+      "letion of your data in appropriate ways and to the greatest extent possible; \n",
+      "where not possible, alternative privacy by design safeguards should be used. \n",
+      "Systems should not employ user experience and design decisions that obfus­\n",
+      "cate user choice or burden users with defaults that are privacy invasive. Con­\n",
+      "sent should only be used to justify collection of data in cases where it can be \n",
+      "appropriately and meaningfully given. Any consent requests should be brief, \n",
+      "be understandable in plain language, and give you agency over data collection \n",
+      "and the specific context of use; current hard-to-understand no­\n",
+      "tice-and-choice practices for broad uses of data should be changed. Enhanced \n",
+      "protections and restrictions for data and inferences related to sensitive do­\n",
+      "mains, including health, work, education, criminal justice, and finance, and \n",
+      "for data pertaining to youth should put you first. In sensitive domains, your \n",
+      "data and related inferences should only be used for necessary functions, and \n",
+      "you should be protected by ethical review and use prohibitions. You and your \n",
+      "communities should be free from unchecked surveillance; surveillance tech­\n",
+      "nologies should be subject to heightened oversight that includes at least \n",
+      "pre-deployment assessment of their potential harms and scope limits to pro­\n",
+      "tect privacy and civil liberties. Continuous surveillance and monitoring \n",
+      "should not be used in education, work, housing, or in other contexts where the \n",
+      "use of such surveillance technologies is likely to limit rights, opportunities, or \n",
+      "access. Whenever possible, you should have access to reporting that confirms \n",
+      "your data decisions have been respected and provides an assessment of the \n",
+      "potential impact of surveillance technologies on your rights, opportunities, or \n",
+      "access. \n",
+      "DATA PRIVACY\n",
+      "30\n",
+      "{'source': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'document_id': 'doc2', '_id': '3969630587ab4c199e299edfb92905b5', '_collection_name': 'ffb2f8aece51430086938e04161466e4'}\n",
+      "---\n",
+      "DATA PRIVACY \n",
+      "WHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\n",
+      "The expectations for automated systems are meant to serve as a blueprint for the development of additional \n",
+      "technical standards and practices that are tailored for particular sectors and contexts. \n",
+      "Traditional terms of service—the block of text that the public is accustomed to clicking through when using a web­\n",
+      "site or digital app—are not an adequate mechanism for protecting privacy. The American public should be protect­\n",
+      "ed via built-in privacy protections, data minimization, use and collection limitations, and transparency, in addition \n",
+      "to being entitled to clear mechanisms to control access to and use of their data—including their metadata—in a \n",
+      "proactive, informed, and ongoing way. Any automated system collecting, using, sharing, or storing personal data \n",
+      "should meet these expectations. \n",
+      "Protect privacy by design and by default \n",
+      "Privacy by design and by default. Automated systems should be designed and built with privacy protect­\n",
+      "ed by default. Privacy risks should be assessed throughout the development life cycle, including privacy risks \n",
+      "from reidentification, and appropriate technical and policy mitigation measures should be implemented. This \n",
+      "includes potential harms to those who are not users of the automated system, but who may be harmed by \n",
+      "inferred data, purposeful privacy violations, or community surveillance or other community harms. Data \n",
+      "collection should be minimized and clearly communicated to the people whose data is collected. Data should \n",
+      "only be collected or used for the purposes of training or testing machine learning models if such collection and \n",
+      "use is legal and consistent with the expectations of the people whose data is collected. User experience \n",
+      "research should be conducted to confirm that people understand what data is being collected about them and \n",
+      "how it will be used, and that this collection matches their expectations and desires. \n",
+      "Data collection and use-case scope limits. Data collection should be limited in scope, with specific, \n",
+      "narrow identified goals, to avoid \"mission creep.\"  Anticipated data collection should be determined to be \n",
+      "strictly necessary to the identified goals and should be minimized as much as possible. Data collected based on \n",
+      "these identified goals and for a specific context should not be used in a different context without assessing for \n",
+      "new privacy risks and implementing appropriate mitigation measures, which may include express consent. \n",
+      "Clear timelines for data retention should be established, with data deleted as soon as possible in accordance \n",
+      "with legal or policy-based limitations. Determined data retention timelines should be documented and justi­\n",
+      "fied. \n",
+      "Risk identification and mitigation. Entities that collect, use, share, or store sensitive data should \n",
+      "attempt to proactively identify harms and seek to manage them so as to avoid, mitigate, and respond appropri­\n",
+      "ately to identified risks. Appropriate responses include determining not to process data when the privacy risks \n",
+      "outweigh the benefits or implementing measures to mitigate acceptable risks. Appropriate responses do not \n",
+      "include sharing or transferring the privacy risks to users via notice or consent requests where users could not \n",
+      "reasonably be expected to understand the risks without further support. \n",
+      "Privacy-preserving security. Entities creating, using, or governing automated systems should follow \n",
+      "privacy and security best practices designed to ensure data and metadata do not leak beyond the specific \n",
+      "consented use case. Best practices could include using privacy-enhancing cryptography or other types of \n",
+      "privacy-enhancing technologies or fine-grained permissions and access control mechanisms, along with \n",
+      "conventional system security protocols. \n",
+      "33\n",
+      "{'source': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'document_id': 'doc2', '_id': '382b9ef10528494a85b60231a0d0981b', '_collection_name': 'ffb2f8aece51430086938e04161466e4'}\n",
+      "---\n",
+      "DATA PRIVACY \n",
+      "WHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\n",
+      "The expectations for automated systems are meant to serve as a blueprint for the development of additional \n",
+      "technical standards and practices that are tailored for particular sectors and contexts. \n",
+      "Traditional terms of service—the block of text that the public is accustomed to clicking through when using a web­\n",
+      "site or digital app—are not an adequate mechanism for protecting privacy. The American public should be protect­\n",
+      "ed via built-in privacy protections, data minimization, use and collection limitations, and transparency, in addition \n",
+      "to being entitled to clear mechanisms to control access to and use of their data—including their metadata—in a \n",
+      "proactive, informed, and ongoing way. Any automated system collecting, using, sharing, or storing personal data \n",
+      "should meet these expectations. \n",
+      "Protect privacy by design and by default \n",
+      "Privacy by design and by default. Automated systems should be designed and built with privacy protect­\n",
+      "ed by default. Privacy risks should be assessed throughout the development life cycle, including privacy risks \n",
+      "from reidentification, and appropriate technical and policy mitigation measures should be implemented. This \n",
+      "includes potential harms to those who are not users of the automated system, but who may be harmed by \n",
+      "inferred data, purposeful privacy violations, or community surveillance or other community harms. Data \n",
+      "collection should be minimized and clearly communicated to the people whose data is collected. Data should \n",
+      "only be collected or used for the purposes of training or testing machine learning models if such collection and \n",
+      "use is legal and consistent with the expectations of the people whose data is collected. User experience \n",
+      "research should be conducted to confirm that people understand what data is being collected about them and \n",
+      "how it will be used, and that this collection matches their expectations and desires. \n",
+      "Data collection and use-case scope limits. Data collection should be limited in scope, with specific, \n",
+      "narrow identified goals, to avoid \"mission creep.\"  Anticipated data collection should be determined to be \n",
+      "strictly necessary to the identified goals and should be minimized as much as possible. Data collected based on \n",
+      "these identified goals and for a specific context should not be used in a different context without assessing for \n",
+      "new privacy risks and implementing appropriate mitigation measures, which may include express consent. \n",
+      "Clear timelines for data retention should be established, with data deleted as soon as possible in accordance \n",
+      "with legal or policy-based limitations. Determined data retention timelines should be documented and justi­\n",
+      "fied. \n",
+      "Risk identification and mitigation. Entities that collect, use, share, or store sensitive data should \n",
+      "attempt to proactively identify harms and seek to manage them so as to avoid, mitigate, and respond appropri­\n",
+      "ately to identified risks. Appropriate responses include determining not to process data when the privacy risks \n",
+      "outweigh the benefits or implementing measures to mitigate acceptable risks. Appropriate responses do not \n",
+      "include sharing or transferring the privacy risks to users via notice or consent requests where users could not \n",
+      "reasonably be expected to understand the risks without further support. \n",
+      "Privacy-preserving security. Entities creating, using, or governing automated systems should follow \n",
+      "privacy and security best practices designed to ensure data and metadata do not leak beyond the specific \n",
+      "consented use case. Best practices could include using privacy-enhancing cryptography or other types of \n",
+      "privacy-enhancing technologies or fine-grained permissions and access control mechanisms, along with \n",
+      "conventional system security protocols. \n",
+      "33\n",
+      "{'source': 'Blueprint for an AI Bill of Rights', 'document_id': 'doc1', '_id': '316b7df2f22c4975ad8621ec0131c8ac', '_collection_name': 'ffb2f8aece51430086938e04161466e4'}\n",
+      "---\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/home/rchrdgwr/anaconda3/envs/llmops-course/lib/python3.11/site-packages/langchain_core/_api/deprecation.py:139: LangChainDeprecationWarning: The method `BaseRetriever.get_relevant_documents` was deprecated in langchain-core 0.1.46 and will be removed in 0.3.0. Use invoke instead.\n",
+      "  warn_deprecated(\n"
+     ]
+    }
+   ],
+   "source": [
+    "query = \"How should you be protected from abusive data practices \"\n",
+    "results = qdrant_retriever.get_relevant_documents(query)\n",
+    "\n",
+    "for result in results:\n",
+    "    print(result.page_content)\n",
+    "    print(result.metadata)\n",
+    "    print(\"---\")\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 53,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "ENDNOTES\n",
+      "75. See., e.g., Sam Sabin. Digital surveillance in a post-Roe world. Politico. May 5, 2022. https://\n",
+      "www.politico.com/newsletters/digital-future-daily/2022/05/05/digital-surveillance-in-a-post-roe­\n",
+      "world-00030459; Federal Trade Commission. FTC Sues Kochava for Selling Data that Tracks People at\n",
+      "Reproductive Health Clinics, Places of Worship, and Other Sensitive Locations. Aug. 29, 2022. https://\n",
+      "www.ftc.gov/news-events/news/press-releases/2022/08/ftc-sues-kochava-selling-data-tracks-people­\n",
+      "reproductive-health-clinics-places-worship-other\n",
+      "76. Todd Feathers. This Private Equity Firm Is Amassing Companies That Collect Data on America’s\n",
+      "Children. The Markup. Jan. 11, 2022.\n",
+      "https://themarkup.org/machine-learning/2022/01/11/this-private-equity-firm-is-amassing-companies­\n",
+      "that-collect-data-on-americas-children\n",
+      "77. Reed Albergotti. Every employee who leaves Apple becomes an ‘associate’: In job databases used by\n",
+      "employers to verify resume information, every former Apple employee’s title gets erased and replaced with\n",
+      "a generic title. The Washington Post. Feb. 10, 2022.\n",
+      "https://www.washingtonpost.com/technology/2022/02/10/apple-associate/\n",
+      "78. National Institute of Standards and Technology. Privacy Framework Perspectives and Success\n",
+      "Stories. Accessed May 2, 2022.\n",
+      "https://www.nist.gov/privacy-framework/getting-started-0/perspectives-and-success-stories\n",
+      "79. ACLU of New York. What You Need to Know About New York’s Temporary Ban on Facial\n",
+      "Recognition in Schools. Accessed May 2, 2022.\n",
+      "https://www.nyclu.org/en/publications/what-you-need-know-about-new-yorks-temporary-ban-facial­\n",
+      "recognition-schools\n",
+      "80. New York State Assembly. Amendment to Education Law. Enacted Dec. 22, 2020.\n",
+      "https://nyassembly.gov/leg/?default_fld=&leg_video=&bn=S05140&term=2019&Summary=Y&Text=Y\n",
+      "81. U.S Department of Labor. Labor-Management Reporting and Disclosure Act of 1959, As Amended.\n",
+      "https://www.dol.gov/agencies/olms/laws/labor-management-reporting-and-disclosure-act (Section\n",
+      "203). See also: U.S Department of Labor. Form LM-10. OLMS Fact Sheet, Accessed May 2, 2022. https://\n",
+      "www.dol.gov/sites/dolgov/files/OLMS/regs/compliance/LM-10_factsheet.pdf\n",
+      "82. See, e.g., Apple. Protecting the User’s Privacy. Accessed May 2, 2022.\n",
+      "https://developer.apple.com/documentation/uikit/protecting_the_user_s_privacy; Google Developers.\n",
+      "Design for Safety: Android is secure by default and private by design. Accessed May 3, 2022.\n",
+      "https://developer.android.com/design-for-safety\n",
+      "83. Karen Hao. The coming war on the hidden algorithms that trap people in poverty. MIT Tech Review.\n",
+      "Dec. 4, 2020.\n",
+      "https://www.technologyreview.com/2020/12/04/1013068/algorithms-create-a-poverty-trap-lawyers­\n",
+      "fight-back/\n",
+      "84. Anjana Samant, Aaron Horowitz, Kath Xu, and Sophie Beiers. Family Surveillance by Algorithm.\n",
+      "ACLU. Accessed May 2, 2022.\n",
+      "https://www.aclu.org/fact-sheet/family-surveillance-algorithm\n",
+      "70\n",
+      "{'source': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'document_id': 'doc2', '_id': '4301d9bb14f44d928a3e254b4da4aa18', '_collection_name': '527fdbdd8d4148b48f493df2f8334866'}\n",
+      "---\n",
+      "ENDNOTES\n",
+      "75. See., e.g., Sam Sabin. Digital surveillance in a post-Roe world. Politico. May 5, 2022. https://\n",
+      "www.politico.com/newsletters/digital-future-daily/2022/05/05/digital-surveillance-in-a-post-roe­\n",
+      "world-00030459; Federal Trade Commission. FTC Sues Kochava for Selling Data that Tracks People at\n",
+      "Reproductive Health Clinics, Places of Worship, and Other Sensitive Locations. Aug. 29, 2022. https://\n",
+      "www.ftc.gov/news-events/news/press-releases/2022/08/ftc-sues-kochava-selling-data-tracks-people­\n",
+      "reproductive-health-clinics-places-worship-other\n",
+      "76. Todd Feathers. This Private Equity Firm Is Amassing Companies That Collect Data on America’s\n",
+      "Children. The Markup. Jan. 11, 2022.\n",
+      "https://themarkup.org/machine-learning/2022/01/11/this-private-equity-firm-is-amassing-companies­\n",
+      "that-collect-data-on-americas-children\n",
+      "77. Reed Albergotti. Every employee who leaves Apple becomes an ‘associate’: In job databases used by\n",
+      "employers to verify resume information, every former Apple employee’s title gets erased and replaced with\n",
+      "a generic title. The Washington Post. Feb. 10, 2022.\n",
+      "https://www.washingtonpost.com/technology/2022/02/10/apple-associate/\n",
+      "78. National Institute of Standards and Technology. Privacy Framework Perspectives and Success\n",
+      "Stories. Accessed May 2, 2022.\n",
+      "https://www.nist.gov/privacy-framework/getting-started-0/perspectives-and-success-stories\n",
+      "79. ACLU of New York. What You Need to Know About New York’s Temporary Ban on Facial\n",
+      "Recognition in Schools. Accessed May 2, 2022.\n",
+      "https://www.nyclu.org/en/publications/what-you-need-know-about-new-yorks-temporary-ban-facial­\n",
+      "recognition-schools\n",
+      "80. New York State Assembly. Amendment to Education Law. Enacted Dec. 22, 2020.\n",
+      "https://nyassembly.gov/leg/?default_fld=&leg_video=&bn=S05140&term=2019&Summary=Y&Text=Y\n",
+      "81. U.S Department of Labor. Labor-Management Reporting and Disclosure Act of 1959, As Amended.\n",
+      "https://www.dol.gov/agencies/olms/laws/labor-management-reporting-and-disclosure-act (Section\n",
+      "203). See also: U.S Department of Labor. Form LM-10. OLMS Fact Sheet, Accessed May 2, 2022. https://\n",
+      "www.dol.gov/sites/dolgov/files/OLMS/regs/compliance/LM-10_factsheet.pdf\n",
+      "82. See, e.g., Apple. Protecting the User’s Privacy. Accessed May 2, 2022.\n",
+      "https://developer.apple.com/documentation/uikit/protecting_the_user_s_privacy; Google Developers.\n",
+      "Design for Safety: Android is secure by default and private by design. Accessed May 3, 2022.\n",
+      "https://developer.android.com/design-for-safety\n",
+      "83. Karen Hao. The coming war on the hidden algorithms that trap people in poverty. MIT Tech Review.\n",
+      "Dec. 4, 2020.\n",
+      "https://www.technologyreview.com/2020/12/04/1013068/algorithms-create-a-poverty-trap-lawyers­\n",
+      "fight-back/\n",
+      "84. Anjana Samant, Aaron Horowitz, Kath Xu, and Sophie Beiers. Family Surveillance by Algorithm.\n",
+      "ACLU. Accessed May 2, 2022.\n",
+      "https://www.aclu.org/fact-sheet/family-surveillance-algorithm\n",
+      "70\n",
+      "{'source': 'Blueprint for an AI Bill of Rights', 'document_id': 'doc1', '_id': '2442bc209b7842858b5a85e83131fd69', '_collection_name': '527fdbdd8d4148b48f493df2f8334866'}\n",
+      "---\n",
+      "ENDNOTES\n",
+      "57. ISO Technical Management Board. ISO/IEC Guide 71:2014. Guide for addressing accessibility in\n",
+      "standards. International Standards Organization. 2021. https://www.iso.org/standard/57385.html\n",
+      "58. World Wide Web Consortium. Web Content Accessibility Guidelines (WCAG) 2.0. Dec. 11, 2008.\n",
+      "https://www.w3.org/TR/WCAG20/\n",
+      "59. Reva Schwartz, Apostol Vassilev, Kristen Greene, Lori Perine, and Andrew Bert. NIST Special\n",
+      "Publication 1270: Towards a Standard for Identifying and Managing Bias in Artificial Intelligence. The\n",
+      "National Institute of Standards and Technology. March, 2022. https://nvlpubs.nist.gov/nistpubs/\n",
+      "SpecialPublications/NIST.SP.1270.pdf\n",
+      "60. See, e.g., the 2014 Federal Trade Commission report “Data Brokers A Call for Transparency and\n",
+      "Accountability”. https://www.ftc.gov/system/files/documents/reports/data-brokers-call-transparency­\n",
+      "accountability-report-federal-trade-commission-may-2014/140527databrokerreport.pdf\n",
+      "61. See, e.g., Nir Kshetri. School surveillance of students via laptops may do more harm than good. The\n",
+      "Conversation. Jan. 21, 2022.\n",
+      "https://theconversation.com/school-surveillance-of-students-via-laptops-may-do-more-harm-than­\n",
+      "good-170983; Matt Scherer. Warning: Bossware May be Hazardous to Your Health. Center for Democracy\n",
+      "& Technology Report.\n",
+      "https://cdt.org/wp-content/uploads/2021/07/2021-07-29-Warning-Bossware-May-Be-Hazardous-To­\n",
+      "Your-Health-Final.pdf; Human Impact Partners and WWRC. The Public Health Crisis Hidden in Amazon\n",
+      "Warehouses. HIP and WWRC report. Jan. 2021.\n",
+      "https://humanimpact.org/wp-content/uploads/2021/01/The-Public-Health-Crisis-Hidden-In-Amazon­\n",
+      "Warehouses-HIP-WWRC-01-21.pdf; Drew Harwell. Contract lawyers face a growing invasion of\n",
+      "surveillance programs that monitor their work. The Washington Post. Nov. 11, 2021. https://\n",
+      "www.washingtonpost.com/technology/2021/11/11/lawyer-facial-recognition-monitoring/;\n",
+      "Virginia Doellgast and Sean O'Brady. Making Call Center Jobs Better: The Relationship between\n",
+      "Management Practices and Worker Stress. A Report for the CWA. June 2020. https://\n",
+      "hdl.handle.net/1813/74307\n",
+      "62. See, e.g., Federal Trade Commission. Data Brokers: A Call for Transparency and Accountability. May\n",
+      "2014.\n",
+      "https://www.ftc.gov/system/files/documents/reports/data-brokers-call-transparency-accountability­\n",
+      "report-federal-trade-commission-may-2014/140527databrokerreport.pdf; Cathy O’Neil.\n",
+      "Weapons of Math Destruction. Penguin Books. 2017.\n",
+      "https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction\n",
+      "63. See, e.g., Rachel Levinson-Waldman, Harsha Pandurnga, and Faiza Patel. Social Media Surveillance by\n",
+      "the U.S. Government. Brennan Center for Justice. Jan. 7, 2022.\n",
+      "https://www.brennancenter.org/our-work/research-reports/social-media-surveillance-us-government;\n",
+      "Shoshana Zuboff. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of\n",
+      "Power. Public Affairs. 2019.\n",
+      "64. Angela Chen. Why the Future of Life Insurance May Depend on Your Online Presence. The Verge. Feb.\n",
+      "7, 2019.\n",
+      "https://www.theverge.com/2019/2/7/18211890/social-media-life-insurance-new-york-algorithms-big­\n",
+      "data-discrimination-online-records\n",
+      "68\n",
+      "{'source': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'document_id': 'doc2', '_id': 'ce568df861214be999caf4c0dac9d07d', '_collection_name': '527fdbdd8d4148b48f493df2f8334866'}\n",
+      "---\n",
+      "ENDNOTES\n",
+      "57. ISO Technical Management Board. ISO/IEC Guide 71:2014. Guide for addressing accessibility in\n",
+      "standards. International Standards Organization. 2021. https://www.iso.org/standard/57385.html\n",
+      "58. World Wide Web Consortium. Web Content Accessibility Guidelines (WCAG) 2.0. Dec. 11, 2008.\n",
+      "https://www.w3.org/TR/WCAG20/\n",
+      "59. Reva Schwartz, Apostol Vassilev, Kristen Greene, Lori Perine, and Andrew Bert. NIST Special\n",
+      "Publication 1270: Towards a Standard for Identifying and Managing Bias in Artificial Intelligence. The\n",
+      "National Institute of Standards and Technology. March, 2022. https://nvlpubs.nist.gov/nistpubs/\n",
+      "SpecialPublications/NIST.SP.1270.pdf\n",
+      "60. See, e.g., the 2014 Federal Trade Commission report “Data Brokers A Call for Transparency and\n",
+      "Accountability”. https://www.ftc.gov/system/files/documents/reports/data-brokers-call-transparency­\n",
+      "accountability-report-federal-trade-commission-may-2014/140527databrokerreport.pdf\n",
+      "61. See, e.g., Nir Kshetri. School surveillance of students via laptops may do more harm than good. The\n",
+      "Conversation. Jan. 21, 2022.\n",
+      "https://theconversation.com/school-surveillance-of-students-via-laptops-may-do-more-harm-than­\n",
+      "good-170983; Matt Scherer. Warning: Bossware May be Hazardous to Your Health. Center for Democracy\n",
+      "& Technology Report.\n",
+      "https://cdt.org/wp-content/uploads/2021/07/2021-07-29-Warning-Bossware-May-Be-Hazardous-To­\n",
+      "Your-Health-Final.pdf; Human Impact Partners and WWRC. The Public Health Crisis Hidden in Amazon\n",
+      "Warehouses. HIP and WWRC report. Jan. 2021.\n",
+      "https://humanimpact.org/wp-content/uploads/2021/01/The-Public-Health-Crisis-Hidden-In-Amazon­\n",
+      "Warehouses-HIP-WWRC-01-21.pdf; Drew Harwell. Contract lawyers face a growing invasion of\n",
+      "surveillance programs that monitor their work. The Washington Post. Nov. 11, 2021. https://\n",
+      "www.washingtonpost.com/technology/2021/11/11/lawyer-facial-recognition-monitoring/;\n",
+      "Virginia Doellgast and Sean O'Brady. Making Call Center Jobs Better: The Relationship between\n",
+      "Management Practices and Worker Stress. A Report for the CWA. June 2020. https://\n",
+      "hdl.handle.net/1813/74307\n",
+      "62. See, e.g., Federal Trade Commission. Data Brokers: A Call for Transparency and Accountability. May\n",
+      "2014.\n",
+      "https://www.ftc.gov/system/files/documents/reports/data-brokers-call-transparency-accountability­\n",
+      "report-federal-trade-commission-may-2014/140527databrokerreport.pdf; Cathy O’Neil.\n",
+      "Weapons of Math Destruction. Penguin Books. 2017.\n",
+      "https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction\n",
+      "63. See, e.g., Rachel Levinson-Waldman, Harsha Pandurnga, and Faiza Patel. Social Media Surveillance by\n",
+      "the U.S. Government. Brennan Center for Justice. Jan. 7, 2022.\n",
+      "https://www.brennancenter.org/our-work/research-reports/social-media-surveillance-us-government;\n",
+      "Shoshana Zuboff. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of\n",
+      "Power. Public Affairs. 2019.\n",
+      "64. Angela Chen. Why the Future of Life Insurance May Depend on Your Online Presence. The Verge. Feb.\n",
+      "7, 2019.\n",
+      "https://www.theverge.com/2019/2/7/18211890/social-media-life-insurance-new-york-algorithms-big­\n",
+      "data-discrimination-online-records\n",
+      "68\n",
+      "{'source': 'Blueprint for an AI Bill of Rights', 'document_id': 'doc1', '_id': '0dea504571ec413bbb96e84737f87a19', '_collection_name': '527fdbdd8d4148b48f493df2f8334866'}\n",
+      "---\n"
+     ]
+    }
+   ],
+   "source": [
+    "query = \"tell me about Karen Hao\"\n",
+    "results = qdrant_retriever.get_relevant_documents(query)\n",
+    "\n",
+    "for result in results:\n",
+    "    print(result.page_content)\n",
+    "    print(result.metadata)\n",
+    "    print(\"---\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "content=\"I don't have enough information, sorry.\" response_metadata={'token_usage': {'completion_tokens': 8, 'prompt_tokens': 112, 'total_tokens': 120, 'completion_tokens_details': {'reasoning_tokens': 0}}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_52a7f40b0b', 'finish_reason': 'stop', 'logprobs': None} id='run-511e9c6e-8c54-4ca3-8991-18bb3ba7c744-0' usage_metadata={'input_tokens': 112, 'output_tokens': 8, 'total_tokens': 120}\n"
+     ]
+    }
+   ],
+   "source": [
+    "from langchain_core.prompts import ChatPromptTemplate\n",
+    "from langchain_openai import ChatOpenAI\n",
+    "system_template = \"\"\"\n",
+    "    You are an expert at explaining technical documents to people.\n",
+    "    You are provided context below to answer the question.\n",
+    "    Only use the information provided below.\n",
+    "    If you cannot answer the question with the content below say 'I don't have enough information, sorry'\n",
+    "    The two documents are 'Blueprint for an AI Bill of Rights' and 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile'\n",
+    "\"\"\"\n",
+    "human_template = \"\"\" \n",
+    "===\n",
+    "question:\n",
+    "{question}\n",
+    "\n",
+    "===\n",
+    "context:\n",
+    "{context}\n",
+    "===\n",
+    "\"\"\"\n",
+    "chat_prompt = ChatPromptTemplate.from_messages([\n",
+    "    (\"system\", system_template),\n",
+    "    (\"human\", human_template)\n",
+    "])\n",
+    "# create the chain\n",
+    "openai_chat_model = ChatOpenAI(model=\"gpt-4o\")\n",
+    "chain = chat_prompt | openai_chat_model\n",
+    "print(chain.invoke({\"question\": \"Can you give me a summary of the 2 documents\", \"context\":\"\"})) # displays \n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{'response': AIMessage(content='The AI Bill of Rights, officially known as the \"Blueprint for an AI Bill of Rights,\" is a set of five principles and associated practices designed to guide the design, use, and deployment of automated systems. Its primary goal is to protect the rights of the American public in the age of artificial intelligence (AI). Developed through extensive consultation with various stakeholders, including impacted communities, industry stakeholders, technology developers, and policymakers, these principles are intended to ensure that AI systems are aligned with democratic values and protect civil rights, civil liberties, and privacy. The Blueprint provides a national values statement and a toolkit that is sector-agnostic, meaning it can be applied across different sectors to inform policy decisions and the technological design process.', response_metadata={'token_usage': {'completion_tokens': 144, 'prompt_tokens': 1848, 'total_tokens': 1992, 'completion_tokens_details': {'reasoning_tokens': 0}}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_52a7f40b0b', 'finish_reason': 'stop', 'logprobs': None}, id='run-eb76dbb2-facd-471e-9752-580bd7f46fc4-0', usage_metadata={'input_tokens': 1848, 'output_tokens': 144, 'total_tokens': 1992}), 'context': [Document(metadata={'source': 'Blueprint for an AI Bill of Rights', 'document_id': 'doc1', '_id': '7bacb7e37ad04ed0881408b49dfa5bd2', '_collection_name': 'ffb2f8aece51430086938e04161466e4'}, page_content='other protected data. Such activities require alternative, compatible safeguards through existing policies that \\ngovern automated systems and AI, such as the Department of Defense (DOD) AI Ethical Principles and \\nResponsible AI Implementation Pathway and the Intelligence Community (IC) AI Ethics Principles and \\nFramework. The implementation of these policies to national security and defense activities can be informed by \\nthe Blueprint for an AI Bill of Rights where feasible. \\nThe Blueprint for an AI Bill of Rights is not intended to, and does not, create any legal right, benefit, or \\ndefense, substantive or procedural, enforceable at law or in equity by any party against the United States, its \\ndepartments, agencies, or entities, its officers, employees, or agents, or any other person, nor does it constitute a \\nwaiver of sovereign immunity. \\nCopyright Information \\nThis document is a work of the United States Government and is in the public domain (see 17 U.S.C. §105). \\n2'), Document(metadata={'source': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'document_id': 'doc2', '_id': '0b5b8214b946430284b040e7e1a8a027', '_collection_name': 'ffb2f8aece51430086938e04161466e4'}, page_content='other protected data. Such activities require alternative, compatible safeguards through existing policies that \\ngovern automated systems and AI, such as the Department of Defense (DOD) AI Ethical Principles and \\nResponsible AI Implementation Pathway and the Intelligence Community (IC) AI Ethics Principles and \\nFramework. The implementation of these policies to national security and defense activities can be informed by \\nthe Blueprint for an AI Bill of Rights where feasible. \\nThe Blueprint for an AI Bill of Rights is not intended to, and does not, create any legal right, benefit, or \\ndefense, substantive or procedural, enforceable at law or in equity by any party against the United States, its \\ndepartments, agencies, or entities, its officers, employees, or agents, or any other person, nor does it constitute a \\nwaiver of sovereign immunity. \\nCopyright Information \\nThis document is a work of the United States Government and is in the public domain (see 17 U.S.C. §105). \\n2'), Document(metadata={'source': 'Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile', 'document_id': 'doc2', '_id': '4d87b814e59541b2ae7ee2d3bd110f96', '_collection_name': 'ffb2f8aece51430086938e04161466e4'}, page_content='ABOUT THIS FRAMEWORK\\xad\\xad\\xad\\xad\\xad\\nThe Blueprint for an AI Bill of Rights is a set of five principles and associated practices to help guide the \\ndesign, use, and deployment of automated systems to protect the rights of the American public in the age of \\nartificial intel-ligence. Developed through extensive consultation with the American public, these principles are \\na blueprint for building and deploying automated systems that are aligned with democratic values and protect \\ncivil rights, civil liberties, and privacy. The Blueprint for an AI Bill of Rights includes this Foreword, the five \\nprinciples, notes on Applying the The Blueprint for an AI Bill of Rights, and a Technical Companion that gives \\nconcrete steps that can be taken by many kinds of organizations—from governments at all levels to companies of \\nall sizes—to uphold these values. Experts from across the private sector, governments, and international \\nconsortia have published principles and frameworks to guide the responsible use of automated systems; this \\nframework provides a national values statement and toolkit that is sector-agnostic to inform building these \\nprotections into policy, practice, or the technological design process.  Where existing law or policy—such as \\nsector-specific privacy laws and oversight requirements—do not already provide guidance, the Blueprint for an \\nAI Bill of Rights should be used to inform policy decisions.\\nLISTENING TO THE AMERICAN PUBLIC\\nThe White House Office of Science and Technology Policy has led a year-long process to seek and distill input \\nfrom people across the country—from impacted communities and industry stakeholders to technology develop-\\ners and other experts across fields and sectors, as well as policymakers throughout the Federal government—on \\nthe issue of algorithmic and data-driven harms and potential remedies. Through panel discussions, public listen-\\ning sessions, meetings, a formal request for information, and input to a publicly accessible and widely-publicized \\nemail address, people throughout the United States, public servants across Federal agencies, and members of the \\ninternational community spoke up about both the promises and potential harms of these technologies, and \\nplayed a central role in shaping the Blueprint for an AI Bill of Rights. The core messages gleaned from these \\ndiscussions include that AI has transformative potential to improve Americans’ lives, and that preventing the \\nharms of these technologies is both necessary and achievable. The Appendix includes a full list of public engage-\\nments. \\n4'), Document(metadata={'source': 'Blueprint for an AI Bill of Rights', 'document_id': 'doc1', '_id': 'd1a69baaaf6244b28fc90ebf898a5d92', '_collection_name': 'ffb2f8aece51430086938e04161466e4'}, page_content='ABOUT THIS FRAMEWORK\\xad\\xad\\xad\\xad\\xad\\nThe Blueprint for an AI Bill of Rights is a set of five principles and associated practices to help guide the \\ndesign, use, and deployment of automated systems to protect the rights of the American public in the age of \\nartificial intel-ligence. Developed through extensive consultation with the American public, these principles are \\na blueprint for building and deploying automated systems that are aligned with democratic values and protect \\ncivil rights, civil liberties, and privacy. The Blueprint for an AI Bill of Rights includes this Foreword, the five \\nprinciples, notes on Applying the The Blueprint for an AI Bill of Rights, and a Technical Companion that gives \\nconcrete steps that can be taken by many kinds of organizations—from governments at all levels to companies of \\nall sizes—to uphold these values. Experts from across the private sector, governments, and international \\nconsortia have published principles and frameworks to guide the responsible use of automated systems; this \\nframework provides a national values statement and toolkit that is sector-agnostic to inform building these \\nprotections into policy, practice, or the technological design process.  Where existing law or policy—such as \\nsector-specific privacy laws and oversight requirements—do not already provide guidance, the Blueprint for an \\nAI Bill of Rights should be used to inform policy decisions.\\nLISTENING TO THE AMERICAN PUBLIC\\nThe White House Office of Science and Technology Policy has led a year-long process to seek and distill input \\nfrom people across the country—from impacted communities and industry stakeholders to technology develop-\\ners and other experts across fields and sectors, as well as policymakers throughout the Federal government—on \\nthe issue of algorithmic and data-driven harms and potential remedies. Through panel discussions, public listen-\\ning sessions, meetings, a formal request for information, and input to a publicly accessible and widely-publicized \\nemail address, people throughout the United States, public servants across Federal agencies, and members of the \\ninternational community spoke up about both the promises and potential harms of these technologies, and \\nplayed a central role in shaping the Blueprint for an AI Bill of Rights. The core messages gleaned from these \\ndiscussions include that AI has transformative potential to improve Americans’ lives, and that preventing the \\nharms of these technologies is both necessary and achievable. The Appendix includes a full list of public engage-\\nments. \\n4')]}\n",
+      "The AI Bill of Rights, officially known as the \"Blueprint for an AI Bill of Rights,\" is a set of five principles and associated practices designed to guide the design, use, and deployment of automated systems. Its primary goal is to protect the rights of the American public in the age of artificial intelligence (AI). Developed through extensive consultation with various stakeholders, including impacted communities, industry stakeholders, technology developers, and policymakers, these principles are intended to ensure that AI systems are aligned with democratic values and protect civil rights, civil liberties, and privacy. The Blueprint provides a national values statement and a toolkit that is sector-agnostic, meaning it can be applied across different sectors to inform policy decisions and the technological design process.\n",
+      "Number of found context: 4\n"
+     ]
+    }
+   ],
+   "source": [
+    "from operator import itemgetter\n",
+    "from langchain.schema.output_parser import StrOutputParser\n",
+    "from langchain.schema.runnable import RunnablePassthrough\n",
+    "retrieval_augmented_qa_chain = (\n",
+    "    {\"context\": itemgetter(\"question\") | qdrant_retriever, \"question\": itemgetter(\"question\")}\n",
+    "    | RunnablePassthrough.assign(context=itemgetter(\"context\"))\n",
+    "\n",
+    "\n",
+    "    | {\"response\": chat_prompt | openai_chat_model, \"context\": itemgetter(\"context\")}\n",
+    ")\n",
+    "response = retrieval_augmented_qa_chain.invoke({\"question\" : \"What is the AI Bill of Rights \"})\n",
+    "print(response)\n",
+    "print(response[\"response\"].content)\n",
+    "print(f\"Number of found context: {len(response['context'])}\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "llmops-course",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

requirements.txt

@@ -0,0 +1,13 @@
+numpy==1.26.4
+chainlit==0.7.700   # 1.1.402 
+openai==1.3.5
+pymupdf==1.24.9
+qdrant-client==1.11.0
+langchain-text-splitters
+langchain-core==0.2.27 
+langchain-community==0.2.10
+langchain-experimental==0.0.64 
+langgraph-checkpoint==1.0.6 
+langgraph==0.2.16 
+langchain-qdrant==0.1.3
+langchain-openai==0.1.9

utilities/debugger.py

@@ -0,0 +1,3 @@
+def dprint(state, msg):
+    if state.debug == True:
+        print(msg)

utilities/get_documents.py

@@ -0,0 +1,33 @@
+import requests
+import os
+from langchain.document_loaders import PyMuPDFLoader
+
+# Define the URLs for the documents
+url_1 = "https://example.com/Blueprint-for-an-AI-Bill-of-Rights.pdf"
+url_2 = "https://example.com/NIST.AI.600-1.pdf"
+
+# Define local file paths for storing the downloaded PDFs
+file_path_1 = "data/Blueprint-for-an-AI-Bill-of-Rights.pdf"
+file_path_2 = "data/NIST.AI.600-1.pdf"
+
+# Function to download a file from a URL
+def download_pdf(url, file_path):
+    # Check if the file already exists to avoid re-downloading
+    if not os.path.exists(file_path):
+        print(f"Downloading {file_path} from {url}...")
+        response = requests.get(url)
+        with open(file_path, 'wb') as f:
+            f.write(response.content)
+    else:
+        print(f"{file_path} already exists, skipping download.")
+        
+# Download the PDFs from the URLs
+download_pdf(url_1, file_path_1)
+download_pdf(url_2, file_path_2)
+
+# Load the PDFs using PyMuPDFLoader
+loader_1 = PyMuPDFLoader(file_path_1)
+documents_1 = loader_1.load()
+
+loader_2 = PyMuPDFLoader(file_path_2)
+documents_2 = loader_2.load()

utilities/pipeline.py

@@ -0,0 +1,27 @@
+from utilities_2.vectordatabase import VectorDatabase
+
+class RetrievalAugmentedQAPipeline:
+    def __init__(self, llm, vector_db_retriever: VectorDatabase, 
+                 system_role_prompt, user_role_prompt
+        ) -> None:
+        self.llm = llm
+        self.vector_db_retriever = vector_db_retriever
+        self.system_role_prompt = system_role_prompt
+        self.user_role_prompt = user_role_prompt
+
+    async def arun_pipeline(self, user_query: str):
+        context_list = self.vector_db_retriever.search_by_text(user_query, k=4)
+
+        context_prompt = ""
+        for context in context_list:
+            context_prompt += context[0] + "\n"
+
+        formatted_system_prompt = self.system_role_prompt.create_message()
+
+        formatted_user_prompt = self.user_role_prompt.create_message(question=user_query, context=context_prompt)
+
+        async def generate_response():
+            async for chunk in self.llm.astream([formatted_system_prompt, formatted_user_prompt]):
+                yield chunk
+
+        return {"response": generate_response(), "context": context_list}

utilities/rag_utilities.py

@@ -0,0 +1,109 @@
+from langchain.text_splitter import RecursiveCharacterTextSplitter
+from langchain.docstore.document import Document
+from langchain_community.document_loaders import PyMuPDFLoader
+from langchain_community.vectorstores import Qdrant
+from langchain_openai.embeddings import OpenAIEmbeddings
+import fitz
+import io
+import tiktoken
+import requests
+import os
+from utilities.debugger import dprint
+
+def tiktoken_len(text):
+    tokens = tiktoken.encoding_for_model("gpt-4o").encode(
+        text,
+    )
+    return len(tokens)
+
+def download_document(state, url, file_name, download_folder):
+    file_path = os.path.join(download_folder, file_name)
+    if not os.path.exists(download_folder):
+        os.makedirs(download_folder)
+    
+    if not os.path.exists(file_path):
+        print(f"Downloading {file_name} from {url}...")
+        response = requests.get(url)
+        if response.status_code == 200:
+            with open(file_path, 'wb') as f:
+                f.write(response.content)
+        else:
+            dprint(state, f"Failed to download document from {url}. Status code: {response.status_code}")
+    else:
+        dprint(state, f"{file_name} already exists locally.")
+    return file_path
+
+def get_documents(state):
+    for url in state.document_urls:
+        dprint(state, f"Downloading and loading document from {url}...")
+        file_name = url.split("/")[-1]
+        file_path = download_document(state, url, file_name, state.download_folder)
+        loader = PyMuPDFLoader(file_path)
+        loaded_document = loader.load()
+        single_text_document = "\n".join([doc.page_content for doc in loaded_document])
+        #state.add_loaded_document(loaded_document)  # Append the loaded documents to the list
+        #state.add_single_text_document(single_text_document)
+        dprint(state, f"Number of pages: {len(loaded_document)}")
+        # lets get titles and metadata
+        pdf = fitz.open(file_path)
+        metadata = pdf.metadata
+        title = metadata.get('title', 'Document 1')
+        #state.add_metadata(metadata)
+        #state.add_title(title)
+        document = {
+            "url": url,
+            "title": title,
+            "metadata": metadata,
+            "single_text_document": single_text_document,
+        }
+        state.add_document(document)
+        dprint(state, f"Title of Document: {title}")
+        dprint(state, f"Full metadata for Document 1: {metadata}")
+        pdf.close()
+    dprint(state, f"documents: {state.documents}")
+
+def create_chunked_documents(state):
+    get_documents(state)
+    # file_path_1 = "data/Blueprint-for-an-AI-Bill-of-Rights.pdf"
+    # file_path_2 = "data/NIST.AI.600-1.pdf"
+    # loader = PyMuPDFLoader(file_path_1)
+    # documents_1 = loader.load()
+    # loader = PyMuPDFLoader(file_path_2)
+    # documents_2 = loader.load()
+    # print(f"Number of pages in 1: {len(documents_1)}")
+    # print(f"Number of pages in 2: {len(documents_2)}")
+
+    
+    text_splitter = RecursiveCharacterTextSplitter(
+        chunk_size=state.chunk_size,  
+        chunk_overlap=state.chunk_overlap,
+        length_function = tiktoken_len,
+    )
+    combined_document_objects = []
+
+    dprint(state, "Chunking documents and creating document objects")
+    for document in state.documents:
+        dprint(state, f"processing documend: {document['title']}")
+        text = document["single_text_document"]
+        dprint(state, text)
+        title = document["title"]
+        chunks_document = text_splitter.split_text(text)
+        dprint(state, len(chunks_document))
+        document_objects = [Document(page_content=chunk, metadata={"source": title, "document_id": "doc1"}) for chunk in chunks_document]
+        dprint(state, f"Number of chunks for Document: {len(chunks_document)}")
+        combined_document_objects = combined_document_objects + document_objects
+    state.add_combined_document_objects(combined_document_objects)
+    
+
+def create_vector_store(state):
+    create_chunked_documents(state)
+    embedding_model = OpenAIEmbeddings(model=state.embedding_model)
+
+    qdrant_vectorstore = Qdrant.from_documents(
+        documents=state.combined_document_objects,
+        embedding=embedding_model,
+        location=":memory:" 
+    )
+    qdrant_retriever = qdrant_vectorstore.as_retriever()
+    state.set_retriever(qdrant_retriever)
+    return qdrant_retriever

utilities/text_utils.py

@@ -0,0 +1,103 @@
+import os
+from typing import List
+import fitz # pymupdf
+import tempfile
+from utilities_2.text_utils import CharacterTextSplitter
+from langchain_text_splitters import RecursiveCharacterTextSplitter
+
+# load the file
+# handle .txt and .pdf
+        
+class FileLoader:
+    
+    def __init__(self, encoding: str = "utf-8"):       
+        self.documents = []
+        self.encoding = encoding
+        self.temp_file_path = ""
+
+
+    def load_file(self, file, use_rct):
+        if use_rct:
+            text_splitter=MyRecursiveCharacterTextSplitter()
+        else:
+            text_splitter=CharacterTextSplitter()
+        file_extension = os.path.splitext(file.name)[1].lower()
+
+        with tempfile.NamedTemporaryFile(mode="wb", delete=False, suffix=file_extension) as temp_file:
+            self.temp_file_path = temp_file.name
+            temp_file.write(file.content)
+
+        if os.path.isfile(self.temp_file_path):
+            if self.temp_file_path.endswith(".txt"):
+                self.load_text_file()
+            elif self.temp_file_path.endswith(".pdf"):
+                self.load_pdf_file()
+            else:
+                raise ValueError(
+                    f"Unsupported file type: {self.temp_file_path}"
+                )
+            return text_splitter.split_text(self.documents)
+        else:
+            raise ValueError(
+                    "Not a file"
+                )
+
+    def load_text_file(self):
+        with open(self.temp_file_path, "r", encoding=self.encoding) as f:
+            self.documents.append(f.read())
+
+    def load_pdf_file(self):
+        # pymupdf
+        pdf_document = fitz.open(self.temp_file_path)
+        for page_num in range(len(pdf_document)):
+            page = pdf_document.load_page(page_num)
+            text = page.get_text()
+            self.documents.append(text)
+
+class CharacterTextSplitter:
+    def __init__(
+        self,
+        chunk_size: int = 1000,
+        chunk_overlap: int = 200,
+    ):
+        assert (
+            chunk_size > chunk_overlap
+        ), "Chunk size must be greater than chunk overlap"
+
+        self.chunk_size = chunk_size
+        self.chunk_overlap = chunk_overlap
+
+    def split(self, text: str) -> List[str]:
+        chunks = []
+        for i in range(0, len(text), self.chunk_size - self.chunk_overlap):
+            chunks.append(text[i : i + self.chunk_size])
+        return chunks
+
+    def split_text(self, texts: List[str]) -> List[str]:
+        chunks = []
+        for text in texts:
+            chunks.extend(self.split(text))
+        return chunks
+
+    
+
+class MyRecursiveCharacterTextSplitter:
+    # uses langChain.RecursiveCharacterTextSplitter
+    def __init__(
+        self
+    ):
+        self.RCTS = RecursiveCharacterTextSplitter(
+            chunk_size=1000,
+            chunk_overlap=20,
+            length_function=len,
+            separators=["\n\n", "\n", " ", ""]
+        )
+
+    def split_text(self, texts: List[str]) -> List[str]:
+        all_chunks = []
+        for doc in texts:
+            chunks = self.RCTS.split_text(doc)
+            all_chunks.extend(chunks)
+        return all_chunks
+
+

utilities/vector_database.py

@@ -0,0 +1,105 @@
+import numpy as np
+from collections import defaultdict
+from typing import List, Tuple, Callable
+from utilities_2.openai_utils.embedding import EmbeddingModel
+import hashlib
+from qdrant_client import QdrantClient
+from qdrant_client.http.models import PointStruct
+from qdrant_client.models import VectorParams
+import uuid
+
+def cosine_similarity(vector_a: np.array, vector_b: np.array) -> float:
+    """Computes the cosine similarity between two vectors."""
+    dot_product = np.dot(vector_a, vector_b)
+    norm_a = np.linalg.norm(vector_a)
+    norm_b = np.linalg.norm(vector_b)
+    return dot_product / (norm_a * norm_b)
+
+class QdrantDatabase:
+    def __init__(self, embedding_model=None):
+        self.qdrant_client = QdrantClient(location=":memory:")
+        self.collection_name = "my_collection"
+        self.embedding_model = embedding_model or EmbeddingModel(embeddings_model_name= "text-embedding-3-small", dimensions=1000)
+        vector_params = VectorParams(
+            size=self.embedding_model.dimensions,  # vector size 
+            distance="Cosine"
+          )  # distance metric
+        self.qdrant_client.create_collection(
+            collection_name=self.collection_name,
+            vectors_config={"text": vector_params},
+        )
+        self.vectors = defaultdict(np.array)  # Still keeps a local copy if needed
+
+    def string_to_int_id(self, s: str) -> int:
+        return int(hashlib.sha256(s.encode('utf-8')).hexdigest(), 16) % (10**8)
+    def get_test_vector(self):
+        retrieved_vector = self.qdrant_client.retrieve(
+                collection_name="my_collection",
+                ids=[self.string_to_int_id("test_key")]
+            )
+        return retrieved_vector
+    def insert(self, key: str, vector: np.array) -> None:
+        point_id = str(uuid.uuid4())
+        payload = {"text": key}  
+
+        point = PointStruct(
+            id=point_id,
+            vector={"default": vector.tolist()},
+            payload=payload
+        )
+        print(f"Inserting vector for key: {key}, ID: {point_id}")
+        # Insert the vector into Qdrant with the associated document
+        self.qdrant_client.upsert(
+            collection_name=self.collection_name,
+            points=[point]  # Qdrant expects a list of PointStruct
+        )
+
+
+    def search(
+        self,
+        query_vector: np.array,
+        k: int=5,
+        distance_measure: Callable = cosine_similarity,
+    ) -> List[Tuple[str, float]]:
+        # Perform search in Qdrant
+        if isinstance(query_vector, np.ndarray):
+            query_vector = query_vector.tolist()
+        print(type(query_vector)) 
+        search_results = self.qdrant_client.search(
+            collection_name=self.collection_name,
+            query_vector=query_vector,  # Pass the vector as a list
+            limit=k
+        )
+        return [(result.payload['text'], result.score) for result in search_results]
+
+    def search_by_text(
+        self,
+        query_text: str,
+        k: int,
+        distance_measure: Callable = cosine_similarity,
+        return_as_text: bool = False,
+    ) -> List[Tuple[str, float]]:
+        
+        query_vector = self.embedding_model.get_embedding(query_text)
+        results = self.search(query_vector, k, distance_measure)
+        return [result[0] for result in results] if return_as_text else results
+
+    async def abuild_from_list(self, list_of_text: List[str]) -> "QdrantDatabase":
+        from qdrant_client.http import models
+        embeddings = await self.embedding_model.async_get_embeddings(list_of_text)
+        points = [
+            models.PointStruct(
+            id=str(uuid.uuid4()),
+            vector={"text": embedding}, # Should be a named vector as per vector_config
+            payload={
+                "text": text
+                }
+            )
+            for text, embedding in zip(list_of_text, embeddings)
+        ]
+        self.qdrant_client.upsert(
+            collection_name=self.collection_name,
+            points=points
+        )
+        return self
+    

utilities_2/openai_utils/chatmodel.py

@@ -0,0 +1,45 @@
+from openai import OpenAI, AsyncOpenAI
+from dotenv import load_dotenv
+import os
+
+load_dotenv()
+
+
+class ChatOpenAI:
+    def __init__(self, model_name: str = "gpt-4o-mini"):
+        self.model_name = model_name
+        self.openai_api_key = os.getenv("OPENAI_API_KEY")
+        if self.openai_api_key is None:
+            raise ValueError("OPENAI_API_KEY is not set")
+
+    def run(self, messages, text_only: bool = True, **kwargs):
+        if not isinstance(messages, list):
+            raise ValueError("messages must be a list")
+
+        client = OpenAI()
+        response = client.chat.completions.create(
+            model=self.model_name, messages=messages, **kwargs
+        )
+
+        if text_only:
+            return response.choices[0].message.content
+
+        return response
+    
+    async def astream(self, messages, **kwargs):
+        if not isinstance(messages, list):
+            raise ValueError("messages must be a list")
+        
+        client = AsyncOpenAI()
+
+        stream = await client.chat.completions.create(
+            model=self.model_name,
+            messages=messages,
+            stream=True,
+            **kwargs
+        )
+
+        async for chunk in stream:
+            content = chunk.choices[0].delta.content
+            if content is not None:
+                yield content

utilities_2/openai_utils/embedding.py

@@ -0,0 +1,60 @@
+from dotenv import load_dotenv
+from openai import AsyncOpenAI, OpenAI
+import openai
+from typing import List
+import os
+import asyncio
+
+
+class EmbeddingModel:
+    def __init__(self, embeddings_model_name: str = "text-embedding-3-small", dimensions: int = None):
+        load_dotenv()
+        self.openai_api_key = os.getenv("OPENAI_API_KEY")
+        self.async_client = AsyncOpenAI()
+        self.client = OpenAI()
+        self.dimensions = dimensions
+
+        if self.openai_api_key is None:
+            raise ValueError(
+                "OPENAI_API_KEY environment variable is not set. Please set it to your OpenAI API key."
+            )
+        openai.api_key = self.openai_api_key
+        self.embeddings_model_name = embeddings_model_name
+
+    async def async_get_embeddings(self, list_of_text: List[str]) -> List[List[float]]:
+        embedding_response = await self.async_client.embeddings.create(
+            input=list_of_text, model=self.embeddings_model_name, dimensions=self.dimensions
+        )
+
+        return [embeddings.embedding for embeddings in embedding_response.data]
+
+    async def async_get_embedding(self, text: str) -> List[float]:
+        embedding = await self.async_client.embeddings.create(
+            input=text, model=self.embeddings_model_name, dimensions=self.dimensions
+        )
+
+        return embedding.data[0].embedding
+
+    def get_embeddings(self, list_of_text: List[str]) -> List[List[float]]:
+        embedding_response = self.client.embeddings.create(
+            input=list_of_text, model=self.embeddings_model_name, dimensions=self.dimensions
+        )
+
+        return [embeddings.embedding for embeddings in embedding_response.data]
+
+    def get_embedding(self, text: str) -> List[float]:
+        embedding = self.client.embeddings.create(
+            input=text, model=self.embeddings_model_name, dimensions=self.dimensions
+        )
+
+        return embedding.data[0].embedding
+
+
+if __name__ == "__main__":
+    embedding_model = EmbeddingModel()
+    print(asyncio.run(embedding_model.async_get_embedding("Hello, world!")))
+    print(
+        asyncio.run(
+            embedding_model.async_get_embeddings(["Hello, world!", "Goodbye, world!"])
+        )
+    )

utilities_2/openai_utils/prompts.py

@@ -0,0 +1,78 @@
+import re
+
+
+class BasePrompt:
+    def __init__(self, prompt):
+        """
+        Initializes the BasePrompt object with a prompt template.
+
+        :param prompt: A string that can contain placeholders within curly braces
+        """
+        self.prompt = prompt
+        self._pattern = re.compile(r"\{([^}]+)\}")
+
+    def format_prompt(self, **kwargs):
+        """
+        Formats the prompt string using the keyword arguments provided.
+
+        :param kwargs: The values to substitute into the prompt string
+        :return: The formatted prompt string
+        """
+        matches = self._pattern.findall(self.prompt)
+        return self.prompt.format(**{match: kwargs.get(match, "") for match in matches})
+
+    def get_input_variables(self):
+        """
+        Gets the list of input variable names from the prompt string.
+
+        :return: List of input variable names
+        """
+        return self._pattern.findall(self.prompt)
+
+
+class RolePrompt(BasePrompt):
+    def __init__(self, prompt, role: str):
+        """
+        Initializes the RolePrompt object with a prompt template and a role.
+
+        :param prompt: A string that can contain placeholders within curly braces
+        :param role: The role for the message ('system', 'user', or 'assistant')
+        """
+        super().__init__(prompt)
+        self.role = role
+
+    def create_message(self, format=True, **kwargs):
+        """
+        Creates a message dictionary with a role and a formatted message.
+
+        :param kwargs: The values to substitute into the prompt string
+        :return: Dictionary containing the role and the formatted message
+        """
+        if format:
+            return {"role": self.role, "content": self.format_prompt(**kwargs)}
+        
+        return {"role": self.role, "content": self.prompt}
+
+
+class SystemRolePrompt(RolePrompt):
+    def __init__(self, prompt: str):
+        super().__init__(prompt, "system")
+
+
+class UserRolePrompt(RolePrompt):
+    def __init__(self, prompt: str):
+        super().__init__(prompt, "user")
+
+
+class AssistantRolePrompt(RolePrompt):
+    def __init__(self, prompt: str):
+        super().__init__(prompt, "assistant")
+
+
+if __name__ == "__main__":
+    prompt = BasePrompt("Hello {name}, you are {age} years old")
+    print(prompt.format_prompt(name="John", age=30))
+
+    prompt = SystemRolePrompt("Hello {name}, you are {age} years old")
+    print(prompt.create_message(name="John", age=30))
+    print(prompt.get_input_variables())

utilities_2/text_utils.py

@@ -0,0 +1,75 @@
+import os
+from typing import List
+
+class TextFileLoader:
+    def __init__(self, path: str, encoding: str = "utf-8"):
+        self.documents = []
+        self.path = path
+        self.encoding = encoding
+
+    def load(self):
+        if os.path.isdir(self.path):
+            self.load_directory()
+        elif os.path.isfile(self.path) and self.path.endswith(".txt"):
+            self.load_file()
+        else:
+            raise ValueError(
+                "Provided path is neither a valid directory nor a .txt file."
+            )
+
+    def load_file(self):
+        with open(self.path, "r", encoding=self.encoding) as f:
+            self.documents.append(f.read())
+
+    def load_directory(self):
+        for root, _, files in os.walk(self.path):
+            for file in files:
+                if file.endswith(".txt"):
+                    with open(
+                        os.path.join(root, file), "r", encoding=self.encoding
+                    ) as f:
+                        self.documents.append(f.read())
+
+    def load_documents(self):
+        self.load()
+        return self.documents
+
+
+class CharacterTextSplitter:
+    def __init__(
+        self,
+        chunk_size: int = 1000,
+        chunk_overlap: int = 200,
+    ):
+        assert (
+            chunk_size > chunk_overlap
+        ), "Chunk size must be greater than chunk overlap"
+
+        self.chunk_size = chunk_size
+        self.chunk_overlap = chunk_overlap
+
+    def split(self, text: str) -> List[str]:
+        chunks = []
+        for i in range(0, len(text), self.chunk_size - self.chunk_overlap):
+            chunks.append(text[i : i + self.chunk_size])
+        return chunks
+
+    def split_text(self, texts: List[str]) -> List[str]:
+        chunks = []
+        for text in texts:
+            chunks.extend(self.split(text))
+        return chunks
+
+if __name__ == "__main__":
+    loader = TextFileLoader("data/KingLear.txt")
+    loader.load()
+    splitter = CharacterTextSplitter()
+    chunks = splitter.split_text(loader.documents)
+    print(len(chunks))
+    print(chunks[0])
+    print("--------")
+    print(chunks[1])
+    print("--------")
+    print(chunks[-2])
+    print("--------")
+    print(chunks[-1])

utilities_2/vectordatabase.py

@@ -0,0 +1,82 @@
+import numpy as np
+from collections import defaultdict
+from typing import List, Tuple, Callable
+from utilities_2.openai_utils.embedding import EmbeddingModel
+import asyncio
+
+
+def cosine_similarity(vector_a: np.array, vector_b: np.array) -> float:
+    """Computes the cosine similarity between two vectors."""
+    dot_product = np.dot(vector_a, vector_b)
+    norm_a = np.linalg.norm(vector_a)
+    norm_b = np.linalg.norm(vector_b)
+    return dot_product / (norm_a * norm_b)
+
+
+class VectorDatabase:
+    def __init__(self, embedding_model: EmbeddingModel = None):
+        self.vectors = defaultdict(np.array)
+        self.embedding_model = embedding_model or EmbeddingModel()
+
+    def insert(self, key: str, vector: np.array) -> None:
+        self.vectors[key] = vector
+
+    def search(
+        self,
+        query_vector: np.array,
+        k: int,
+        distance_measure: Callable = cosine_similarity,
+    ) -> List[Tuple[str, float]]:
+        scores = [
+            (key, distance_measure(query_vector, vector))
+            for key, vector in self.vectors.items()
+        ]
+        return sorted(scores, key=lambda x: x[1], reverse=True)[:k]
+
+    def search_by_text(
+        self,
+        query_text: str,
+        k: int,
+        distance_measure: Callable = cosine_similarity,
+        return_as_text: bool = False,
+    ) -> List[Tuple[str, float]]:
+        query_vector = self.embedding_model.get_embedding(query_text)
+        results = self.search(query_vector, k, distance_measure)
+        return [result[0] for result in results] if return_as_text else results
+
+    def retrieve_from_key(self, key: str) -> np.array:
+        return self.vectors.get(key, None)
+
+    async def abuild_from_list(self, list_of_text: List[str]) -> "VectorDatabase":
+        embeddings = await self.embedding_model.async_get_embeddings(list_of_text)
+        for text, embedding in zip(list_of_text, embeddings):
+            self.insert(text, np.array(embedding))
+        return self
+
+
+
+if __name__ == "__main__":
+    list_of_text = [
+        "I like to eat broccoli and bananas.",
+        "I ate a banana and spinach smoothie for breakfast.",
+        "Chinchillas and kittens are cute.",
+        "My sister adopted a kitten yesterday.",
+        "Look at this cute hamster munching on a piece of broccoli.",
+    ]
+
+    vector_db = VectorDatabase()
+    vector_db = asyncio.run(vector_db.abuild_from_list(list_of_text))
+    k = 2
+
+    searched_vector = vector_db.search_by_text("I think fruit is awesome!", k=k)
+    print(f"Closest {k} vector(s):", searched_vector)
+
+    retrieved_vector = vector_db.retrieve_from_key(
+        "I like to eat broccoli and bananas."
+    )
+    print("Retrieved vector:", retrieved_vector)
+
+    relevant_texts = vector_db.search_by_text(
+        "I think fruit is awesome!", k=k, return_as_text=True
+    )
+    print(f"Closest {k} text(s):", relevant_texts)