community_contributions/ecrg_3_lab3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Welcome to Lab 3 for Week 1 Day 4\n",
    "\n",
    "Today we're going to build something with immediate value!\n",
    "\n",
    "In the folder `me` I've put a single file `linkedin.pdf` - it's a PDF download of my LinkedIn profile.\n",
    "\n",
    "Please replace it with yours!\n",
    "\n",
    "I've also made a file called `summary.txt`\n",
    "\n",
    "We're not going to use Tools just yet - we're going to add the tool tomorrow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import necessary libraries:\n",
    "# - load_dotenv: Loads environment variables from a .env file (e.g., your OpenAI API key).\n",
    "# - OpenAI: The official OpenAI client to interact with their API.\n",
    "# - PdfReader: Used to read and extract text from PDF files.\n",
    "# - gr: Gradio is a UI library to quickly build web interfaces for machine learning apps.\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "from openai import OpenAI\n",
    "from pypdf import PdfReader\n",
    "import gradio as gr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "load_dotenv(override=True)\n",
    "openai = OpenAI()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This script reads a PDF file located at 'me/profile.pdf' and extracts all the text from each page.\n",
    "The extracted text is concatenated into a single string variable named 'linkedin'.\n",
    "This can be useful for feeding structured content (like a resume or profile) into an AI model or for further text processing.\n",
    "\"\"\"\n",
    "reader = PdfReader(\"me/profile.pdf\")\n",
    "linkedin = \"\"\n",
    "for page in reader.pages:\n",
    "    text = page.extract_text()\n",
    "    if text:\n",
    "        linkedin += text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This script loads a PDF file named 'projects.pdf' from the 'me' directory\n",
    "and extracts text from each page. The extracted text is combined into a single\n",
    "string variable called 'projects', which can be used later for analysis,\n",
    "summarization, or input into an AI model.\n",
    "\"\"\"\n",
    "\n",
    "reader = PdfReader(\"me/projects.pdf\")\n",
    "projects = \"\"\n",
    "for page in reader.pages:\n",
    "    text = page.extract_text()\n",
    "    if text:\n",
    "        projects += text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print for sanity checks\n",
    "\"Print for sanity checks\"\n",
    "\n",
    "print(linkedin)\n",
    "print(projects)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"me/summary.txt\", \"r\", encoding=\"utf-8\") as f:\n",
    "    summary = f.read()\n",
    "\n",
    "name = \"Cristina Rodriguez\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This code constructs a system prompt for an AI agent to role-play as a specific person (defined by `name`).\n",
    "The prompt guides the AI to answer questions as if it were that person, using their career summary,\n",
    "LinkedIn profile, and project information for context. The final prompt ensures that the AI stays\n",
    "in character and responds professionally and helpfully to visitors on the user's website.\n",
    "\"\"\"\n",
    "\n",
    "system_prompt = f\"You are acting as {name}. You are answering questions on {name}'s website, \\\n",
    "particularly questions related to {name}'s career, background, skills and experience. \\\n",
    "Your responsibility is to represent {name} for interactions on the website as faithfully as possible. \\\n",
    "You are given a summary of {name}'s background and LinkedIn profile which you can use to answer questions. \\\n",
    "Be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n",
    "If you don't know the answer, say so.\"\n",
    "\n",
    "system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\\n\\n## Projects:\\n{projects}\\n\\n\"\n",
    "system_prompt += f\"With this context, please chat with the user, always staying in character as {name}.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "system_prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This function handles a chat interaction with the OpenAI API.\n",
    "\n",
    "It takes the user's latest message and conversation history,\n",
    "prepends a system prompt to define the AI's role and context,\n",
    "and sends the full message list to the GPT-4o-mini model.\n",
    "\n",
    "The function returns the AI's response text from the API's output.\n",
    "\"\"\"\n",
    "\n",
    "def chat(message, history):\n",
    "    messages = [{\"role\": \"system\", \"content\": system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This line launches a Gradio chat interface using the `chat` function to handle user input.\n",
    "\n",
    "- `gr.ChatInterface(chat, type=\"messages\")` creates a UI that supports message-style chat interactions.\n",
    "- `launch(share=True)` starts the web app and generates a public shareable link so others can access it.\n",
    "\"\"\"\n",
    "\n",
    "gr.ChatInterface(chat, type=\"messages\").launch(share=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A lot is about to happen...\n",
    "\n",
    "1. Be able to ask an LLM to evaluate an answer\n",
    "2. Be able to rerun if the answer fails evaluation\n",
    "3. Put this together into 1 workflow\n",
    "\n",
    "All without any Agentic framework!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This code defines a Pydantic model named 'Evaluation' to structure evaluation data.\n",
    "\n",
    "The model includes:\n",
    "- is_acceptable (bool): Indicates whether the submission meets the criteria.\n",
    "- feedback (str): Provides written feedback or suggestions for improvement.\n",
    "\n",
    "Pydantic ensures type validation and data consistency.\n",
    "\"\"\"\n",
    "\n",
    "from pydantic import BaseModel\n",
    "\n",
    "class Evaluation(BaseModel):\n",
    "    is_acceptable: bool\n",
    "    feedback: str\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This code builds a system prompt for an AI evaluator agent.\n",
    "\n",
    "The evaluator's role is to assess the quality of an Agent's response in a simulated conversation,\n",
    "where the Agent is acting as {name} on their personal/professional website.\n",
    "\n",
    "The evaluator receives context including {name}'s summary and LinkedIn profile,\n",
    "and is instructed to determine whether the Agent's latest reply is acceptable,\n",
    "while providing constructive feedback.\n",
    "\"\"\"\n",
    "\n",
    "evaluator_system_prompt = f\"You are an evaluator that decides whether a response to a question is acceptable. \\\n",
    "You are provided with a conversation between a User and an Agent. Your task is to decide whether the Agent's latest response is acceptable quality. \\\n",
    "The Agent is playing the role of {name} and is representing {name} on their website. \\\n",
    "The Agent has been instructed to be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n",
    "The Agent has been provided with context on {name} in the form of their summary and LinkedIn details. Here's the information:\"\n",
    "\n",
    "evaluator_system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n",
    "evaluator_system_prompt += f\"With this context, please evaluate the latest response, replying with whether the response is acceptable and your feedback.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This function generates a user prompt for the evaluator agent.\n",
    "\n",
    "It organizes the full conversation context by including:\n",
    "- the full chat history,\n",
    "- the most recent user message,\n",
    "- and the most recent agent reply.\n",
    "\n",
    "The final prompt instructs the evaluator to assess the quality of the agent’s response,\n",
    "and return both an acceptability judgment and constructive feedback.\n",
    "\"\"\"\n",
    "\n",
    "def evaluator_user_prompt(reply, message, history):\n",
    "    user_prompt = f\"Here's the conversation between the User and the Agent: \\n\\n{history}\\n\\n\"\n",
    "    user_prompt += f\"Here's the latest message from the User: \\n\\n{message}\\n\\n\"\n",
    "    user_prompt += f\"Here's the latest response from the Agent: \\n\\n{reply}\\n\\n\"\n",
    "    user_prompt += f\"Please evaluate the response, replying with whether it is acceptable and your feedback.\"\n",
    "    return user_prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This script tests whether the Google Generative AI API key is working correctly.\n",
    "\n",
    "- It loads the API key from a .env file using `dotenv`.\n",
    "- Initializes a genai.Client with the loaded key.\n",
    "- Attempts to generate a simple response using the \"gemini-2.0-flash\" model.\n",
    "- Prints confirmation if the key is valid, or shows an error message if the request fails.\n",
    "\"\"\"\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "import os\n",
    "from google import genai\n",
    "\n",
    "load_dotenv()\n",
    "\n",
    "client = genai.Client(api_key=os.environ.get(\"GOOGLE_API_KEY\"))\n",
    "\n",
    "try:\n",
    "    # Use the correct method for genai.Client\n",
    "    test_response = client.models.generate_content(\n",
    "        model=\"gemini-2.0-flash\",\n",
    "        contents=\"Hello\"\n",
    "    )\n",
    "    print(\"✅ API key is working!\")\n",
    "    print(f\"Response: {test_response.text}\")\n",
    "except Exception as e:\n",
    "    print(f\"❌ API key test failed: {e}\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This line initializes an OpenAI-compatible client for accessing Google's Generative Language API.\n",
    "\n",
    "- `api_key` is retrieved from environment variables.\n",
    "- `base_url` points to Google's OpenAI-compatible endpoint.\n",
    "\n",
    "This setup allows you to use OpenAI-style syntax to interact with Google's Gemini models.\n",
    "\"\"\"\n",
    "\n",
    "gemini = OpenAI(\n",
    "    api_key=os.environ.get(\"GOOGLE_API_KEY\"),\n",
    "    base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This function sends a structured evaluation request to the Gemini API and returns a parsed `Evaluation` object.\n",
    "\n",
    "- It constructs the message list using:\n",
    "  - a system prompt defining the evaluator's role and context\n",
    "  - a user prompt containing the conversation history, user message, and agent reply\n",
    "\n",
    "- It uses Gemini's OpenAI-compatible API to process the evaluation request,\n",
    "  specifying `response_format=Evaluation` to get a structured response.\n",
    "\n",
    "- The function returns the parsed evaluation result (acceptability and feedback).\n",
    "\"\"\"\n",
    "\n",
    "def evaluate(reply, message, history) -> Evaluation:\n",
    "\n",
    "    messages = [{\"role\": \"system\", \"content\": evaluator_system_prompt}] + [{\"role\": \"user\", \"content\": evaluator_user_prompt(reply, message, history)}]\n",
    "    response = gemini.beta.chat.completions.parse(model=\"gemini-2.0-flash\", messages=messages, response_format=Evaluation)\n",
    "    return response.choices[0].message.parsed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This code sends a test question to the AI agent and evaluates its response.\n",
    "\n",
    "1. It builds a message list including:\n",
    "   - the system prompt that defines the agent’s role\n",
    "   - a user question: \"do you hold a patent?\"\n",
    "\n",
    "2. The message list is sent to OpenAI's GPT-4o-mini model to generate a response.\n",
    "\n",
    "3. The reply is extracted from the API response.\n",
    "\n",
    "4. The `evaluate()` function is then called with:\n",
    "   - the agent’s reply\n",
    "   - the original user message\n",
    "   - and just the system prompt as history (no prior user/agent exchange)\n",
    "\n",
    "This allows automated evaluation of how well the agent answers the question.\n",
    "\"\"\"\n",
    "\n",
    "messages = [{\"role\": \"system\", \"content\": system_prompt}] + [{\"role\": \"user\", \"content\": \"do you hold a patent?\"}]\n",
    "response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "reply = response.choices[0].message.content\n",
    "reply\n",
    "evaluate(reply, \"do you hold a patent?\", messages[:1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This function re-generates a response after a previous reply was rejected during evaluation.\n",
    "\n",
    "It:\n",
    "1. Appends rejection feedback to the original system prompt to inform the agent of:\n",
    "   - its previous answer,\n",
    "   - and the reason it was rejected.\n",
    "\n",
    "2. Reconstructs the full message list including:\n",
    "   - the updated system prompt,\n",
    "   - the prior conversation history,\n",
    "   - and the original user message.\n",
    "\n",
    "3. Sends the updated prompt to OpenAI's GPT-4o-mini model.\n",
    "\n",
    "4. Returns a revised response from the model that ideally addresses the feedback.\n",
    "\"\"\"\n",
    "def rerun(reply, message, history, feedback):\n",
    "    updated_system_prompt = system_prompt + f\"\\n\\n## Previous answer rejected\\nYou just tried to reply, but the quality control rejected your reply\\n\"\n",
    "    updated_system_prompt += f\"## Your attempted answer:\\n{reply}\\n\\n\"\n",
    "    updated_system_prompt += f\"## Reason for rejection:\\n{feedback}\\n\\n\"\n",
    "    messages = [{\"role\": \"system\", \"content\": updated_system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This function handles a chat interaction with conditional behavior and automatic quality control.\n",
    "\n",
    "Steps:\n",
    "1. If the user's message contains the word \"patent\", the agent is instructed to respond entirely in Pig Latin by appending an instruction to the system prompt.\n",
    "2. Constructs the full message history including the updated system prompt, prior conversation, and the new user message.\n",
    "3. Sends the request to OpenAI's GPT-4o-mini model and receives a reply.\n",
    "4. Evaluates the reply using a separate evaluator agent to determine if the response meets quality standards.\n",
    "5. If the evaluation passes, the reply is returned.\n",
    "6. If the evaluation fails, the function logs the feedback and calls `rerun()` to generate a corrected reply based on the feedback.\n",
    "\"\"\"\n",
    "\n",
    "def chat(message, history):\n",
    "    if \"patent\" in message:\n",
    "        system = system_prompt + \"\\n\\nEverything in your reply needs to be in pig latin - \\\n",
    "              it is mandatory that you respond only and entirely in pig latin\"\n",
    "    else:\n",
    "        system = system_prompt\n",
    "    messages = [{\"role\": \"system\", \"content\": system}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "    reply =response.choices[0].message.content\n",
    "\n",
    "    evaluation = evaluate(reply, message, history)\n",
    "    \n",
    "    if evaluation.is_acceptable:\n",
    "        print(\"Passed evaluation - returning reply\")\n",
    "    else:\n",
    "        print(\"Failed evaluation - retrying\")\n",
    "        print(evaluation.feedback)\n",
    "        reply = rerun(reply, message, history, evaluation.feedback)       \n",
    "    return reply"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\nThis launches a Gradio chat interface using the `chat` function.\\n\\n- `type=\"messages\"` enables multi-turn chat with message bubbles.\\n- `share=True` generates a public link so others can interact with the app.\\n'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"\"\"\n",
    "This launches a Gradio chat interface using the `chat` function.\n",
    "\n",
    "- `type=\"messages\"` enables multi-turn chat with message bubbles.\n",
    "- `share=True` generates a public link so others can interact with the app.\n",
    "\"\"\"\n",
    "gr.ChatInterface(chat, type=\"messages\").launch(share=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}