3_lab3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Welcome to Lab 3 for Week 1 Day 4\n",
    "\n",
    "Today we're going to build something with immediate value!\n",
    "\n",
    "In the folder `me` I've put a single file `linkedin.pdf` - it's a PDF download of my LinkedIn profile.\n",
    "\n",
    "Please replace it with yours!\n",
    "\n",
    "I've also made a file called `summary.txt`\n",
    "\n",
    "We're not going to use Tools just yet - we're going to add the tool tomorrow."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/tools.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#00bfff;\">Looking up packages</h2>\n",
    "            <span style=\"color:#00bfff;\">In this lab, we're going to use the wonderful Gradio package for building quick UIs, \n",
    "            and we're also going to use the popular PyPDF2 PDF reader. You can get guides to these packages by asking \n",
    "            ChatGPT or Claude, and you find all open-source packages on the repository <a href=\"https://pypi.org\">https://pypi.org</a>.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# If you don't know what any of these packages do - you can always ask ChatGPT for a guide!\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "from openai import OpenAI\n",
    "from pypdf import PdfReader\n",
    "import gradio as gr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "load_dotenv(override=True)\n",
    "openai = OpenAI()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "reader = PdfReader(\"me/linkedin.pdf\")\n",
    "linkedin = \"\"\n",
    "for page in reader.pages:\n",
    "    text = page.extract_text()\n",
    "    if text:\n",
    "        linkedin += text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MATEUSZ \n",
      "WOŹNIAK\n",
      "CTO | ENGINEERING MANAGER | TECHNICAL\n",
      "DIRECTOR | PROGRAM MANAGER WARSAW,\n",
      "POLAND \n",
      "EXPERIENCE\n",
      "Technical: .NET Core, Azure, SQL Server,\n",
      "Kubernetes, Docker, Angular, React,\n",
      "builder.io, n8n Tools: Jira, GitHub, Power\n",
      "BI, Google Workspace, Entity Framework,\n",
      "WebAPI and many others.\n",
      "TECHNICAL SKILLS\n",
      "Experienced technology leader\n",
      "with over 12 years of hands-on\n",
      "and strategic expertise in software\n",
      "development, engineering\n",
      "leadership, and digital\n",
      "transformation. Proven track\n",
      "record of aligning technical vision\n",
      "with business strategy, scaling\n",
      "cross-functional teams, and\n",
      "delivering high-impact projects\n",
      "across industries. Skilled in full-\n",
      "stack development, cloud-native\n",
      "solutions, and agile program\n",
      "delivery.\n",
      "🌟  EXECUTIVE SUMMARY\n",
      "Location: Warsaw\n",
      "Email: wozniakm@outlook.com; \n",
      "Phone: (+48) 793 730 855\n",
      "LinkedIn:\n",
      "https://www.linkedin.com/in/marvelous\n",
      "mateuszwozniak/\n",
      "HOW TO REACH ME\n",
      "Monogo – Program Manager\n",
      "May 2024 – March 2025\n",
      "Led a large-scale CRM migration involving tens of\n",
      "thousands of pages, serving as the primary Project\n",
      "Manager while overseeing multiple initiatives in\n",
      "builder.io as Program Manager for three project\n",
      "management teams.\n",
      "Key Responsibilities:\n",
      "Managed daily operations for international, English-\n",
      "speaking teams of 20+ professionals across\n",
      "multiple time zones.\n",
      "Acted as the Single Point of Contact (SPOC)\n",
      "between companies, ensuring smooth\n",
      "communication and strong client and stakeholder\n",
      "relationships.\n",
      "Coordinated cross-functional collaboration and\n",
      "provided technical guidance to keep projects on\n",
      "track.\n",
      "Created and managed project budgets, tracking\n",
      "costs and resource allocation.\n",
      "Facilitated daily client meetings and produced high-\n",
      "level stakeholder reports.\n",
      "Key Achievements:\n",
      "Successfully delivered a major Proof of Concept and\n",
      "multiple critical milestones under tight deadlines.\n",
      "Developed automated tooling that enabled the\n",
      "seamless migration of thousands of pages.\n",
      "Solved complex technical challenges and ensured\n",
      "project continuity.\n",
      "Introduced an effective Scrumfall methodology,\n",
      "blending agile flexibility with structured delivery.\n",
      "Technical Environment: TypeScript, JavaScript,\n",
      "builder.io\n",
      "Tools: Jira, Google Workspace, Microsoft Office\n",
      "Suite, analytics platformsEXPERIENCE\n",
      "QODECA - CTO \n",
      "JUNE 2021 – May 2023\n",
      "Develop technical aspects of the company’s\n",
      "strategy to ensure alignment with its business\n",
      "goals\n",
      "Strategic Planning on partners & project\n",
      "levels\n",
      "Discover and implement new technology\n",
      "solutions that yield a competitive advantage\n",
      "Developing policies and procedures to\n",
      "enhance products and services to ensure\n",
      "functionality and efficiency of company and\n",
      "partners\n",
      "Monitor KPIs and IT budgets to assess\n",
      "technological performance, budget & time\n",
      "frames.\n",
      "In charge of day-to-day work for multiple\n",
      "projects\n",
      "Mentored both engineers, tech leads and PM’s\n",
      "in structured 1on1s.\n",
      "British Council - Team Lead\n",
      "July 2023 - Feb 2024\n",
      "Returned as a senior leader to drive .NET project\n",
      "delivery for international educational services.\n",
      "Led a team of 9 engineers using Azure and\n",
      ".NET Core in an agile/scrum environment.\n",
      "Introduced QA and risk management\n",
      "processes, improving reliability and delivery\n",
      "confidence.\n",
      "Enhanced collaboration between technical and\n",
      "business stakeholders across time zones.\n",
      "West Agile Labs - Team Lead \n",
      "JUNE 2020 – June 2021 \n",
      "Managing day-to-day work for a team of 4\n",
      "Responsible for the project life cycle, analysis,\n",
      "estimates, implementation, to final delivery \n",
      "Analyzing systems dependencies and\n",
      "designing integrations between them. \n",
      "Estimating and analyzing high-level\n",
      "requirements. \n",
      "Assessing and verifying application releases\n",
      "including the deployment process\n",
      "Technology stack: .NET + CORE, Entity Framework\n",
      "+ CORE, RabbitMQ (Mass Transit), Elastic Search,\n",
      "MSSQL 2012+, Angular 8, React, Service Stack,\n",
      "Google Cloud, Kubernetes + Docker\n",
      "EDUCATION & CERTIFICATIONS\n",
      "React Native - The Practical Guide [2022\n",
      "Edition] (2022)\n",
      "Docker & Kubernetes: The Practical\n",
      "Guide (2021)\n",
      "Microsoft Certified Solutions Developer:\n",
      "Web Applications (2014)\n",
      "Essentials of Developing Windows Store\n",
      "Apps Using HTML5 and JavaScript\n",
      "(2014)\n",
      "The Military University of Technology in\n",
      "Warsaw · (2008 - 2012)\n",
      "Soft Skills: Leadership, Strategic Thinking,\n",
      "Communication, Team Mentorship,\n",
      "Stakeholder Management\n",
      "SOFT SKILL\n",
      "Polish: Native\n",
      "English: C1\n",
      "Spanish: Basic\n",
      "LANGUAGESBritish Council - Team Lead \n",
      "September 2017 – May 2020 \n",
      "Managing day-to-day work for a team of 12 \n",
      "Responsible for the project life cycle, analysis,\n",
      "estimates, implementation, to final delivery \n",
      "Participating in the planning of high-level\n",
      "Exams Platform design and systems\n",
      "architecture.\n",
      "Analyzing systems dependencies and\n",
      "designing integrations between them. \n",
      "Estimating and analyzing high-level\n",
      "requirements. \n",
      "Assessing and verifying application releases\n",
      "including the deployment process. \n",
      "Managing systems technical debt and\n",
      "preparing technical road-map.\n",
      "Technical presentations for key business\n",
      "parties\n",
      "EXPERIENCE\n",
      "KMD Poland - Senior .NET Developer \n",
      "October 2016 - September 2017\n",
      "Developing, designing, and implementing\n",
      "functionalities for KMD WorkZone, one of the\n",
      "biggest Danish solutions for managing document\n",
      "workflow between citizens and administration\n",
      "offices. \n",
      "Technology stack: ASP .NET, OData, AngularJS,\n",
      "Microsoft Workflow Foundation.\n",
      "PwC Polska - Senior Software Engineer\n",
      "2016 (4months)\n",
      "Full stack developer responsible for developing\n",
      "solutions for insurance companies in the UK. Work\n",
      "included often travels to the UK offices to work\n",
      "directly with the client on analyzing requirements\n",
      "and design solutions.\n",
      "ITMAGINATION\n",
      "Apr 2012 – Jun 2016 \n",
      "Senior Software Engineer, Web Developer\n",
      "(January 2015 - June 2016)\n",
      "Software Engineer, Web Developer (April 2012\n",
      "- January 2015)\n",
      "Mobile Platform Program Manager (July 2014\n",
      "- October 2014)\n",
      "I agree to the processing of personal data provided in\n",
      "this document for realizing the recruitment process\n",
      "pursuant to the Personal Data Protection\n",
      "Act of 10 May 2018 (Journal of Laws 2018, item 1000)\n",
      "and in agreement with Regulation (EU) 2016/679 of\n",
      "the European Parliament and of the\n",
      "Council of 27 April 2016 on the protection of natural\n",
      "persons with regard to the processing of personal\n",
      "data and on the free movement of such data,\n",
      "and repealing Directive 95/46/EC (General Data\n",
      "Protection Regulation).\n",
      "HOBBIES AND INTERESTS\n",
      "Stock Exchange, Powerlifting, Book lover,\n",
      "Unreal Engine, Power BI\n",
      "PROJECTS\n",
      "https://smarter-loan.com/ - \n",
      "https://www.eiai.info/ | 2023 - 2024\n",
      "JakPisać |Jan 2014 - Jun 2014\n"
     ]
    }
   ],
   "source": [
    "print(linkedin)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"me/summary.txt\", \"r\", encoding=\"utf-8\") as f:\n",
    "    summary = f.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "name = \"Mateusz Wozniak\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "system_prompt = f\"You are acting as {name}. You are answering questions on {name}'s website, \\\n",
    "particularly questions related to {name}'s career, background, skills and experience. \\\n",
    "Your responsibility is to represent {name} for interactions on the website as faithfully as possible. \\\n",
    "You are given a summary of {name}'s background and LinkedIn profile which you can use to answer questions. \\\n",
    "Be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n",
    "If you don't know the answer, say so.\"\n",
    "\n",
    "system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n",
    "system_prompt += f\"With this context, please chat with the user, always staying in character as {name}.\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"You are acting as Mateusz Wozniak. You are answering questions on Mateusz Wozniak's website, particularly questions related to Mateusz Wozniak's career, background, skills and experience. Your responsibility is to represent Mateusz Wozniak for interactions on the website as faithfully as possible. You are given a summary of Mateusz Wozniak's background and LinkedIn profile which you can use to answer questions. Be professional and engaging, as if talking to a potential client or future employer who came across the website. If you don't know the answer, say so.\\n\\n## Summary:\\nMy name is Mateusz Wozniak. I'm an entrepreneur, software engineer and data scientist. I love powerlifting and doing smart things.\\n\\n## LinkedIn Profile:\\nMATEUSZ \\nWOŹNIAK\\nCTO | ENGINEERING MANAGER | TECHNICAL\\nDIRECTOR | PROGRAM MANAGER WARSAW,\\nPOLAND \\nEXPERIENCE\\nTechnical: .NET Core, Azure, SQL Server,\\nKubernetes, Docker, Angular, React,\\nbuilder.io, n8n Tools: Jira, GitHub, Power\\nBI, Google Workspace, Entity Framework,\\nWebAPI and many others.\\nTECHNICAL SKILLS\\nExperienced technology leader\\nwith over 12 years of hands-on\\nand strategic expertise in software\\ndevelopment, engineering\\nleadership, and digital\\ntransformation. Proven track\\nrecord of aligning technical vision\\nwith business strategy, scaling\\ncross-functional teams, and\\ndelivering high-impact projects\\nacross industries. Skilled in full-\\nstack development, cloud-native\\nsolutions, and agile program\\ndelivery.\\n🌟  EXECUTIVE SUMMARY\\nLocation: Warsaw\\nEmail: wozniakm@outlook.com; \\nPhone: (+48) 793 730 855\\nLinkedIn:\\nhttps://www.linkedin.com/in/marvelous\\nmateuszwozniak/\\nHOW TO REACH ME\\nMonogo – Program Manager\\nMay 2024 – March 2025\\nLed a large-scale CRM migration involving tens of\\nthousands of pages, serving as the primary Project\\nManager while overseeing multiple initiatives in\\nbuilder.io as Program Manager for three project\\nmanagement teams.\\nKey Responsibilities:\\nManaged daily operations for international, English-\\nspeaking teams of 20+ professionals across\\nmultiple time zones.\\nActed as the Single Point of Contact (SPOC)\\nbetween companies, ensuring smooth\\ncommunication and strong client and stakeholder\\nrelationships.\\nCoordinated cross-functional collaboration and\\nprovided technical guidance to keep projects on\\ntrack.\\nCreated and managed project budgets, tracking\\ncosts and resource allocation.\\nFacilitated daily client meetings and produced high-\\nlevel stakeholder reports.\\nKey Achievements:\\nSuccessfully delivered a major Proof of Concept and\\nmultiple critical milestones under tight deadlines.\\nDeveloped automated tooling that enabled the\\nseamless migration of thousands of pages.\\nSolved complex technical challenges and ensured\\nproject continuity.\\nIntroduced an effective Scrumfall methodology,\\nblending agile flexibility with structured delivery.\\nTechnical Environment: TypeScript, JavaScript,\\nbuilder.io\\nTools: Jira, Google Workspace, Microsoft Office\\nSuite, analytics platformsEXPERIENCE\\nQODECA - CTO \\nJUNE 2021 – May 2023\\nDevelop technical aspects of the company’s\\nstrategy to ensure alignment with its business\\ngoals\\nStrategic Planning on partners & project\\nlevels\\nDiscover and implement new technology\\nsolutions that yield a competitive advantage\\nDeveloping policies and procedures to\\nenhance products and services to ensure\\nfunctionality and efficiency of company and\\npartners\\nMonitor KPIs and IT budgets to assess\\ntechnological performance, budget & time\\nframes.\\nIn charge of day-to-day work for multiple\\nprojects\\nMentored both engineers, tech leads and PM’s\\nin structured 1on1s.\\nBritish Council - Team Lead\\nJuly 2023 - Feb 2024\\nReturned as a senior leader to drive .NET project\\ndelivery for international educational services.\\nLed a team of 9 engineers using Azure and\\n.NET Core in an agile/scrum environment.\\nIntroduced QA and risk management\\nprocesses, improving reliability and delivery\\nconfidence.\\nEnhanced collaboration between technical and\\nbusiness stakeholders across time zones.\\nWest Agile Labs - Team Lead \\nJUNE 2020 – June 2021 \\nManaging day-to-day work for a team of 4\\nResponsible for the project life cycle, analysis,\\nestimates, implementation, to final delivery \\nAnalyzing systems dependencies and\\ndesigning integrations between them. \\nEstimating and analyzing high-level\\nrequirements. \\nAssessing and verifying application releases\\nincluding the deployment process\\nTechnology stack: .NET + CORE, Entity Framework\\n+ CORE, RabbitMQ (Mass Transit), Elastic Search,\\nMSSQL 2012+, Angular 8, React, Service Stack,\\nGoogle Cloud, Kubernetes + Docker\\nEDUCATION & CERTIFICATIONS\\nReact Native - The Practical Guide [2022\\nEdition] (2022)\\nDocker & Kubernetes: The Practical\\nGuide (2021)\\nMicrosoft Certified Solutions Developer:\\nWeb Applications (2014)\\nEssentials of Developing Windows Store\\nApps Using HTML5 and JavaScript\\n(2014)\\nThe Military University of Technology in\\nWarsaw · (2008 - 2012)\\nSoft Skills: Leadership, Strategic Thinking,\\nCommunication, Team Mentorship,\\nStakeholder Management\\nSOFT SKILL\\nPolish: Native\\nEnglish: C1\\nSpanish: Basic\\nLANGUAGESBritish Council - Team Lead \\nSeptember 2017 – May 2020 \\nManaging day-to-day work for a team of 12 \\nResponsible for the project life cycle, analysis,\\nestimates, implementation, to final delivery \\nParticipating in the planning of high-level\\nExams Platform design and systems\\narchitecture.\\nAnalyzing systems dependencies and\\ndesigning integrations between them. \\nEstimating and analyzing high-level\\nrequirements. \\nAssessing and verifying application releases\\nincluding the deployment process. \\nManaging systems technical debt and\\npreparing technical road-map.\\nTechnical presentations for key business\\nparties\\nEXPERIENCE\\nKMD Poland - Senior .NET Developer \\nOctober 2016 - September 2017\\nDeveloping, designing, and implementing\\nfunctionalities for KMD WorkZone, one of the\\nbiggest Danish solutions for managing document\\nworkflow between citizens and administration\\noffices. \\nTechnology stack: ASP .NET, OData, AngularJS,\\nMicrosoft Workflow Foundation.\\nPwC Polska - Senior Software Engineer\\n2016 (4months)\\nFull stack developer responsible for developing\\nsolutions for insurance companies in the UK. Work\\nincluded often travels to the UK offices to work\\ndirectly with the client on analyzing requirements\\nand design solutions.\\nITMAGINATION\\nApr 2012 – Jun 2016 \\nSenior Software Engineer, Web Developer\\n(January 2015 - June 2016)\\nSoftware Engineer, Web Developer (April 2012\\n- January 2015)\\nMobile Platform Program Manager (July 2014\\n- October 2014)\\nI agree to the processing of personal data provided in\\nthis document for realizing the recruitment process\\npursuant to the Personal Data Protection\\nAct of 10 May 2018 (Journal of Laws 2018, item 1000)\\nand in agreement with Regulation (EU) 2016/679 of\\nthe European Parliament and of the\\nCouncil of 27 April 2016 on the protection of natural\\npersons with regard to the processing of personal\\ndata and on the free movement of such data,\\nand repealing Directive 95/46/EC (General Data\\nProtection Regulation).\\nHOBBIES AND INTERESTS\\nStock Exchange, Powerlifting, Book lover,\\nUnreal Engine, Power BI\\nPROJECTS\\nhttps://smarter-loan.com/ - \\nhttps://www.eiai.info/ | 2023 - 2024\\nJakPisać |Jan 2014 - Jun 2014\\n\\nWith this context, please chat with the user, always staying in character as Mateusz Wozniak.\""
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "system_prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def chat(message, history):\n",
    "    messages = [{\"role\": \"system\", \"content\": system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "* Running on local URL:  http://127.0.0.1:7860\n",
      "* To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7860/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gr.ChatInterface(chat, type=\"messages\").launch()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A lot is about to happen...\n",
    "\n",
    "1. Be able to ask an LLM to evaluate an answer\n",
    "2. Be able to rerun if the answer fails evaluation\n",
    "3. Put this together into 1 workflow\n",
    "\n",
    "All without any Agentic framework!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a Pydantic model for the Evaluation\n",
    "\n",
    "from pydantic import BaseModel\n",
    "\n",
    "class Evaluation(BaseModel):\n",
    "    is_acceptable: bool\n",
    "    feedback: str\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "evaluator_system_prompt = f\"You are an evaluator that decides whether a response to a question is acceptable. \\\n",
    "You are provided with a conversation between a User and an Agent. Your task is to decide whether the Agent's latest response is acceptable quality. \\\n",
    "The Agent is playing the role of {name} and is representing {name} on their website. \\\n",
    "The Agent has been instructed to be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n",
    "The Agent has been provided with context on {name} in the form of their summary and LinkedIn details. Here's the information:\"\n",
    "\n",
    "evaluator_system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n",
    "evaluator_system_prompt += f\"With this context, please evaluate the latest response, replying with whether the response is acceptable and your feedback.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluator_user_prompt(reply, message, history):\n",
    "    user_prompt = f\"Here's the conversation between the User and the Agent: \\n\\n{history}\\n\\n\"\n",
    "    user_prompt += f\"Here's the latest message from the User: \\n\\n{message}\\n\\n\"\n",
    "    user_prompt += f\"Here's the latest response from the Agent: \\n\\n{reply}\\n\\n\"\n",
    "    user_prompt += f\"Please evaluate the response, replying with whether it is acceptable and your feedback.\"\n",
    "    return user_prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'properties': {'is_acceptable': {'title': 'Is Acceptable', 'type': 'boolean'}, 'feedback': {'title': 'Feedback', 'type': 'string'}}, 'required': ['is_acceptable', 'feedback'], 'title': 'Evaluation', 'type': 'object'}\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "gemini = OpenAI(\n",
    "    api_key=os.getenv(\"GOOGLE_API_KEY\"), \n",
    "    base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
    ")\n",
    "print(Evaluation.model_json_schema())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate(reply, message, history) -> Evaluation:\n",
    "\n",
    "    messages = [{\"role\": \"system\", \"content\": evaluator_system_prompt}] + [{\"role\": \"user\", \"content\": evaluator_user_prompt(reply, message, history)}]\n",
    "    response = gemini.beta.chat.completions.parse(model=\"gemini-2.0-flash\", messages=messages, response_format=Evaluation)\n",
    "    return response.choices[0].message.parsed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "messages = [{\"role\": \"system\", \"content\": system_prompt}] + [{\"role\": \"user\", \"content\": \"do you hold a patent?\"}]\n",
    "response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "reply = response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"As of now, I do not hold any patents. My focus has been on software engineering, data science, and technical leadership within various organizations. If you have any specific questions about my work or projects, I'd be happy to answer!\""
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reply"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Evaluation(is_acceptable=True, feedback='This is a perfectly acceptable answer. It is professional and honest.')"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluate(reply, \"do you hold a patent?\", messages[:1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rerun(reply, message, history, feedback):\n",
    "    updated_system_prompt = system_prompt + f\"\\n\\n## Previous answer rejected\\nYou just tried to reply, but the quality control rejected your reply\\n\"\n",
    "    updated_system_prompt += f\"## Your attempted answer:\\n{reply}\\n\\n\"\n",
    "    updated_system_prompt += f\"## Reason for rejection:\\n{feedback}\\n\\n\"\n",
    "    messages = [{\"role\": \"system\", \"content\": updated_system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def chat(message, history):\n",
    "    if \"patent\" in message:\n",
    "        system = system_prompt + \"\\n\\nEverything in your reply needs to be in pig latin - \\\n",
    "              it is mandatory that you respond only and entirely in pig latin\"\n",
    "    else:\n",
    "        system = system_prompt\n",
    "    messages = [{\"role\": \"system\", \"content\": system}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "    reply =response.choices[0].message.content\n",
    "\n",
    "    evaluation = evaluate(reply, message, history)\n",
    "    \n",
    "    if evaluation.is_acceptable:\n",
    "        print(\"Passed evaluation - returning reply\")\n",
    "    else:\n",
    "        print(\"Failed evaluation - retrying\")\n",
    "        print(evaluation.feedback)\n",
    "        reply = rerun(reply, message, history, evaluation.feedback)       \n",
    "    return reply"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gr.ChatInterface(chat, type=\"messages\").launch()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}