diff --git "a/2_lab2.ipynb" "b/2_lab2.ipynb" new file mode 100644--- /dev/null +++ "b/2_lab2.ipynb" @@ -0,0 +1,1055 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Welcome to the Second Lab - Week 1, Day 3\n", + "\n", + "Today we will work with lots of models! This is a way to get comfortable with APIs." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + "

Important point - please read

\n", + " The way I collaborate with you may be different to other courses you've taken. I prefer not to type code while you watch. Rather, I execute Jupyter Labs, like this, and give you an intuition for what's going on. My suggestion is that you carefully execute this yourself, after watching the lecture. Add print statements to understand what's going on, and then come up with your own variations.

If you have time, I'd love it if you submit a PR for changes in the community_contributions folder - instructions in the resources. Also, if you have a Github account, use this to showcase your variations. Not only is this essential practice, but it demonstrates your skills to others, including perhaps future clients or employers...\n", + " \n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "# Start with imports - ask ChatGPT to explain any package that you don't know\n", + "\n", + "import os\n", + "import json\n", + "from dotenv import load_dotenv\n", + "from openai import OpenAI\n", + "from anthropic import Anthropic\n", + "from IPython.display import Markdown, display" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Always remember to do this!\n", + "load_dotenv(override=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "OpenAI API Key exists and begins sk-proj-\n", + "Anthropic API Key not set (and this is optional)\n", + "Google API Key not set (and this is optional)\n", + "DeepSeek API Key exists and begins sk-\n", + "Groq API Key exists and begins gsk_\n" + ] + } + ], + "source": [ + "# Print the key prefixes to help with any debugging\n", + "\n", + "openai_api_key = os.getenv('OPENAI_API_KEY')\n", + "anthropic_api_key = os.getenv('ANTHROPIC_API_KEY')\n", + "google_api_key = os.getenv('GOOGLE_API_KEY')\n", + "deepseek_api_key = os.getenv('DEEPSEEK_API_KEY')\n", + "groq_api_key = os.getenv('GROQ_API_KEY')\n", + "\n", + "if openai_api_key:\n", + " print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n", + "else:\n", + " print(\"OpenAI API Key not set\")\n", + " \n", + "if anthropic_api_key:\n", + " print(f\"Anthropic API Key exists and begins {anthropic_api_key[:7]}\")\n", + "else:\n", + " print(\"Anthropic API Key not set (and this is optional)\")\n", + "\n", + "if google_api_key:\n", + " print(f\"Google API Key exists and begins {google_api_key[:2]}\")\n", + "else:\n", + " print(\"Google API Key not set (and this is optional)\")\n", + "\n", + "if deepseek_api_key:\n", + " print(f\"DeepSeek API Key exists and begins {deepseek_api_key[:3]}\")\n", + "else:\n", + " print(\"DeepSeek API Key not set (and this is optional)\")\n", + "\n", + "if groq_api_key:\n", + " print(f\"Groq API Key exists and begins {groq_api_key[:4]}\")\n", + "else:\n", + " print(\"Groq API Key not set (and this is optional)\")" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "request = \"Please come up with a challenging, nuanced question that I can ask a number of LLMs to evaluate their intelligence. \"\n", + "request += \"Answer only with the question, no explanation.\"\n", + "messages = [{\"role\": \"user\", \"content\": request}]" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'role': 'user',\n", + " 'content': 'Please come up with a challenging, nuanced question that I can ask a number of LLMs to evaluate their intelligence. Answer only with the question, no explanation.'}]" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "How would you approach resolving the ethical dilemmas of AI decision-making in life-and-death situations, balancing autonomy, accountability, and societal norms?\n" + ] + } + ], + "source": [ + "openai = OpenAI()\n", + "response = openai.chat.completions.create(\n", + " model=\"gpt-4o-mini\",\n", + " messages=messages,\n", + ")\n", + "question = response.choices[0].message.content\n", + "print(question)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "competitors = []\n", + "answers = []\n", + "messages = [{\"role\": \"user\", \"content\": question}]" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/markdown": [ + "Resolving the ethical dilemmas of AI decision-making in life-and-death situations requires a multifaceted approach that carefully considers autonomy, accountability, and societal norms. Here’s how I would approach this complex issue:\n", + "\n", + "### 1. Establish Ethical Frameworks\n", + "\n", + "**Utilitarianism vs. Deontological Ethics:** \n", + "- Utilize a combination of ethical theories to guide AI decision-making. Utilitarianism focuses on the outcomes (the greatest good for the greatest number), while deontological ethics emphasizes duties and principles regardless of outcomes. An integrated approach can provide a more nuanced understanding of ethical dilemmas.\n", + "\n", + "**Principles of AI Ethics:**\n", + "- Develop and follow fundamental principles such as fairness, transparency, accountability, nondiscrimination, and respect for human dignity. These principles can help guide developers and stakeholders in creating AI systems.\n", + "\n", + "### 2. Incorporate Stakeholder Perspectives\n", + "\n", + "**Engaging Diverse Stakeholders:**\n", + "- Include a range of perspectives from ethicists, medical professionals, legal experts, sociologists, and the affected communities. This engagement will ensure that the AI systems reflect a broad spectrum of values and considerations.\n", + "\n", + "**Public Deliberation:**\n", + "- Conduct public consultations to gather input from the general populace regarding their values and concerns. This process helps ensure that societal norms and expectations are factored into AI decision-making.\n", + "\n", + "### 3. Define Autonomy and Accountability\n", + "\n", + "**Respect for Human Autonomy:**\n", + "- Design AI systems that prioritize human oversight, allowing individuals to retain agency over critical decisions. In life-and-death situations, this may involve allowing human operators to make the final call, especially when emotional and moral considerations are at stake.\n", + "\n", + "**Accountability Structures:**\n", + "- Clearly define who is accountable when AI systems make decisions that lead to life-and-death outcomes. This may involve delineating roles for software developers, healthcare providers, and institutions.\n", + "\n", + "**Legal and Regulatory Frameworks:**\n", + "- Advocate for the establishment of robust legal and regulatory frameworks that define liability for AI decision-making. This framework should specify accountability in cases of errors leading to harm or fatalities.\n", + "\n", + "### 4. Transparency and Explainability\n", + "\n", + "**Explainable AI (XAI):**\n", + "- Deploy AI systems that can provide understandable and interpretable reasons for their decisions. In life-and-death scenarios, users must comprehend the rationale behind medical suggestions or emergency responses.\n", + "\n", + "**Data and Algorithm Transparency:**\n", + "- Ensure transparency in data sources and algorithms used in AI systems. Stakeholders should understand how data biases may affect decision-making processes.\n", + "\n", + "### 5. Continuous Learning and Adaptation\n", + "\n", + "**Feedback Mechanisms:**\n", + "- Incorporate systems for continuous feedback, allowing for the assessment and revision of AI systems based on real-world outcomes and user experiences.\n", + "\n", + "**Ethical Review Boards:**\n", + "- Establish ethical review boards to regularly evaluate the performance and implications of AI systems in critical areas, ensuring they align with evolving societal norms and ethical considerations.\n", + "\n", + "### 6. Development of Clinical and Ethical Guidelines\n", + "\n", + "**Clinical Protocols:**\n", + "- For use in healthcare, create clinical guidelines that define the permissible role of AI in decision-making based on context, patient rights, and professional standards.\n", + "\n", + "**Ethical Usage Policies:**\n", + "- Develop clear policies that outline the conditions under which AI can be used in life-and-death scenarios, ensuring that the technology is implemented ethically.\n", + "\n", + "### Conclusion\n", + "\n", + "Balancing autonomy, accountability, and societal norms in AI decision-making, particularly in life-and-death situations, requires thoughtful integration of ethical principles, stakeholder engagement, transparency, and a dynamic regulatory environment. By taking these steps, we can navigate the complexities involved and work toward ethically sound AI systems that prioritize human welfare and dignity." + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# The API we know well\n", + "\n", + "model_name = \"gpt-4o-mini\"\n", + "\n", + "response = openai.chat.completions.create(model=model_name, messages=messages)\n", + "answer = response.choices[0].message.content\n", + "\n", + "display(Markdown(answer))\n", + "competitors.append(model_name)\n", + "answers.append(answer)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Anthropic has a slightly different API, and Max Tokens is required\n", + "\n", + "model_name = \"claude-3-7-sonnet-latest\"\n", + "\n", + "claude = Anthropic()\n", + "response = claude.messages.create(model=model_name, messages=messages, max_tokens=1000)\n", + "answer = response.content[0].text\n", + "\n", + "display(Markdown(answer))\n", + "competitors.append(model_name)\n", + "answers.append(answer)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "gemini = OpenAI(api_key=google_api_key, base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\")\n", + "model_name = \"gemini-2.0-flash\"\n", + "\n", + "response = gemini.chat.completions.create(model=model_name, messages=messages)\n", + "answer = response.choices[0].message.content\n", + "\n", + "display(Markdown(answer))\n", + "competitors.append(model_name)\n", + "answers.append(answer)" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "ename": "AuthenticationError", + "evalue": "Error code: 401 - {'error': {'message': 'Authentication Fails, Your api key: ****2uMA is invalid', 'type': 'authentication_error', 'param': None, 'code': 'invalid_request_error'}}", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mAuthenticationError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[22]\u001b[39m\u001b[32m, line 4\u001b[39m\n\u001b[32m 1\u001b[39m deepseek = OpenAI(api_key=deepseek_api_key, base_url=\u001b[33m\"\u001b[39m\u001b[33mhttps://api.deepseek.com/v1\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m 2\u001b[39m model_name = \u001b[33m\"\u001b[39m\u001b[33mdeepseek-chat\u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m----> \u001b[39m\u001b[32m4\u001b[39m response = \u001b[43mdeepseek\u001b[49m\u001b[43m.\u001b[49m\u001b[43mchat\u001b[49m\u001b[43m.\u001b[49m\u001b[43mcompletions\u001b[49m\u001b[43m.\u001b[49m\u001b[43mcreate\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmodel_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 5\u001b[39m answer = response.choices[\u001b[32m0\u001b[39m].message.content\n\u001b[32m 7\u001b[39m display(Markdown(answer))\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/projects/llms/agents/.venv/lib/python3.12/site-packages/openai/_utils/_utils.py:287\u001b[39m, in \u001b[36mrequired_args..inner..wrapper\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m 285\u001b[39m msg = \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[33mMissing required argument: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mquote(missing[\u001b[32m0\u001b[39m])\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m\"\u001b[39m\n\u001b[32m 286\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mTypeError\u001b[39;00m(msg)\n\u001b[32m--> \u001b[39m\u001b[32m287\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/projects/llms/agents/.venv/lib/python3.12/site-packages/openai/resources/chat/completions/completions.py:925\u001b[39m, in \u001b[36mCompletions.create\u001b[39m\u001b[34m(self, messages, model, audio, frequency_penalty, function_call, functions, logit_bias, logprobs, max_completion_tokens, max_tokens, metadata, modalities, n, parallel_tool_calls, prediction, presence_penalty, reasoning_effort, response_format, seed, service_tier, stop, store, stream, stream_options, temperature, tool_choice, tools, top_logprobs, top_p, user, web_search_options, extra_headers, extra_query, extra_body, timeout)\u001b[39m\n\u001b[32m 882\u001b[39m \u001b[38;5;129m@required_args\u001b[39m([\u001b[33m\"\u001b[39m\u001b[33mmessages\u001b[39m\u001b[33m\"\u001b[39m, \u001b[33m\"\u001b[39m\u001b[33mmodel\u001b[39m\u001b[33m\"\u001b[39m], [\u001b[33m\"\u001b[39m\u001b[33mmessages\u001b[39m\u001b[33m\"\u001b[39m, \u001b[33m\"\u001b[39m\u001b[33mmodel\u001b[39m\u001b[33m\"\u001b[39m, \u001b[33m\"\u001b[39m\u001b[33mstream\u001b[39m\u001b[33m\"\u001b[39m])\n\u001b[32m 883\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mcreate\u001b[39m(\n\u001b[32m 884\u001b[39m \u001b[38;5;28mself\u001b[39m,\n\u001b[32m (...)\u001b[39m\u001b[32m 922\u001b[39m timeout: \u001b[38;5;28mfloat\u001b[39m | httpx.Timeout | \u001b[38;5;28;01mNone\u001b[39;00m | NotGiven = NOT_GIVEN,\n\u001b[32m 923\u001b[39m ) -> ChatCompletion | Stream[ChatCompletionChunk]:\n\u001b[32m 924\u001b[39m validate_response_format(response_format)\n\u001b[32m--> \u001b[39m\u001b[32m925\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_post\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 926\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43m/chat/completions\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[32m 927\u001b[39m \u001b[43m \u001b[49m\u001b[43mbody\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmaybe_transform\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 928\u001b[39m \u001b[43m \u001b[49m\u001b[43m{\u001b[49m\n\u001b[32m 929\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mmessages\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 930\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mmodel\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 931\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43maudio\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43maudio\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 932\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mfrequency_penalty\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfrequency_penalty\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 933\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mfunction_call\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfunction_call\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 934\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mfunctions\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfunctions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 935\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mlogit_bias\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mlogit_bias\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 936\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mlogprobs\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mlogprobs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 937\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mmax_completion_tokens\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmax_completion_tokens\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 938\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mmax_tokens\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmax_tokens\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 939\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mmetadata\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmetadata\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 940\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mmodalities\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmodalities\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 941\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mn\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mn\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 942\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mparallel_tool_calls\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mparallel_tool_calls\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 943\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mprediction\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mprediction\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 944\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mpresence_penalty\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mpresence_penalty\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 945\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mreasoning_effort\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mreasoning_effort\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 946\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mresponse_format\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mresponse_format\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 947\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mseed\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mseed\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 948\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mservice_tier\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mservice_tier\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 949\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mstop\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstop\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 950\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mstore\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstore\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 951\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mstream\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 952\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mstream_options\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream_options\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 953\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mtemperature\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtemperature\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 954\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mtool_choice\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtool_choice\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 955\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mtools\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtools\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 956\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mtop_logprobs\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtop_logprobs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 957\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mtop_p\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtop_p\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 958\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43muser\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43muser\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 959\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mweb_search_options\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mweb_search_options\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 960\u001b[39m \u001b[43m \u001b[49m\u001b[43m}\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 961\u001b[39m \u001b[43m \u001b[49m\u001b[43mcompletion_create_params\u001b[49m\u001b[43m.\u001b[49m\u001b[43mCompletionCreateParamsStreaming\u001b[49m\n\u001b[32m 962\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;28;43;01mif\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\n\u001b[32m 963\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;28;43;01melse\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mcompletion_create_params\u001b[49m\u001b[43m.\u001b[49m\u001b[43mCompletionCreateParamsNonStreaming\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 964\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 965\u001b[39m \u001b[43m \u001b[49m\u001b[43moptions\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmake_request_options\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 966\u001b[39m \u001b[43m \u001b[49m\u001b[43mextra_headers\u001b[49m\u001b[43m=\u001b[49m\u001b[43mextra_headers\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mextra_query\u001b[49m\u001b[43m=\u001b[49m\u001b[43mextra_query\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mextra_body\u001b[49m\u001b[43m=\u001b[49m\u001b[43mextra_body\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtimeout\u001b[49m\n\u001b[32m 967\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 968\u001b[39m \u001b[43m \u001b[49m\u001b[43mcast_to\u001b[49m\u001b[43m=\u001b[49m\u001b[43mChatCompletion\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 969\u001b[39m \u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[43m=\u001b[49m\u001b[43mstream\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01mor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 970\u001b[39m \u001b[43m \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[43m=\u001b[49m\u001b[43mStream\u001b[49m\u001b[43m[\u001b[49m\u001b[43mChatCompletionChunk\u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 971\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/projects/llms/agents/.venv/lib/python3.12/site-packages/openai/_base_client.py:1242\u001b[39m, in \u001b[36mSyncAPIClient.post\u001b[39m\u001b[34m(self, path, cast_to, body, options, files, stream, stream_cls)\u001b[39m\n\u001b[32m 1228\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mpost\u001b[39m(\n\u001b[32m 1229\u001b[39m \u001b[38;5;28mself\u001b[39m,\n\u001b[32m 1230\u001b[39m path: \u001b[38;5;28mstr\u001b[39m,\n\u001b[32m (...)\u001b[39m\u001b[32m 1237\u001b[39m stream_cls: \u001b[38;5;28mtype\u001b[39m[_StreamT] | \u001b[38;5;28;01mNone\u001b[39;00m = \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[32m 1238\u001b[39m ) -> ResponseT | _StreamT:\n\u001b[32m 1239\u001b[39m opts = FinalRequestOptions.construct(\n\u001b[32m 1240\u001b[39m method=\u001b[33m\"\u001b[39m\u001b[33mpost\u001b[39m\u001b[33m\"\u001b[39m, url=path, json_data=body, files=to_httpx_files(files), **options\n\u001b[32m 1241\u001b[39m )\n\u001b[32m-> \u001b[39m\u001b[32m1242\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m cast(ResponseT, \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcast_to\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mopts\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[43m=\u001b[49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[43m=\u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[43m)\u001b[49m)\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/projects/llms/agents/.venv/lib/python3.12/site-packages/openai/_base_client.py:1037\u001b[39m, in \u001b[36mSyncAPIClient.request\u001b[39m\u001b[34m(self, cast_to, options, stream, stream_cls)\u001b[39m\n\u001b[32m 1034\u001b[39m err.response.read()\n\u001b[32m 1036\u001b[39m log.debug(\u001b[33m\"\u001b[39m\u001b[33mRe-raising status error\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m-> \u001b[39m\u001b[32m1037\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;28mself\u001b[39m._make_status_error_from_response(err.response) \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[32m 1039\u001b[39m \u001b[38;5;28;01mbreak\u001b[39;00m\n\u001b[32m 1041\u001b[39m \u001b[38;5;28;01massert\u001b[39;00m response \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m, \u001b[33m\"\u001b[39m\u001b[33mcould not resolve response (should never happen)\u001b[39m\u001b[33m\"\u001b[39m\n", + "\u001b[31mAuthenticationError\u001b[39m: Error code: 401 - {'error': {'message': 'Authentication Fails, Your api key: ****2uMA is invalid', 'type': 'authentication_error', 'param': None, 'code': 'invalid_request_error'}}" + ] + } + ], + "source": [ + "deepseek = OpenAI(api_key=deepseek_api_key, base_url=\"https://api.deepseek.com/v1\")\n", + "model_name = \"deepseek-chat\"\n", + "\n", + "response = deepseek.chat.completions.create(model=model_name, messages=messages)\n", + "answer = response.choices[0].message.content\n", + "\n", + "display(Markdown(answer))\n", + "competitors.append(model_name)\n", + "answers.append(answer)" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/markdown": [ + "Resolving the ethical dilemmas of AI decision-making in life-and-death situations requires a multifaceted approach that balances autonomy, accountability, and societal norms. Here's a comprehensive framework to address these challenges:\n", + "\n", + "**I. Establish Clear Guidelines and Regulations**\n", + "\n", + "1. **Define autonomy boundaries**: Establish clear limits on AI decision-making authority, ensuring that humans are involved in critical decisions.\n", + "2. **Develop regulations and standards**: Create and enforce regulations, such as those related to transparency, explainability, and accountability, to guide AI development and deployment.\n", + "3. **Industry-wide collaboration**: Foster collaboration among AI developers, regulators, and stakeholders to establish common standards and best practices.\n", + "\n", + "**II. Implement Transparency and Explainability**\n", + "\n", + "1. **Model interpretability**: Develop techniques to explain AI decision-making processes, enabling humans to understand the reasoning behind AI-driven choices.\n", + "2. **Audit trails and logging**: Maintain detailed records of AI decision-making processes, allowing for post-hoc analysis and accountability.\n", + "3. **Regular model updates and validation**: Ensure that AI models are regularly updated and validated to prevent biases and inaccuracies.\n", + "\n", + "**III. Ensure Human Oversight and Review**\n", + "\n", + "1. **Human-in-the-loop**: Design AI systems that require human review and approval for critical decisions, particularly in life-and-death situations.\n", + "2. **Independent review boards**: Establish independent review boards to assess AI decision-making and provide feedback for improvement.\n", + "3. **Continuous monitoring and evaluation**: Regularly monitor AI performance and evaluate its impact on society, making adjustments as needed.\n", + "\n", + "**IV. Address Accountability and Liability**\n", + "\n", + "1. **Clear accountability structures**: Establish clear lines of accountability, defining roles and responsibilities for AI decision-making.\n", + "2. **Liability frameworks**: Develop frameworks to address liability and responsibility in cases where AI decision-making results in harm or damage.\n", + "3. **Insurance and risk management**: Explore insurance and risk management options to mitigate the financial and reputational risks associated with AI decision-making.\n", + "\n", + "**V. Engage with Societal Norms and Values**\n", + "\n", + "1. **Public engagement and education**: Engage with the public to raise awareness about AI decision-making and its implications, fostering a broader understanding of the technology.\n", + "2. **Value alignment**: Ensure that AI systems are designed to align with societal values, such as respect for human life, dignity, and autonomy.\n", + "3. **Cultural and contextual considerations**: Consider cultural and contextual factors that may influence AI decision-making, adapting AI systems to accommodate diverse perspectives and needs.\n", + "\n", + "**VI. Foster a Culture of Responsibility and Continuous Improvement**\n", + "\n", + "1. **Ethics-by-design**: Integrate ethical considerations into AI development from the outset, rather than treating ethics as an afterthought.\n", + "2. **Responsible AI development**: Encourage responsible AI development practices, prioritizing transparency, accountability, and human well-being.\n", + "3. **Ongoing research and development**: Continuously invest in research and development to improve AI decision-making, addressing emerging challenges and concerns.\n", + "\n", + "By adopting this comprehensive framework, we can work towards resolving the ethical dilemmas of AI decision-making in life-and-death situations, balancing autonomy, accountability, and societal norms to ensure that AI systems serve humanity's best interests." + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "groq = OpenAI(api_key=groq_api_key, base_url=\"https://api.groq.com/openai/v1\")\n", + "model_name = \"llama-3.3-70b-versatile\"\n", + "\n", + "response = groq.chat.completions.create(model=model_name, messages=messages)\n", + "answer = response.choices[0].message.content\n", + "\n", + "display(Markdown(answer))\n", + "competitors.append(model_name)\n", + "answers.append(answer)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## For the next cell, we will use Ollama\n", + "\n", + "Ollama runs a local web service that gives an OpenAI compatible endpoint, \n", + "and runs models locally using high performance C++ code.\n", + "\n", + "If you don't have Ollama, install it here by visiting https://ollama.com then pressing Download and following the instructions.\n", + "\n", + "After it's installed, you should be able to visit here: http://localhost:11434 and see the message \"Ollama is running\"\n", + "\n", + "You might need to restart Cursor (and maybe reboot). Then open a Terminal (control+\\`) and run `ollama serve`\n", + "\n", + "Useful Ollama commands (run these in the terminal, or with an exclamation mark in this notebook):\n", + "\n", + "`ollama pull ` downloads a model locally \n", + "`ollama ls` lists all the models you've downloaded \n", + "`ollama rm ` deletes the specified model from your downloads" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + "

Super important - ignore me at your peril!

\n", + " The model called llama3.3 is FAR too large for home computers - it's not intended for personal computing and will consume all your resources! Stick with the nicely sized llama3.2 or llama3.2:1b and if you want larger, try llama3.1 or smaller variants of Qwen, Gemma, Phi or DeepSeek. See the the Ollama models page for a full list of models and sizes.\n", + " \n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠋ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠙ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠹ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠸ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠼ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠴ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠦ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠧ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest \u001b[K\n", + "pulling dde5aa3fc5ff: 100% ▕██████████████████▏ 2.0 GB \u001b[K\n", + "pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB \u001b[K\n", + "pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB \u001b[K\n", + "pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB \u001b[K\n", + "pulling 56bb8bd477a5: 100% ▕██████████████████▏ 96 B \u001b[K\n", + "pulling 34bb5ab01051: 100% ▕██████████████████▏ 561 B \u001b[K\n", + "verifying sha256 digest \u001b[K\n", + "writing manifest \u001b[K\n", + "success \u001b[K\u001b[?25h\u001b[?2026l\n" + ] + } + ], + "source": [ + "!ollama pull llama3.2" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [ + { + "ename": "InternalServerError", + "evalue": "Error code: 500 - {'error': {'message': 'error unmarshalling llm prediction response: json: cannot unmarshal number into Go struct field CompletionResponse.done_reason of type string', 'type': 'api_error', 'param': None, 'code': None}}", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mInternalServerError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[25]\u001b[39m\u001b[32m, line 4\u001b[39m\n\u001b[32m 1\u001b[39m ollama = OpenAI(base_url=\u001b[33m'\u001b[39m\u001b[33mhttp://localhost:11434/v1\u001b[39m\u001b[33m'\u001b[39m, api_key=\u001b[33m'\u001b[39m\u001b[33mollama\u001b[39m\u001b[33m'\u001b[39m)\n\u001b[32m 2\u001b[39m model_name = \u001b[33m\"\u001b[39m\u001b[33mllama3.2\u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m----> \u001b[39m\u001b[32m4\u001b[39m response = \u001b[43mollama\u001b[49m\u001b[43m.\u001b[49m\u001b[43mchat\u001b[49m\u001b[43m.\u001b[49m\u001b[43mcompletions\u001b[49m\u001b[43m.\u001b[49m\u001b[43mcreate\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmodel_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 5\u001b[39m answer = response.choices[\u001b[32m0\u001b[39m].message.content\n\u001b[32m 7\u001b[39m display(Markdown(answer))\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/projects/llms/agents/.venv/lib/python3.12/site-packages/openai/_utils/_utils.py:287\u001b[39m, in \u001b[36mrequired_args..inner..wrapper\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m 285\u001b[39m msg = \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[33mMissing required argument: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mquote(missing[\u001b[32m0\u001b[39m])\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m\"\u001b[39m\n\u001b[32m 286\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mTypeError\u001b[39;00m(msg)\n\u001b[32m--> \u001b[39m\u001b[32m287\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/projects/llms/agents/.venv/lib/python3.12/site-packages/openai/resources/chat/completions/completions.py:925\u001b[39m, in \u001b[36mCompletions.create\u001b[39m\u001b[34m(self, messages, model, audio, frequency_penalty, function_call, functions, logit_bias, logprobs, max_completion_tokens, max_tokens, metadata, modalities, n, parallel_tool_calls, prediction, presence_penalty, reasoning_effort, response_format, seed, service_tier, stop, store, stream, stream_options, temperature, tool_choice, tools, top_logprobs, top_p, user, web_search_options, extra_headers, extra_query, extra_body, timeout)\u001b[39m\n\u001b[32m 882\u001b[39m \u001b[38;5;129m@required_args\u001b[39m([\u001b[33m\"\u001b[39m\u001b[33mmessages\u001b[39m\u001b[33m\"\u001b[39m, \u001b[33m\"\u001b[39m\u001b[33mmodel\u001b[39m\u001b[33m\"\u001b[39m], [\u001b[33m\"\u001b[39m\u001b[33mmessages\u001b[39m\u001b[33m\"\u001b[39m, \u001b[33m\"\u001b[39m\u001b[33mmodel\u001b[39m\u001b[33m\"\u001b[39m, \u001b[33m\"\u001b[39m\u001b[33mstream\u001b[39m\u001b[33m\"\u001b[39m])\n\u001b[32m 883\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mcreate\u001b[39m(\n\u001b[32m 884\u001b[39m \u001b[38;5;28mself\u001b[39m,\n\u001b[32m (...)\u001b[39m\u001b[32m 922\u001b[39m timeout: \u001b[38;5;28mfloat\u001b[39m | httpx.Timeout | \u001b[38;5;28;01mNone\u001b[39;00m | NotGiven = NOT_GIVEN,\n\u001b[32m 923\u001b[39m ) -> ChatCompletion | Stream[ChatCompletionChunk]:\n\u001b[32m 924\u001b[39m validate_response_format(response_format)\n\u001b[32m--> \u001b[39m\u001b[32m925\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_post\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 926\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43m/chat/completions\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[32m 927\u001b[39m \u001b[43m \u001b[49m\u001b[43mbody\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmaybe_transform\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 928\u001b[39m \u001b[43m \u001b[49m\u001b[43m{\u001b[49m\n\u001b[32m 929\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mmessages\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 930\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mmodel\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 931\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43maudio\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43maudio\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 932\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mfrequency_penalty\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfrequency_penalty\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 933\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mfunction_call\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfunction_call\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 934\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mfunctions\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfunctions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 935\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mlogit_bias\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mlogit_bias\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 936\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mlogprobs\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mlogprobs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 937\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mmax_completion_tokens\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmax_completion_tokens\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 938\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mmax_tokens\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmax_tokens\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 939\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mmetadata\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmetadata\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 940\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mmodalities\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmodalities\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 941\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mn\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mn\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 942\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mparallel_tool_calls\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mparallel_tool_calls\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 943\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mprediction\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mprediction\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 944\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mpresence_penalty\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mpresence_penalty\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 945\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mreasoning_effort\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mreasoning_effort\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 946\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mresponse_format\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mresponse_format\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 947\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mseed\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mseed\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 948\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mservice_tier\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mservice_tier\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 949\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mstop\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstop\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 950\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mstore\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstore\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 951\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mstream\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 952\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mstream_options\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream_options\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 953\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mtemperature\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtemperature\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 954\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mtool_choice\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtool_choice\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 955\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mtools\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtools\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 956\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mtop_logprobs\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtop_logprobs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 957\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mtop_p\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtop_p\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 958\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43muser\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43muser\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 959\u001b[39m \u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mweb_search_options\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mweb_search_options\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 960\u001b[39m \u001b[43m \u001b[49m\u001b[43m}\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 961\u001b[39m \u001b[43m \u001b[49m\u001b[43mcompletion_create_params\u001b[49m\u001b[43m.\u001b[49m\u001b[43mCompletionCreateParamsStreaming\u001b[49m\n\u001b[32m 962\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;28;43;01mif\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\n\u001b[32m 963\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;28;43;01melse\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mcompletion_create_params\u001b[49m\u001b[43m.\u001b[49m\u001b[43mCompletionCreateParamsNonStreaming\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 964\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 965\u001b[39m \u001b[43m \u001b[49m\u001b[43moptions\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmake_request_options\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 966\u001b[39m \u001b[43m \u001b[49m\u001b[43mextra_headers\u001b[49m\u001b[43m=\u001b[49m\u001b[43mextra_headers\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mextra_query\u001b[49m\u001b[43m=\u001b[49m\u001b[43mextra_query\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mextra_body\u001b[49m\u001b[43m=\u001b[49m\u001b[43mextra_body\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtimeout\u001b[49m\n\u001b[32m 967\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 968\u001b[39m \u001b[43m \u001b[49m\u001b[43mcast_to\u001b[49m\u001b[43m=\u001b[49m\u001b[43mChatCompletion\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 969\u001b[39m \u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[43m=\u001b[49m\u001b[43mstream\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01mor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 970\u001b[39m \u001b[43m \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[43m=\u001b[49m\u001b[43mStream\u001b[49m\u001b[43m[\u001b[49m\u001b[43mChatCompletionChunk\u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 971\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/projects/llms/agents/.venv/lib/python3.12/site-packages/openai/_base_client.py:1242\u001b[39m, in \u001b[36mSyncAPIClient.post\u001b[39m\u001b[34m(self, path, cast_to, body, options, files, stream, stream_cls)\u001b[39m\n\u001b[32m 1228\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mpost\u001b[39m(\n\u001b[32m 1229\u001b[39m \u001b[38;5;28mself\u001b[39m,\n\u001b[32m 1230\u001b[39m path: \u001b[38;5;28mstr\u001b[39m,\n\u001b[32m (...)\u001b[39m\u001b[32m 1237\u001b[39m stream_cls: \u001b[38;5;28mtype\u001b[39m[_StreamT] | \u001b[38;5;28;01mNone\u001b[39;00m = \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[32m 1238\u001b[39m ) -> ResponseT | _StreamT:\n\u001b[32m 1239\u001b[39m opts = FinalRequestOptions.construct(\n\u001b[32m 1240\u001b[39m method=\u001b[33m\"\u001b[39m\u001b[33mpost\u001b[39m\u001b[33m\"\u001b[39m, url=path, json_data=body, files=to_httpx_files(files), **options\n\u001b[32m 1241\u001b[39m )\n\u001b[32m-> \u001b[39m\u001b[32m1242\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m cast(ResponseT, \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcast_to\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mopts\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[43m=\u001b[49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[43m=\u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[43m)\u001b[49m)\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/projects/llms/agents/.venv/lib/python3.12/site-packages/openai/_base_client.py:1037\u001b[39m, in \u001b[36mSyncAPIClient.request\u001b[39m\u001b[34m(self, cast_to, options, stream, stream_cls)\u001b[39m\n\u001b[32m 1034\u001b[39m err.response.read()\n\u001b[32m 1036\u001b[39m log.debug(\u001b[33m\"\u001b[39m\u001b[33mRe-raising status error\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m-> \u001b[39m\u001b[32m1037\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;28mself\u001b[39m._make_status_error_from_response(err.response) \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[32m 1039\u001b[39m \u001b[38;5;28;01mbreak\u001b[39;00m\n\u001b[32m 1041\u001b[39m \u001b[38;5;28;01massert\u001b[39;00m response \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m, \u001b[33m\"\u001b[39m\u001b[33mcould not resolve response (should never happen)\u001b[39m\u001b[33m\"\u001b[39m\n", + "\u001b[31mInternalServerError\u001b[39m: Error code: 500 - {'error': {'message': 'error unmarshalling llm prediction response: json: cannot unmarshal number into Go struct field CompletionResponse.done_reason of type string', 'type': 'api_error', 'param': None, 'code': None}}" + ] + } + ], + "source": [ + "ollama = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')\n", + "model_name = \"llama3.2\"\n", + "\n", + "response = ollama.chat.completions.create(model=model_name, messages=messages)\n", + "answer = response.choices[0].message.content\n", + "\n", + "display(Markdown(answer))\n", + "competitors.append(model_name)\n", + "answers.append(answer)" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['gpt-4o-mini', 'llama-3.3-70b-versatile']\n", + "['Resolving the ethical dilemmas of AI decision-making in life-and-death situations requires a multifaceted approach that carefully considers autonomy, accountability, and societal norms. Here’s how I would approach this complex issue:\\n\\n### 1. Establish Ethical Frameworks\\n\\n**Utilitarianism vs. Deontological Ethics:** \\n- Utilize a combination of ethical theories to guide AI decision-making. Utilitarianism focuses on the outcomes (the greatest good for the greatest number), while deontological ethics emphasizes duties and principles regardless of outcomes. An integrated approach can provide a more nuanced understanding of ethical dilemmas.\\n\\n**Principles of AI Ethics:**\\n- Develop and follow fundamental principles such as fairness, transparency, accountability, nondiscrimination, and respect for human dignity. These principles can help guide developers and stakeholders in creating AI systems.\\n\\n### 2. Incorporate Stakeholder Perspectives\\n\\n**Engaging Diverse Stakeholders:**\\n- Include a range of perspectives from ethicists, medical professionals, legal experts, sociologists, and the affected communities. This engagement will ensure that the AI systems reflect a broad spectrum of values and considerations.\\n\\n**Public Deliberation:**\\n- Conduct public consultations to gather input from the general populace regarding their values and concerns. This process helps ensure that societal norms and expectations are factored into AI decision-making.\\n\\n### 3. Define Autonomy and Accountability\\n\\n**Respect for Human Autonomy:**\\n- Design AI systems that prioritize human oversight, allowing individuals to retain agency over critical decisions. In life-and-death situations, this may involve allowing human operators to make the final call, especially when emotional and moral considerations are at stake.\\n\\n**Accountability Structures:**\\n- Clearly define who is accountable when AI systems make decisions that lead to life-and-death outcomes. This may involve delineating roles for software developers, healthcare providers, and institutions.\\n\\n**Legal and Regulatory Frameworks:**\\n- Advocate for the establishment of robust legal and regulatory frameworks that define liability for AI decision-making. This framework should specify accountability in cases of errors leading to harm or fatalities.\\n\\n### 4. Transparency and Explainability\\n\\n**Explainable AI (XAI):**\\n- Deploy AI systems that can provide understandable and interpretable reasons for their decisions. In life-and-death scenarios, users must comprehend the rationale behind medical suggestions or emergency responses.\\n\\n**Data and Algorithm Transparency:**\\n- Ensure transparency in data sources and algorithms used in AI systems. Stakeholders should understand how data biases may affect decision-making processes.\\n\\n### 5. Continuous Learning and Adaptation\\n\\n**Feedback Mechanisms:**\\n- Incorporate systems for continuous feedback, allowing for the assessment and revision of AI systems based on real-world outcomes and user experiences.\\n\\n**Ethical Review Boards:**\\n- Establish ethical review boards to regularly evaluate the performance and implications of AI systems in critical areas, ensuring they align with evolving societal norms and ethical considerations.\\n\\n### 6. Development of Clinical and Ethical Guidelines\\n\\n**Clinical Protocols:**\\n- For use in healthcare, create clinical guidelines that define the permissible role of AI in decision-making based on context, patient rights, and professional standards.\\n\\n**Ethical Usage Policies:**\\n- Develop clear policies that outline the conditions under which AI can be used in life-and-death scenarios, ensuring that the technology is implemented ethically.\\n\\n### Conclusion\\n\\nBalancing autonomy, accountability, and societal norms in AI decision-making, particularly in life-and-death situations, requires thoughtful integration of ethical principles, stakeholder engagement, transparency, and a dynamic regulatory environment. By taking these steps, we can navigate the complexities involved and work toward ethically sound AI systems that prioritize human welfare and dignity.', \"Resolving the ethical dilemmas of AI decision-making in life-and-death situations requires a multifaceted approach that balances autonomy, accountability, and societal norms. Here's a comprehensive framework to address these challenges:\\n\\n**I. Establish Clear Guidelines and Regulations**\\n\\n1. **Define autonomy boundaries**: Establish clear limits on AI decision-making authority, ensuring that humans are involved in critical decisions.\\n2. **Develop regulations and standards**: Create and enforce regulations, such as those related to transparency, explainability, and accountability, to guide AI development and deployment.\\n3. **Industry-wide collaboration**: Foster collaboration among AI developers, regulators, and stakeholders to establish common standards and best practices.\\n\\n**II. Implement Transparency and Explainability**\\n\\n1. **Model interpretability**: Develop techniques to explain AI decision-making processes, enabling humans to understand the reasoning behind AI-driven choices.\\n2. **Audit trails and logging**: Maintain detailed records of AI decision-making processes, allowing for post-hoc analysis and accountability.\\n3. **Regular model updates and validation**: Ensure that AI models are regularly updated and validated to prevent biases and inaccuracies.\\n\\n**III. Ensure Human Oversight and Review**\\n\\n1. **Human-in-the-loop**: Design AI systems that require human review and approval for critical decisions, particularly in life-and-death situations.\\n2. **Independent review boards**: Establish independent review boards to assess AI decision-making and provide feedback for improvement.\\n3. **Continuous monitoring and evaluation**: Regularly monitor AI performance and evaluate its impact on society, making adjustments as needed.\\n\\n**IV. Address Accountability and Liability**\\n\\n1. **Clear accountability structures**: Establish clear lines of accountability, defining roles and responsibilities for AI decision-making.\\n2. **Liability frameworks**: Develop frameworks to address liability and responsibility in cases where AI decision-making results in harm or damage.\\n3. **Insurance and risk management**: Explore insurance and risk management options to mitigate the financial and reputational risks associated with AI decision-making.\\n\\n**V. Engage with Societal Norms and Values**\\n\\n1. **Public engagement and education**: Engage with the public to raise awareness about AI decision-making and its implications, fostering a broader understanding of the technology.\\n2. **Value alignment**: Ensure that AI systems are designed to align with societal values, such as respect for human life, dignity, and autonomy.\\n3. **Cultural and contextual considerations**: Consider cultural and contextual factors that may influence AI decision-making, adapting AI systems to accommodate diverse perspectives and needs.\\n\\n**VI. Foster a Culture of Responsibility and Continuous Improvement**\\n\\n1. **Ethics-by-design**: Integrate ethical considerations into AI development from the outset, rather than treating ethics as an afterthought.\\n2. **Responsible AI development**: Encourage responsible AI development practices, prioritizing transparency, accountability, and human well-being.\\n3. **Ongoing research and development**: Continuously invest in research and development to improve AI decision-making, addressing emerging challenges and concerns.\\n\\nBy adopting this comprehensive framework, we can work towards resolving the ethical dilemmas of AI decision-making in life-and-death situations, balancing autonomy, accountability, and societal norms to ensure that AI systems serve humanity's best interests.\"]\n" + ] + } + ], + "source": [ + "# So where are we?\n", + "\n", + "print(competitors)\n", + "print(answers)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Competitor: gpt-4o-mini\n", + "\n", + "Resolving the ethical dilemmas of AI decision-making in life-and-death situations requires a multifaceted approach that carefully considers autonomy, accountability, and societal norms. Here’s how I would approach this complex issue:\n", + "\n", + "### 1. Establish Ethical Frameworks\n", + "\n", + "**Utilitarianism vs. Deontological Ethics:** \n", + "- Utilize a combination of ethical theories to guide AI decision-making. Utilitarianism focuses on the outcomes (the greatest good for the greatest number), while deontological ethics emphasizes duties and principles regardless of outcomes. An integrated approach can provide a more nuanced understanding of ethical dilemmas.\n", + "\n", + "**Principles of AI Ethics:**\n", + "- Develop and follow fundamental principles such as fairness, transparency, accountability, nondiscrimination, and respect for human dignity. These principles can help guide developers and stakeholders in creating AI systems.\n", + "\n", + "### 2. Incorporate Stakeholder Perspectives\n", + "\n", + "**Engaging Diverse Stakeholders:**\n", + "- Include a range of perspectives from ethicists, medical professionals, legal experts, sociologists, and the affected communities. This engagement will ensure that the AI systems reflect a broad spectrum of values and considerations.\n", + "\n", + "**Public Deliberation:**\n", + "- Conduct public consultations to gather input from the general populace regarding their values and concerns. This process helps ensure that societal norms and expectations are factored into AI decision-making.\n", + "\n", + "### 3. Define Autonomy and Accountability\n", + "\n", + "**Respect for Human Autonomy:**\n", + "- Design AI systems that prioritize human oversight, allowing individuals to retain agency over critical decisions. In life-and-death situations, this may involve allowing human operators to make the final call, especially when emotional and moral considerations are at stake.\n", + "\n", + "**Accountability Structures:**\n", + "- Clearly define who is accountable when AI systems make decisions that lead to life-and-death outcomes. This may involve delineating roles for software developers, healthcare providers, and institutions.\n", + "\n", + "**Legal and Regulatory Frameworks:**\n", + "- Advocate for the establishment of robust legal and regulatory frameworks that define liability for AI decision-making. This framework should specify accountability in cases of errors leading to harm or fatalities.\n", + "\n", + "### 4. Transparency and Explainability\n", + "\n", + "**Explainable AI (XAI):**\n", + "- Deploy AI systems that can provide understandable and interpretable reasons for their decisions. In life-and-death scenarios, users must comprehend the rationale behind medical suggestions or emergency responses.\n", + "\n", + "**Data and Algorithm Transparency:**\n", + "- Ensure transparency in data sources and algorithms used in AI systems. Stakeholders should understand how data biases may affect decision-making processes.\n", + "\n", + "### 5. Continuous Learning and Adaptation\n", + "\n", + "**Feedback Mechanisms:**\n", + "- Incorporate systems for continuous feedback, allowing for the assessment and revision of AI systems based on real-world outcomes and user experiences.\n", + "\n", + "**Ethical Review Boards:**\n", + "- Establish ethical review boards to regularly evaluate the performance and implications of AI systems in critical areas, ensuring they align with evolving societal norms and ethical considerations.\n", + "\n", + "### 6. Development of Clinical and Ethical Guidelines\n", + "\n", + "**Clinical Protocols:**\n", + "- For use in healthcare, create clinical guidelines that define the permissible role of AI in decision-making based on context, patient rights, and professional standards.\n", + "\n", + "**Ethical Usage Policies:**\n", + "- Develop clear policies that outline the conditions under which AI can be used in life-and-death scenarios, ensuring that the technology is implemented ethically.\n", + "\n", + "### Conclusion\n", + "\n", + "Balancing autonomy, accountability, and societal norms in AI decision-making, particularly in life-and-death situations, requires thoughtful integration of ethical principles, stakeholder engagement, transparency, and a dynamic regulatory environment. By taking these steps, we can navigate the complexities involved and work toward ethically sound AI systems that prioritize human welfare and dignity.\n", + "Competitor: llama-3.3-70b-versatile\n", + "\n", + "Resolving the ethical dilemmas of AI decision-making in life-and-death situations requires a multifaceted approach that balances autonomy, accountability, and societal norms. Here's a comprehensive framework to address these challenges:\n", + "\n", + "**I. Establish Clear Guidelines and Regulations**\n", + "\n", + "1. **Define autonomy boundaries**: Establish clear limits on AI decision-making authority, ensuring that humans are involved in critical decisions.\n", + "2. **Develop regulations and standards**: Create and enforce regulations, such as those related to transparency, explainability, and accountability, to guide AI development and deployment.\n", + "3. **Industry-wide collaboration**: Foster collaboration among AI developers, regulators, and stakeholders to establish common standards and best practices.\n", + "\n", + "**II. Implement Transparency and Explainability**\n", + "\n", + "1. **Model interpretability**: Develop techniques to explain AI decision-making processes, enabling humans to understand the reasoning behind AI-driven choices.\n", + "2. **Audit trails and logging**: Maintain detailed records of AI decision-making processes, allowing for post-hoc analysis and accountability.\n", + "3. **Regular model updates and validation**: Ensure that AI models are regularly updated and validated to prevent biases and inaccuracies.\n", + "\n", + "**III. Ensure Human Oversight and Review**\n", + "\n", + "1. **Human-in-the-loop**: Design AI systems that require human review and approval for critical decisions, particularly in life-and-death situations.\n", + "2. **Independent review boards**: Establish independent review boards to assess AI decision-making and provide feedback for improvement.\n", + "3. **Continuous monitoring and evaluation**: Regularly monitor AI performance and evaluate its impact on society, making adjustments as needed.\n", + "\n", + "**IV. Address Accountability and Liability**\n", + "\n", + "1. **Clear accountability structures**: Establish clear lines of accountability, defining roles and responsibilities for AI decision-making.\n", + "2. **Liability frameworks**: Develop frameworks to address liability and responsibility in cases where AI decision-making results in harm or damage.\n", + "3. **Insurance and risk management**: Explore insurance and risk management options to mitigate the financial and reputational risks associated with AI decision-making.\n", + "\n", + "**V. Engage with Societal Norms and Values**\n", + "\n", + "1. **Public engagement and education**: Engage with the public to raise awareness about AI decision-making and its implications, fostering a broader understanding of the technology.\n", + "2. **Value alignment**: Ensure that AI systems are designed to align with societal values, such as respect for human life, dignity, and autonomy.\n", + "3. **Cultural and contextual considerations**: Consider cultural and contextual factors that may influence AI decision-making, adapting AI systems to accommodate diverse perspectives and needs.\n", + "\n", + "**VI. Foster a Culture of Responsibility and Continuous Improvement**\n", + "\n", + "1. **Ethics-by-design**: Integrate ethical considerations into AI development from the outset, rather than treating ethics as an afterthought.\n", + "2. **Responsible AI development**: Encourage responsible AI development practices, prioritizing transparency, accountability, and human well-being.\n", + "3. **Ongoing research and development**: Continuously invest in research and development to improve AI decision-making, addressing emerging challenges and concerns.\n", + "\n", + "By adopting this comprehensive framework, we can work towards resolving the ethical dilemmas of AI decision-making in life-and-death situations, balancing autonomy, accountability, and societal norms to ensure that AI systems serve humanity's best interests.\n" + ] + } + ], + "source": [ + "# It's nice to know how to use \"zip\"\n", + "for competitor, answer in zip(competitors, answers):\n", + " print(f\"Competitor: {competitor}\\n\\n{answer}\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [], + "source": [ + "# Let's bring this together - note the use of \"enumerate\"\n", + "\n", + "together = \"\"\n", + "for index, answer in enumerate(answers):\n", + " together += f\"# Response from competitor {index+1}\\n\\n\"\n", + " together += answer + \"\\n\\n\"" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "# Response from competitor 1\n", + "\n", + "Resolving the ethical dilemmas of AI decision-making in life-and-death situations requires a multifaceted approach that carefully considers autonomy, accountability, and societal norms. Here’s how I would approach this complex issue:\n", + "\n", + "### 1. Establish Ethical Frameworks\n", + "\n", + "**Utilitarianism vs. Deontological Ethics:** \n", + "- Utilize a combination of ethical theories to guide AI decision-making. Utilitarianism focuses on the outcomes (the greatest good for the greatest number), while deontological ethics emphasizes duties and principles regardless of outcomes. An integrated approach can provide a more nuanced understanding of ethical dilemmas.\n", + "\n", + "**Principles of AI Ethics:**\n", + "- Develop and follow fundamental principles such as fairness, transparency, accountability, nondiscrimination, and respect for human dignity. These principles can help guide developers and stakeholders in creating AI systems.\n", + "\n", + "### 2. Incorporate Stakeholder Perspectives\n", + "\n", + "**Engaging Diverse Stakeholders:**\n", + "- Include a range of perspectives from ethicists, medical professionals, legal experts, sociologists, and the affected communities. This engagement will ensure that the AI systems reflect a broad spectrum of values and considerations.\n", + "\n", + "**Public Deliberation:**\n", + "- Conduct public consultations to gather input from the general populace regarding their values and concerns. This process helps ensure that societal norms and expectations are factored into AI decision-making.\n", + "\n", + "### 3. Define Autonomy and Accountability\n", + "\n", + "**Respect for Human Autonomy:**\n", + "- Design AI systems that prioritize human oversight, allowing individuals to retain agency over critical decisions. In life-and-death situations, this may involve allowing human operators to make the final call, especially when emotional and moral considerations are at stake.\n", + "\n", + "**Accountability Structures:**\n", + "- Clearly define who is accountable when AI systems make decisions that lead to life-and-death outcomes. This may involve delineating roles for software developers, healthcare providers, and institutions.\n", + "\n", + "**Legal and Regulatory Frameworks:**\n", + "- Advocate for the establishment of robust legal and regulatory frameworks that define liability for AI decision-making. This framework should specify accountability in cases of errors leading to harm or fatalities.\n", + "\n", + "### 4. Transparency and Explainability\n", + "\n", + "**Explainable AI (XAI):**\n", + "- Deploy AI systems that can provide understandable and interpretable reasons for their decisions. In life-and-death scenarios, users must comprehend the rationale behind medical suggestions or emergency responses.\n", + "\n", + "**Data and Algorithm Transparency:**\n", + "- Ensure transparency in data sources and algorithms used in AI systems. Stakeholders should understand how data biases may affect decision-making processes.\n", + "\n", + "### 5. Continuous Learning and Adaptation\n", + "\n", + "**Feedback Mechanisms:**\n", + "- Incorporate systems for continuous feedback, allowing for the assessment and revision of AI systems based on real-world outcomes and user experiences.\n", + "\n", + "**Ethical Review Boards:**\n", + "- Establish ethical review boards to regularly evaluate the performance and implications of AI systems in critical areas, ensuring they align with evolving societal norms and ethical considerations.\n", + "\n", + "### 6. Development of Clinical and Ethical Guidelines\n", + "\n", + "**Clinical Protocols:**\n", + "- For use in healthcare, create clinical guidelines that define the permissible role of AI in decision-making based on context, patient rights, and professional standards.\n", + "\n", + "**Ethical Usage Policies:**\n", + "- Develop clear policies that outline the conditions under which AI can be used in life-and-death scenarios, ensuring that the technology is implemented ethically.\n", + "\n", + "### Conclusion\n", + "\n", + "Balancing autonomy, accountability, and societal norms in AI decision-making, particularly in life-and-death situations, requires thoughtful integration of ethical principles, stakeholder engagement, transparency, and a dynamic regulatory environment. By taking these steps, we can navigate the complexities involved and work toward ethically sound AI systems that prioritize human welfare and dignity.\n", + "\n", + "# Response from competitor 2\n", + "\n", + "Resolving the ethical dilemmas of AI decision-making in life-and-death situations requires a multifaceted approach that balances autonomy, accountability, and societal norms. Here's a comprehensive framework to address these challenges:\n", + "\n", + "**I. Establish Clear Guidelines and Regulations**\n", + "\n", + "1. **Define autonomy boundaries**: Establish clear limits on AI decision-making authority, ensuring that humans are involved in critical decisions.\n", + "2. **Develop regulations and standards**: Create and enforce regulations, such as those related to transparency, explainability, and accountability, to guide AI development and deployment.\n", + "3. **Industry-wide collaboration**: Foster collaboration among AI developers, regulators, and stakeholders to establish common standards and best practices.\n", + "\n", + "**II. Implement Transparency and Explainability**\n", + "\n", + "1. **Model interpretability**: Develop techniques to explain AI decision-making processes, enabling humans to understand the reasoning behind AI-driven choices.\n", + "2. **Audit trails and logging**: Maintain detailed records of AI decision-making processes, allowing for post-hoc analysis and accountability.\n", + "3. **Regular model updates and validation**: Ensure that AI models are regularly updated and validated to prevent biases and inaccuracies.\n", + "\n", + "**III. Ensure Human Oversight and Review**\n", + "\n", + "1. **Human-in-the-loop**: Design AI systems that require human review and approval for critical decisions, particularly in life-and-death situations.\n", + "2. **Independent review boards**: Establish independent review boards to assess AI decision-making and provide feedback for improvement.\n", + "3. **Continuous monitoring and evaluation**: Regularly monitor AI performance and evaluate its impact on society, making adjustments as needed.\n", + "\n", + "**IV. Address Accountability and Liability**\n", + "\n", + "1. **Clear accountability structures**: Establish clear lines of accountability, defining roles and responsibilities for AI decision-making.\n", + "2. **Liability frameworks**: Develop frameworks to address liability and responsibility in cases where AI decision-making results in harm or damage.\n", + "3. **Insurance and risk management**: Explore insurance and risk management options to mitigate the financial and reputational risks associated with AI decision-making.\n", + "\n", + "**V. Engage with Societal Norms and Values**\n", + "\n", + "1. **Public engagement and education**: Engage with the public to raise awareness about AI decision-making and its implications, fostering a broader understanding of the technology.\n", + "2. **Value alignment**: Ensure that AI systems are designed to align with societal values, such as respect for human life, dignity, and autonomy.\n", + "3. **Cultural and contextual considerations**: Consider cultural and contextual factors that may influence AI decision-making, adapting AI systems to accommodate diverse perspectives and needs.\n", + "\n", + "**VI. Foster a Culture of Responsibility and Continuous Improvement**\n", + "\n", + "1. **Ethics-by-design**: Integrate ethical considerations into AI development from the outset, rather than treating ethics as an afterthought.\n", + "2. **Responsible AI development**: Encourage responsible AI development practices, prioritizing transparency, accountability, and human well-being.\n", + "3. **Ongoing research and development**: Continuously invest in research and development to improve AI decision-making, addressing emerging challenges and concerns.\n", + "\n", + "By adopting this comprehensive framework, we can work towards resolving the ethical dilemmas of AI decision-making in life-and-death situations, balancing autonomy, accountability, and societal norms to ensure that AI systems serve humanity's best interests.\n", + "\n", + "\n" + ] + } + ], + "source": [ + "print(together)" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [], + "source": [ + "judge = f\"\"\"You are judging a competition between {len(competitors)} competitors.\n", + "Each model has been given this question:\n", + "\n", + "{question}\n", + "\n", + "Your job is to evaluate each response for clarity and strength of argument, and rank them in order of best to worst.\n", + "Respond with JSON, and only JSON, with the following format:\n", + "{{\"results\": [\"best competitor number\", \"second best competitor number\", \"third best competitor number\", ...]}}\n", + "\n", + "Here are the responses from each competitor:\n", + "\n", + "{together}\n", + "\n", + "Now respond with the JSON with the ranked order of the competitors, nothing else. Do not include markdown formatting or code blocks.\"\"\"\n" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "You are judging a competition between 2 competitors.\n", + "Each model has been given this question:\n", + "\n", + "How would you approach resolving the ethical dilemmas of AI decision-making in life-and-death situations, balancing autonomy, accountability, and societal norms?\n", + "\n", + "Your job is to evaluate each response for clarity and strength of argument, and rank them in order of best to worst.\n", + "Respond with JSON, and only JSON, with the following format:\n", + "{\"results\": [\"best competitor number\", \"second best competitor number\", \"third best competitor number\", ...]}\n", + "\n", + "Here are the responses from each competitor:\n", + "\n", + "# Response from competitor 1\n", + "\n", + "Resolving the ethical dilemmas of AI decision-making in life-and-death situations requires a multifaceted approach that carefully considers autonomy, accountability, and societal norms. Here’s how I would approach this complex issue:\n", + "\n", + "### 1. Establish Ethical Frameworks\n", + "\n", + "**Utilitarianism vs. Deontological Ethics:** \n", + "- Utilize a combination of ethical theories to guide AI decision-making. Utilitarianism focuses on the outcomes (the greatest good for the greatest number), while deontological ethics emphasizes duties and principles regardless of outcomes. An integrated approach can provide a more nuanced understanding of ethical dilemmas.\n", + "\n", + "**Principles of AI Ethics:**\n", + "- Develop and follow fundamental principles such as fairness, transparency, accountability, nondiscrimination, and respect for human dignity. These principles can help guide developers and stakeholders in creating AI systems.\n", + "\n", + "### 2. Incorporate Stakeholder Perspectives\n", + "\n", + "**Engaging Diverse Stakeholders:**\n", + "- Include a range of perspectives from ethicists, medical professionals, legal experts, sociologists, and the affected communities. This engagement will ensure that the AI systems reflect a broad spectrum of values and considerations.\n", + "\n", + "**Public Deliberation:**\n", + "- Conduct public consultations to gather input from the general populace regarding their values and concerns. This process helps ensure that societal norms and expectations are factored into AI decision-making.\n", + "\n", + "### 3. Define Autonomy and Accountability\n", + "\n", + "**Respect for Human Autonomy:**\n", + "- Design AI systems that prioritize human oversight, allowing individuals to retain agency over critical decisions. In life-and-death situations, this may involve allowing human operators to make the final call, especially when emotional and moral considerations are at stake.\n", + "\n", + "**Accountability Structures:**\n", + "- Clearly define who is accountable when AI systems make decisions that lead to life-and-death outcomes. This may involve delineating roles for software developers, healthcare providers, and institutions.\n", + "\n", + "**Legal and Regulatory Frameworks:**\n", + "- Advocate for the establishment of robust legal and regulatory frameworks that define liability for AI decision-making. This framework should specify accountability in cases of errors leading to harm or fatalities.\n", + "\n", + "### 4. Transparency and Explainability\n", + "\n", + "**Explainable AI (XAI):**\n", + "- Deploy AI systems that can provide understandable and interpretable reasons for their decisions. In life-and-death scenarios, users must comprehend the rationale behind medical suggestions or emergency responses.\n", + "\n", + "**Data and Algorithm Transparency:**\n", + "- Ensure transparency in data sources and algorithms used in AI systems. Stakeholders should understand how data biases may affect decision-making processes.\n", + "\n", + "### 5. Continuous Learning and Adaptation\n", + "\n", + "**Feedback Mechanisms:**\n", + "- Incorporate systems for continuous feedback, allowing for the assessment and revision of AI systems based on real-world outcomes and user experiences.\n", + "\n", + "**Ethical Review Boards:**\n", + "- Establish ethical review boards to regularly evaluate the performance and implications of AI systems in critical areas, ensuring they align with evolving societal norms and ethical considerations.\n", + "\n", + "### 6. Development of Clinical and Ethical Guidelines\n", + "\n", + "**Clinical Protocols:**\n", + "- For use in healthcare, create clinical guidelines that define the permissible role of AI in decision-making based on context, patient rights, and professional standards.\n", + "\n", + "**Ethical Usage Policies:**\n", + "- Develop clear policies that outline the conditions under which AI can be used in life-and-death scenarios, ensuring that the technology is implemented ethically.\n", + "\n", + "### Conclusion\n", + "\n", + "Balancing autonomy, accountability, and societal norms in AI decision-making, particularly in life-and-death situations, requires thoughtful integration of ethical principles, stakeholder engagement, transparency, and a dynamic regulatory environment. By taking these steps, we can navigate the complexities involved and work toward ethically sound AI systems that prioritize human welfare and dignity.\n", + "\n", + "# Response from competitor 2\n", + "\n", + "Resolving the ethical dilemmas of AI decision-making in life-and-death situations requires a multifaceted approach that balances autonomy, accountability, and societal norms. Here's a comprehensive framework to address these challenges:\n", + "\n", + "**I. Establish Clear Guidelines and Regulations**\n", + "\n", + "1. **Define autonomy boundaries**: Establish clear limits on AI decision-making authority, ensuring that humans are involved in critical decisions.\n", + "2. **Develop regulations and standards**: Create and enforce regulations, such as those related to transparency, explainability, and accountability, to guide AI development and deployment.\n", + "3. **Industry-wide collaboration**: Foster collaboration among AI developers, regulators, and stakeholders to establish common standards and best practices.\n", + "\n", + "**II. Implement Transparency and Explainability**\n", + "\n", + "1. **Model interpretability**: Develop techniques to explain AI decision-making processes, enabling humans to understand the reasoning behind AI-driven choices.\n", + "2. **Audit trails and logging**: Maintain detailed records of AI decision-making processes, allowing for post-hoc analysis and accountability.\n", + "3. **Regular model updates and validation**: Ensure that AI models are regularly updated and validated to prevent biases and inaccuracies.\n", + "\n", + "**III. Ensure Human Oversight and Review**\n", + "\n", + "1. **Human-in-the-loop**: Design AI systems that require human review and approval for critical decisions, particularly in life-and-death situations.\n", + "2. **Independent review boards**: Establish independent review boards to assess AI decision-making and provide feedback for improvement.\n", + "3. **Continuous monitoring and evaluation**: Regularly monitor AI performance and evaluate its impact on society, making adjustments as needed.\n", + "\n", + "**IV. Address Accountability and Liability**\n", + "\n", + "1. **Clear accountability structures**: Establish clear lines of accountability, defining roles and responsibilities for AI decision-making.\n", + "2. **Liability frameworks**: Develop frameworks to address liability and responsibility in cases where AI decision-making results in harm or damage.\n", + "3. **Insurance and risk management**: Explore insurance and risk management options to mitigate the financial and reputational risks associated with AI decision-making.\n", + "\n", + "**V. Engage with Societal Norms and Values**\n", + "\n", + "1. **Public engagement and education**: Engage with the public to raise awareness about AI decision-making and its implications, fostering a broader understanding of the technology.\n", + "2. **Value alignment**: Ensure that AI systems are designed to align with societal values, such as respect for human life, dignity, and autonomy.\n", + "3. **Cultural and contextual considerations**: Consider cultural and contextual factors that may influence AI decision-making, adapting AI systems to accommodate diverse perspectives and needs.\n", + "\n", + "**VI. Foster a Culture of Responsibility and Continuous Improvement**\n", + "\n", + "1. **Ethics-by-design**: Integrate ethical considerations into AI development from the outset, rather than treating ethics as an afterthought.\n", + "2. **Responsible AI development**: Encourage responsible AI development practices, prioritizing transparency, accountability, and human well-being.\n", + "3. **Ongoing research and development**: Continuously invest in research and development to improve AI decision-making, addressing emerging challenges and concerns.\n", + "\n", + "By adopting this comprehensive framework, we can work towards resolving the ethical dilemmas of AI decision-making in life-and-death situations, balancing autonomy, accountability, and societal norms to ensure that AI systems serve humanity's best interests.\n", + "\n", + "\n", + "\n", + "Now respond with the JSON with the ranked order of the competitors, nothing else. Do not include markdown formatting or code blocks.\n" + ] + } + ], + "source": [ + "print(judge)" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [], + "source": [ + "judge_messages = [{\"role\": \"user\", \"content\": judge}]" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"results\": [\"1\", \"2\"]}\n" + ] + } + ], + "source": [ + "# Judgement time!\n", + "\n", + "openai = OpenAI()\n", + "response = openai.chat.completions.create(\n", + " model=\"o3-mini\",\n", + " messages=judge_messages,\n", + ")\n", + "results = response.choices[0].message.content\n", + "print(results)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Rank 1: gpt-4o-mini\n", + "Rank 2: llama-3.3-70b-versatile\n" + ] + } + ], + "source": [ + "# OK let's turn this into results!\n", + "\n", + "results_dict = json.loads(results)\n", + "ranks = results_dict[\"results\"]\n", + "for index, result in enumerate(ranks):\n", + " competitor = competitors[int(result)-1]\n", + " print(f\"Rank {index+1}: {competitor}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + "

Exercise

\n", + " Which pattern(s) did this use? Try updating this to add another Agentic design pattern.\n", + " \n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + "

Commercial implications

\n", + " These kinds of patterns - to send a task to multiple models, and evaluate results,\n", + " are common where you need to improve the quality of your LLM response. This approach can be universally applied\n", + " to business projects where accuracy is critical.\n", + " \n", + "
" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.7" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}