diff --git "a/finetuning/Fine_tuning_Embedding_Model_for_RAG_InsightFlowAI.ipynb" "b/finetuning/Fine_tuning_Embedding_Model_for_RAG_InsightFlowAI.ipynb" new file mode 100644--- /dev/null +++ "b/finetuning/Fine_tuning_Embedding_Model_for_RAG_InsightFlowAI.ipynb" @@ -0,0 +1,16732 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "ckbbj5diaHkg" + }, + "source": [ + "# Fine-tuning Embeddings for RAG on Specific Data\n", + "\n", + "As we start our \"fine-tuning\" week, we'll start with the lowest hanging improvement one can do for RAG - which is:\n", + "\n", + "Fine-tuning embeddings!\n", + "\n", + "- 🀝 Breakout Room #1:\n", + " - Task 1: Dependencies and Boilerplate\n", + " - Task 2: Loading Data\n", + " - Task 3: Constructing a Fine-tuning Dataset\n", + " - Task 4: Fine-tuning `snowflake-arctic-embed-l`\n", + " - Task 5: Evaluating our Retriever\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "2xwor_3X6ODX" + }, + "source": [ + "#### Basic Overview of Fine-tuning Embeddings\n", + "\n", + "In essence, what we want to do when we fine-tune our embedding models is very simple:\n", + "\n", + "```\n", + "Move the embeddings for questions relating to a document\n", + "closer together with that document\n", + "```\n", + "\n", + "We can think of fine-tuning our embedding models as follows:\n", + "\n", + "1) We have some pair of text items that *should* be closer together\n", + " - `Question`, `Document` pairs\n", + " - EX: `Who drives the bus?`, `The bus was driven by Kyle, the Bus Driver`.\n", + "\n", + "2) We use these pairs as labeled data to fine-tune our embedding model.\n", + "\n", + "The process of training helps the model more accurately associate our questions with the correct documents." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "DX5R3HVz6FOQ" + }, + "source": [ + "##### ❓ Question #1:\n", + "\n", + "Describe the nuance between using Q&D pairs to train the embedding model vs. inter-document pairs/related sentences.\n", + "\n", + "What caveats does this approach have? Are there any special considerations for what kind of Q's we should use?\n", + "\n", + " Q&D (Question & Document/Answer): This is the approach we're taking in this notebook. We generate questions that should be answered by a specific chunk of text (our \"document\" or context). The goal is to train the model to make the embedding for the question very similar (close in vector space) to the embedding of the document chunk that answers it. This directly optimizes for the core RAG task: finding the right context for a given question.\n", + "\n", + "Inter-document Pairs/Related Sentences: This approach involves identifying pairs of sentences or document chunks that are inherently related or similar without necessarily being a question-answer pair. For example, two paragraphs discussing the same specific concept, or a statement and its elaboration. Training on these pairs teaches the model general semantic similarity – making embeddings for related content closer together.\n", + "\n", + "The Q&D pair approach is more targeted because it directly mimics the core task of a RAG system: retrieving relevant context (D) based on a user's query (Q). By training the model to pull question embeddings closer to their corresponding answer/context embeddings, **we are explicitly optimizing the retriever for its intended function within the RAG pipeline.**\n", + "\n", + "The effectiveness of this fine-tuning heavily relies on the representativeness of the generated questions. If the synthetic questions we create for training don't reflect the types of questions users will actually ask the RAG system, the fine-tuning might not translate into real-world performance improvements.\n", + "\n", + "\n", + "If The LLM generates very simple pointed questions for our fine tuning and the users end up asking more nuanced thematic questions, our fine tuning will not help with that. LLM is also big brained and one-size-fits-all so it is more general and our use case might be specific so **its important that our fine tuning q and a set captures the specificity of our use case without being tied to the exact wording (that will make the embeddings too brittle) and that it captures the nuance and complexity of the domain as well.** So, well formed, relevant, specific to domain without being brittle, and representative of what users will ask.\n", + "\n", + "\n", + "\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "-NkSaurzbpyS" + }, + "source": [ + "## Task 1: Dependencies and Boilerplate\n", + "\n", + "We'll set up our `nest_asyncio` so we can leverage async loops in our Notebook.\n", + "\n", + "We'll also install the required libraries we'll be using today, and set up our OpenAI API key!" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9c_EUibmcDU3" + }, + "source": [ + "### Nest Asyncio" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "id": "zq-6s7LbPnKH" + }, + "outputs": [], + "source": [ + "import nest_asyncio\n", + "\n", + "nest_asyncio.apply()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "R8uFz8RVcFFu" + }, + "source": [ + "### Install Dependencies\n", + "\n", + ">> NOTE: You do not need to do these steps if you are running this notebook locally with `uv`." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "id": "ulZIBA1ZoSsV", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "bc59fca3-197d-4d8d-a45a-edb635f8d674" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/62.8 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.8/62.8 kB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/2.5 MB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[91mβ•Έ\u001b[0m \u001b[32m2.5/2.5 MB\u001b[0m \u001b[31m124.7 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.5/2.5 MB\u001b[0m \u001b[31m72.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/44.4 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m44.4/44.4 kB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.9/50.9 kB\u001b[0m \u001b[31m4.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m363.4/363.4 MB\u001b[0m \u001b[31m3.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m13.8/13.8 MB\u001b[0m \u001b[31m128.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m24.6/24.6 MB\u001b[0m \u001b[31m105.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m883.7/883.7 kB\u001b[0m \u001b[31m58.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m664.8/664.8 MB\u001b[0m \u001b[31m1.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m211.5/211.5 MB\u001b[0m \u001b[31m11.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m56.3/56.3 MB\u001b[0m \u001b[31m41.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m127.9/127.9 MB\u001b[0m \u001b[31m19.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m207.5/207.5 MB\u001b[0m \u001b[31m3.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.1/21.1 MB\u001b[0m \u001b[31m108.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h" + ] + } + ], + "source": [ + "!pip install -qU langchain_openai langchain_huggingface langchain_core langchain langchain_community langchain-text-splitters" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "id": "3GFD7B-tOCrx", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "3c0ddc70-77bc-45c3-e02c-196e6f00ca5a" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m472.8/472.8 kB\u001b[0m \u001b[31m19.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m31.3/31.3 MB\u001b[0m \u001b[31m80.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.0/20.0 MB\u001b[0m \u001b[31m108.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m169.4/169.4 kB\u001b[0m \u001b[31m15.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h" + ] + } + ], + "source": [ + "!pip install -qU faiss-cpu python-pptx==1.0.2 nltk==3.9.1 pymupdf beautifulsoup4 lxml" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0FM-eUlrcI8a" + }, + "source": [ + "### Provide OpenAI API Key" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "id": "wA_mlurVqtrp" + }, + "outputs": [], + "source": [ + "import os\n", + "import getpass\n", + "from google.colab import userdata\n", + "\n", + "os.environ[\"OPENAI_API_KEY\"] = userdata.get('OPENAI_API_KEY') # getpass.getpass(\"Enter Your OpenAI API Key: \")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "TFZ217gCDVTr" + }, + "source": [ + "## Task 2: Loading Data\n", + "\n", + "We'll prepare our data - and download our webpages which we'll be using for our data today.\n", + "\n", + "These webpages are from [Simon Willison's](https://simonwillison.net/) yearly \"AI learnings\".\n", + "\n", + "- [2023 Blog](https://simonwillison.net/2023/Dec/31/ai-in-2023/)\n", + "- [2024 Blog](https://simonwillison.net/2024/Dec/31/llms-in-2024/)\n", + "\n", + "Let's start by collecting our data into a useful pile!" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "id": "tSoqUqKeXkWR" + }, + "outputs": [], + "source": [ + "!mkdir data\n", + "!mkdir data_sources" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "id": "Fh9b7aJ7XkWR", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "dcaafe6b-2cae-422f-81a2-aa5a2dbeb540" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "100 31500 0 31500 0 0 71791 0 --:--:-- --:--:-- --:--:-- 71753\n" + ] + } + ], + "source": [ + "!curl https://simonwillison.net/2023/Dec/31/ai-in-2023/ -o data/2023_llms.html" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "id": "hP6vNjx0XkWS", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "8c8acc8e-313d-4af3-873e-627c5be88613" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 70667 0 70667 0 0 395k 0 --:--:-- --:--:-- --:--:-- 396k\n" + ] + } + ], + "source": [ + "!curl https://simonwillison.net/2024/Dec/31/llms-in-2024/ -o data/2024_llms.html" + ] + }, + { + "cell_type": "code", + "source": [ + "from langchain_community.document_loaders import DirectoryLoader, UnstructuredFileLoader\n", + "from langchain_core.documents import Document\n", + "import os\n", + "\n", + "# Try to import Colab specific modules\n", + "try:\n", + " from google.colab import files\n", + " IN_COLAB = True\n", + "except ImportError:\n", + " IN_COLAB = False\n", + "\n", + "all_documents = []\n", + "\n", + "# Section for optional direct file upload (primarily for Colab)\n", + "if IN_COLAB:\n", + " print(\"This script can prompt for a direct file upload if run in Google Colab.\")\n", + " try:\n", + " upload_choice = input(\"Do you want to upload an additional single text file directly? (yes/no): \").strip().lower()\n", + " if upload_choice == 'yes':\n", + " print(\"Please use the file dialog that appears to select a .txt file to upload.\")\n", + " # files.upload() returns a dictionary of {filename: content_bytes}\n", + " uploaded_files = files.upload()\n", + "\n", + " if not uploaded_files:\n", + " print(\"No file was selected or the upload was cancelled.\")\n", + " else:\n", + " for file_name, file_content_bytes in uploaded_files.items():\n", + " if file_name.endswith(\".txt\"):\n", + " try:\n", + " # Decode the bytes to string (assuming UTF-8 encoding for text files)\n", + " file_content_str = file_content_bytes.decode('utf-8')\n", + "\n", + " # Prompt for a persona_id for the uploaded file\n", + " persona_id_for_uploaded = input(f\"Enter a persona_id for the uploaded file '{file_name}' (e.g., analytical, philosophical, custom_source): \").strip()\n", + " if not persona_id_for_uploaded:\n", + " persona_id_for_uploaded = \"uploaded_file\" # Assign a default if none is provided\n", + "\n", + " # Create a Langchain Document\n", + " doc = Document(page_content=file_content_str, metadata={\"source\": file_name, \"persona_id\": persona_id_for_uploaded})\n", + " all_documents.append(doc)\n", + " print(f\"Successfully processed and added uploaded file: '{file_name}' with persona_id: '{persona_id_for_uploaded}'\")\n", + " except Exception as e:\n", + " print(f\"Error decoding or processing uploaded file '{file_name}': {e}\")\n", + " else:\n", + " print(f\"Skipping uploaded file '{file_name}' as it does not end with .txt.\")\n", + " else:\n", + " print(\"Skipping direct file upload based on your choice.\")\n", + " except RuntimeError:\n", + " print(\"File upload failed. This can happen if the browser blocks third-party cookies or due to other Colab-specific issues. Continuing without direct upload.\")\n", + " except Exception as e:\n", + " # Catch any other unexpected errors during the upload prompt/process\n", + " print(f\"An unexpected error occurred during the file upload attempt: {e}\")\n", + " print(\"Continuing without direct file upload.\")\n", + "\n", + "else:\n", + " print(\"Note: Direct file upload prompt is skipped as this environment does not appear to be Google Colab.\")\n", + " print(\"Please ensure your files are in the 'data_sources' directories as described below to load them.\")\n", + "\n", + "# --- Loading from data_sources directories (continues as before) ---\n", + "persona_paths = {\n", + " \"analytical\": \"data_sources/analytical/\",\n", + " \"philosophical\": \"data_sources/philosophical/\",\n", + " \"metaphorical\": \"data_sources/metaphorical/\"\n", + "}\n", + "\n", + "# Create directories if they don't exist and remind about uploads\n", + "for persona_id, path in persona_paths.items():\n", + " if not os.path.exists(path):\n", + " print(f\"Creating directory: {path} (if it doesn't exist)\")\n", + " os.makedirs(path, exist_ok=True)\n", + "\n", + "print(\"\\\\nIf you didn't use the direct upload, or if you want to load more files,\")\n", + "print(\"please ensure your .txt files are now in the respective 'data_sources/' folders.\\\\n\")\n", + "\n", + "# Load documents from directories\n", + "for persona_id, path in persona_paths.items():\n", + " if not os.path.exists(path):\n", + " # This check is somewhat redundant if os.makedirs worked, but good for robustness\n", + " print(f\"Warning: Directory {path} still does not exist. Skipping {persona_id} data.\")\n", + " continue\n", + "\n", + " # Check if the directory is empty before attempting to load\n", + " if not os.listdir(path):\n", + " print(f\"Notice: Directory {path} is empty. No files to load for {persona_id}.\")\n", + " continue\n", + "\n", + " print(f\"Attempting to load documents for {persona_id} from {path}...\")\n", + " loader = DirectoryLoader(\n", + " path,\n", + " glob=\"**/*.txt\", # Ensure it loads only .txt files\n", + " loader_cls=UnstructuredFileLoader, # Use for .txt files\n", + " show_progress=True,\n", + " use_multithreading=True,\n", + " silent_errors=True # Silently ignore files that can't be loaded\n", + " )\n", + " try:\n", + " loaded_docs_from_dir = loader.load()\n", + " for doc in loaded_docs_from_dir: # Add persona_id to metadata\n", + " doc.metadata[\"persona_id\"] = persona_id\n", + " # The 'source' metadata from DirectoryLoader usually contains the full file path\n", + " all_documents.extend(loaded_docs_from_dir)\n", + " if loaded_docs_from_dir:\n", + " print(f\"Successfully loaded {len(loaded_docs_from_dir)} documents for {persona_id} from {path}.\")\n", + " else:\n", + " print(f\"No .txt documents found or loaded for {persona_id} from {path} (directory might contain other file types or be empty of .txt).\")\n", + " except Exception as e:\n", + " print(f\"An error occurred while loading documents for {persona_id} from {path}: {e}\")\n", + "\n", + "# Final check and summary\n", + "if not all_documents:\n", + " print(\"\\\\nError: No documents were loaded in total (neither via upload nor from directories).\")\n", + " print(\"Please ensure your 'data_sources/' directories are populated with .txt files or use the upload option if available.\")\n", + "else:\n", + " print(f\"\\\\nTotal documents loaded into 'all_documents': {len(all_documents)}\")\n", + "\n", + "# You can then inspect the first few documents if needed:\n", + "# print(\"\\\\n--- Sample of Loaded Documents ---\")\n", + "# for i, doc in enumerate(all_documents[:3]): # Show up to 3 sample docs\n", + "# print(f\"--- Document {i+1} ---\")\n", + "# print(f\"Content (first 200 chars): {doc.page_content[:200]}...\")\n", + "# print(f\"Metadata: {doc.metadata}\")\n", + "# print(\"-----------------------------\")" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 769 + }, + "id": "EzzuaTzeDMCx", + "outputId": "47e8b4f8-56ba-4810-cfb4-6722bbea72f6" + }, + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "This script can prompt for a direct file upload if run in Google Colab.\n", + "Do you want to upload an additional single text file directly? (yes/no): yes\n", + "Please use the file dialog that appears to select a .txt file to upload.\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n", + " \n", + " \n", + " Upload widget is only available when the cell has been executed in the\n", + " current browser session. Please rerun this cell to enable.\n", + " \n", + " " + ] + }, + "metadata": {} + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Saving examples.txt to examples.txt\n", + "Saving excerpts.txt to excerpts.txt\n", + "Saving generated_analytical_example.txt to generated_analytical_example.txt\n", + "Saving hannah_fry_mathematics_of_love_transcript.html to hannah_fry_mathematics_of_love_transcript.html\n", + "Saving hannah_fry_uncertainty_unavoidable_transcript.html to hannah_fry_uncertainty_unavoidable_transcript.html\n", + "Saving pew_research_ai_views_2023.html to pew_research_ai_views_2023.html\n", + "Saving pew_research_report_ai_views_2023.txt to pew_research_report_ai_views_2023.txt\n", + "Saving sagan_baloney_detection.txt to sagan_baloney_detection.txt\n" + ] + }, + { + "output_type": "error", + "ename": "KeyboardInterrupt", + "evalue": "Interrupted by user", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)", + "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 32\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 33\u001b[0m \u001b[0;31m# Prompt for a persona_id for the uploaded file\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 34\u001b[0;31m \u001b[0mpersona_id_for_uploaded\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minput\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mf\"Enter a persona_id for the uploaded file '{file_name}' (e.g., analytical, philosophical, custom_source): \"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstrip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 35\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mpersona_id_for_uploaded\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 36\u001b[0m \u001b[0mpersona_id_for_uploaded\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"uploaded_file\"\u001b[0m \u001b[0;31m# Assign a default if none is provided\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/ipykernel/kernelbase.py\u001b[0m in \u001b[0;36mraw_input\u001b[0;34m(self, prompt)\u001b[0m\n\u001b[1;32m 1175\u001b[0m \u001b[0;34m\"raw_input was called, but this frontend does not support input requests.\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1176\u001b[0m )\n\u001b[0;32m-> 1177\u001b[0;31m return self._input_request(\n\u001b[0m\u001b[1;32m 1178\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprompt\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1179\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_parent_ident\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"shell\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/ipykernel/kernelbase.py\u001b[0m in \u001b[0;36m_input_request\u001b[0;34m(self, prompt, ident, parent, password)\u001b[0m\n\u001b[1;32m 1217\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mKeyboardInterrupt\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1218\u001b[0m \u001b[0;31m# re-raise KeyboardInterrupt, to truncate traceback\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1219\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mKeyboardInterrupt\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Interrupted by user\"\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1220\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mException\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1221\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlog\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwarning\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Invalid Message:\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexc_info\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;31mKeyboardInterrupt\u001b[0m: Interrupted by user" + ] + } + ] + }, + { + "cell_type": "code", + "source": [ + "from google.colab import drive\n", + "drive.mount('/content/drive')" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "A1k4VOHQElIv", + "outputId": "02806f07-0d65-43f8-810d-bda93058df82" + }, + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Mounted at /content/drive\n" + ] + } + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "id": "DHJhTzsvN75t" + }, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import DirectoryLoader\n", + "from langchain_community.document_loaders import BSHTMLLoader\n", + "\n", + "path = \"data/\"\n", + "text_loader = DirectoryLoader(path, glob=\"*.txt\", loader_cls=BSHTMLLoader)" + ] + }, + { + "cell_type": "code", + "source": [ + " from langchain_community.document_loaders import DirectoryLoader, TextLoader\n", + " from langchain_text_splitters import RecursiveCharacterTextSplitter # Ensure this import is present\n", + "\n", + " # Define the base path to where you unzipped your data\n", + " # This should match the '-d' argument from your !unzip command in Step 3d\n", + " base_data_path = \"data/\"\n", + "\n", + " # If your zip file directly contained 'analytical', 'philosophical', 'metaphorical' folders:\n", + " # Option 1: Load all .txt files from all subdirectories within base_data_path\n", + " print(f\"Loading .txt files from all subdirectories in: {base_data_path}\")\n", + " text_loader = DirectoryLoader(\n", + " base_data_path,\n", + " glob=\"**/*.txt\", # This means all .txt files in all subfolders\n", + " loader_cls=TextLoader,\n", + " show_progress=True,\n", + " use_multithreading=True, # Can speed up loading\n", + " loader_kwargs={'autodetect_encoding': True} # Helps with different text encodings\n", + " )\n", + " all_loaded_documents = text_loader.load()\n", + " print(f\"Successfully loaded {len(all_loaded_documents)} documents.\")\n", + "\n", + " # The notebook later defines text_splitter and uses it:\n", + " # text_splitter = RecursiveCharacterTextSplitter(\n", + " # chunk_size = 750, # You can adjust this if needed\n", + " # chunk_overlap = 20, # You can adjust this if needed\n", + " # length_function = len\n", + " # )\n", + " # training_documents = text_splitter.split_documents(text_loader.load()) # OLD LINE\n", + "\n", + " # Make sure 'training_documents' uses your loaded data.\n", + " # First, ensure text_splitter is defined by running its cell.\n", + " # Then, split your documents:\n", + " # training_documents = text_splitter.split_documents(all_loaded_documents)\n", + " # print(f\"Split into {len(training_documents)} chunks for training/validation/testing.\")" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "sHSFMfyySHYZ", + "outputId": "a27756c7-761d-4405-9808-3981e7d2248b" + }, + "execution_count": 7, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Loading .txt files from all subdirectories in: data/\n" + ] + }, + { + "output_type": "stream", + "name": "stderr", + "text": [ + "100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 18/18 [00:00<00:00, 1525.79it/s]" + ] + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Successfully loaded 18 documents.\n" + ] + }, + { + "output_type": "stream", + "name": "stderr", + "text": [ + "\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "-UbKa6-V0nvp" + }, + "source": [ + "Next, we'll set up a classic naive chunking strategy as we only care that the documents get parsed into chunks that we can generate synthetic questions about.\n", + "\n", + "Chunk size:\n", + "\n", + "Too Small: Chunks might lack sufficient context to fully capture an idea or answer a question. The embedding might not be specific enough.\n", + "\n", + "Too Large: The embedding might become too general, averaging the meaning across multiple distinct points. When retrieved, this large chunk might contain the relevant info but also a lot of noise, potentially making it harder for the final LLM to pinpoint the answer (diluting the key information).\n", + "\n", + "The 750-character size with 20-character overlap is a common starting point, aiming for that balance. The RecursiveCharacterTextSplitter is helpful because it tries to split along natural boundaries (paragraphs, sentences) first before resorting to a hard character limit, which helps keep the chunks more coherent.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "id": "NsPrOOqXOsNX" + }, + "outputs": [], + "source": [ + "from langchain_text_splitters import RecursiveCharacterTextSplitter\n", + "\n", + "text_splitter = RecursiveCharacterTextSplitter(\n", + " chunk_size = 750,\n", + " chunk_overlap = 20,\n", + " length_function = len\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "lf_PoX7l09Rg" + }, + "source": [ + "Next we can load/split these documents as follows.\n", + "\n", + "> NOTE: You may need to run this cell twice to get it to work." + ] + }, + { + "cell_type": "code", + "source": [ + "# Ensure text_splitter is defined from its cell, then:\n", + "training_documents = text_splitter.split_documents(all_loaded_documents) # Use your variable here\n", + "print(f\"Split into {len(training_documents)} chunks for training/validation/testing.\")" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "mz2r_42ZV1ns", + "outputId": "e32765ff-e986-4549-98ae-23bfa83e14f8" + }, + "execution_count": 12, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Split into 2261 chunks for training/validation/testing.\n" + ] + } + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "id": "OMYPX6N6Os8M", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "c573618a-aecd-4ba7-a73d-6340435da559" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 18/18 [00:00<00:00, 1641.36it/s]\n" + ] + } + ], + "source": [ + "training_documents = text_splitter.split_documents(text_loader.load())" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "id": "PAozuMoNOvnp", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "12b884a5-cddb-4dc0-cd79-42ab085409d7" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "2261" + ] + }, + "metadata": {}, + "execution_count": 13 + } + ], + "source": [ + "len(training_documents)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0yE2TFIq1BuJ" + }, + "source": [ + "Next, we're going to associate each of our chunks with a unique identifier." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "id": "AwyIForybIpo" + }, + "outputs": [], + "source": [ + "import uuid\n", + "\n", + "id_set = set()\n", + "\n", + "for document in training_documents:\n", + " id = str(uuid.uuid4())\n", + " while id in id_set:\n", + " id = uuid.uuid4()\n", + " id_set.add(id)\n", + " document.metadata[\"id\"] = id" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "PJnL4oNg341U" + }, + "source": [ + "Next, we'll simply use naive Python slicing to create a training, test, and validation set to prepare our data for the next step. \\\n", + "78 chunks: training\\\n", + "12 chunks: validation\\\n", + "12 chunks: testing\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "VvczX7VuXkWS" + }, + "source": [ + "**Validation vs. Test Sets:**\n", + "\n", + "The training set is used directly to update the model's parameters during the fine-tuning process. The model \"learns\" from these examples.\n", + "\n", + "The validation set plays a crucial role **during** training. Periodically (e.g., after each epoch or a certain number of steps), the model's performance is checked against the validation set. This helps us:\n", + "\n", + "Tune hyperparameters: See if different learning rates, batch sizes, etc., lead to better performance on data the model hasn't directly trained on.\n", + "\n", + "Prevent overfitting: Monitor if the model is getting really good at the training data but worse on the validation data (meaning it's just memorizing the training set and not generalizing). We can use this to decide when to stop training (early stopping).\n", + "\n", + "Model selection: If we train multiple versions of the model, the validation set helps us pick the best one.\n", + "\n", + "The test set is held back until the very end. After we've finished training and selected our final model (using the training and validation sets), we evaluate its performance one last time on the test set. This gives us an unbiased estimate of how well the model is likely to perform on completely new, unseen data in the real world. We don't use the test set to make any decisions about training or model selection; it's purely for the final report card." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "id": "MTS4GTSEcnG4" + }, + "outputs": [], + "source": [ + "training_split_documents = training_documents[:len(training_documents) - 480]\n", + "val_split_documents = training_documents[len(training_documents) - 480:len(training_documents) - 240]\n", + "test_split_documents = training_documents[len(training_documents) - 240:]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tzlvKbONDWvQ" + }, + "source": [ + "## Task 3: Constructing a Fine-tuning Dataset\n", + "\n", + "Using the nodes we created above, we can finally start constructing a fine-tuning dataset utilizing OpenAI's `gpt-4.1-mini`\n", + "\n", + "The basic idea here is straightforward enough:\n", + "\n", + "1. We look at a document\n", + "2. We generate questions that could be answered by that node\n", + "\n", + "This gives us a number of question/context pairs that we can use to fine-tune our Embeddings model." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "fbPzMxFZXkWT" + }, + "source": [ + "Why temperature=0?\n", + "\n", + "In LLMs, temperature controls the randomness of the output.\n", + "\n", + "A higher temperature (e.g., 0.7-1.0) makes the output more random and creative. The model is more likely to explore less probable word choices.\n", + "\n", + "A lower temperature (closer to 0) makes the output more deterministic and focused. The model tends to pick the most likely next word.\n", + "\n", + "When generating questions for our fine-tuning dataset, we want them to be factual, directly based on the provided context, and consistent.\n", + "\n", + "We don't want creative or unexpected questions here. Setting temperature=0 helps ensure the LLM produces the most probable, focused, and contextually grounded questions, minimizing randomness and increasing reproducibility." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "id": "_EWfmIscMrvg" + }, + "outputs": [], + "source": [ + "from langchain_openai import ChatOpenAI\n", + "\n", + "qa_chat_model = ChatOpenAI(\n", + " model=\"gpt-4.1-mini\",\n", + " temperature=0\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "8-hLnsSB6Y-S" + }, + "source": [ + "We'll create a simple Question Generation prompt to query `gpt-4o-mini` to generate Questions for each retrieved context." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "id": "diEWcw00NMSj" + }, + "outputs": [], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "qa_prompt = \"\"\"\\\n", + "Given the following context, you must generate questions based on only the provided context.\n", + "\n", + "You are to generate {n_questions} questions which should be provided in the following format:\n", + "\n", + "1. QUESTION #1\n", + "2. QUESTION #2\n", + "...\n", + "\n", + "Context:\n", + "{context}\n", + "\"\"\"\n", + "\n", + "qa_prompt_template = ChatPromptTemplate.from_template(qa_prompt)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "u87Izpgm6_fk" + }, + "source": [ + "We'll create a simple chain to query the LLM!" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "id": "ggl9SSjiNbpG" + }, + "outputs": [], + "source": [ + "question_generation_chain = qa_prompt_template | qa_chat_model" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4duvHirh7DQv" + }, + "source": [ + "There's a lot going on in this function - let's take a deeper look:\n", + "\n", + "1. First, we provide a list of documents and a number of questions\n", + "2. We, for each document in our list, generate `n_questions` of questions.\n", + "3. We then associate those questions and contexts via a `UUID`.\n", + "\n", + "> NOTE: The reason we're doing this `UUID` association is for ease of use later in the notebook." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "1Lm2JvgC9X37" + }, + "source": [ + "##### πŸ—οΈ Activity #1:\n", + "\n", + "**DONE** Vibe Coded :)\n", + "\n", + "We have:\n", + "\n", + "- Lists of `Documents` with the `metadata` field `id`.\n", + "\n", + "We need:\n", + "\n", + "- An object with key `id`, which have values `str` questions.\n", + "- An object with key `question_id`, which have values `List(str)` which will be a list of associated `context_id`.\n", + "\n", + "An Example:\n", + "\n", + "question_object:\n", + "```python\n", + "{\n", + "'b4b95fb6-f827-4454-aa5b-20e62733f172': 'What types of accessible formats are available for persons with disabilities?',\n", + "'df58ee4f-714c-419e-8324-94e5870574e2': 'How do accessible formats benefit persons with disabilities?',\n", + "'505fce8b-0e56-48de-a251-61027e396918': 'What are some of the risks associated with the increasing capabilities of AI systems that generate synthetic content?',\n", + "'8ff0ab33-60dc-4fee-8958-91bfb686aca8': 'Why is it important for providers of AI systems to embed technical solutions for marking and detecting synthetic content?'\n", + "}\n", + " ```\n", + "\n", + " context_object:\n", + " ```python\n", + "{\n", + "'b4b95fb6-f827-4454-aa5b-20e62733f172': ['dd75bf94-75f3-4603-8e4b-5522f6925638'],\n", + "'df58ee4f-714c-419e-8324-94e5870574e2': ['dd75bf94-75f3-4603-8e4b-5522f6925638'],\n", + "'505fce8b-0e56-48de-a251-61027e396918': ['ffe3893f-688c-48e8-90bd-7a9feb953d90'],\n", + "'8ff0ab33-60dc-4fee-8958-91bfb686aca8': ['ffe3893f-688c-48e8-90bd-7a9feb953d90'],\n", + "}\n", + " ```\n", + "\n", + " As you can see, a piece of context can be associated with more than 1 question.\n", + "\n", + " The task is to write the Python function(s) to accomplish this task.\n", + "\n", + " Your function signature is provided below, along with the desired return values.\n", + "\n", + " > NOTE: You can make any modifications that you desire - assuming that you have the correct input and outputs." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "id": "U4yi4NfTCnLc" + }, + "outputs": [], + "source": [ + "import tqdm\n", + "import asyncio\n", + "import re # Import regex for parsing questions\n", + "\n", + "\n", + "\"\"\"\n", + "Sample Usage of TQDM:\n", + "\n", + "for i in tqdm.tqdm(range(10)):\n", + " time.sleep(1)\n", + "\"\"\"\n", + "\n", + "async def create_questions_old(documents, n_questions):\n", + "\n", + " questions = {}\n", + " relevant_docs = {}\n", + "\n", + " ### YOUR CODE HERE\n", + "\n", + " return questions, relevant_docs\n", + "\n", + "\n", + "async def create_questions(documents, n_questions):\n", + " questions = {}\n", + " relevant_docs = {}\n", + "\n", + " tasks = []\n", + " # Prepare async tasks for each document\n", + " for document in documents:\n", + " context = document.page_content\n", + " doc_id = document.metadata[\"id\"]\n", + " # Create a coroutine for each document processing\n", + " task = process_document(context, doc_id, n_questions, question_generation_chain)\n", + " tasks.append(task)\n", + "\n", + " # Run tasks concurrently with progress bar\n", + " results = []\n", + " for future in tqdm.tqdm(asyncio.as_completed(tasks), total=len(tasks), desc=\"Generating Questions\"):\n", + " try:\n", + " result = await future\n", + " if result:\n", + " results.append(result)\n", + " except Exception as e:\n", + " print(f\"Error processing document: {e}\") # Basic error logging\n", + "\n", + " # Process results to populate the dictionaries\n", + " for q_list, doc_id in results:\n", + " for question_text in q_list:\n", + " question_id = str(uuid.uuid4())\n", + " # Add the check for collisions\n", + " while question_id in questions:\n", + " question_id = str(uuid.uuid4()) # Regenerate if collision\n", + " questions[question_id] = question_text\n", + " relevant_docs[question_id] = [doc_id] # Store doc_id in a list as per example\n", + "\n", + " return questions, relevant_docs\n", + "\n", + "async def process_document(context, doc_id, n_questions, chain):\n", + " \"\"\"Helper coroutine to process a single document.\"\"\"\n", + " # Invoke the LLM chain to generate questions\n", + " response = await chain.ainvoke({\"context\": context, \"n_questions\": n_questions})\n", + "\n", + " # Basic parsing assuming \"1. QUESTION\\n2. QUESTION\\n...\" format\n", + " # Use regex to find lines starting with number and dot\n", + " # Adjust regex if the format is slightly different\n", + " parsed_questions = re.findall(r\"^\\d+\\.\\s*(.*)\", response.content, re.MULTILINE)\n", + "\n", + " # Fallback or alternative parsing if needed\n", + " if not parsed_questions:\n", + " # Try splitting by newline if regex fails (less robust)\n", + " parsed_questions = [q.strip() for q in response.content.strip().split('\\n') if q.strip()]\n", + " # Filter out potential non-question lines if necessary (heuristic)\n", + " parsed_questions = [q for q in parsed_questions if len(q) > 10 and '?' in q] # Example filter\n", + "\n", + " # Ensure we don't exceed n_questions, even if LLM gave more/less\n", + " # Or handle cases where fewer than n_questions were generated\n", + " final_questions = parsed_questions[:n_questions]\n", + "\n", + " if not final_questions:\n", + " print(f\"Warning: No questions parsed for doc_id {doc_id}. Raw response: {response.content[:100]}...\")\n", + " return None # Return None if no questions could be parsed\n", + "\n", + " return final_questions, doc_id" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "5W0eWOUo4QGL" + }, + "source": [ + "### REMOVE `await` IF NOT USING ASYNC (HINT: Use `async`)" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "id": "85Dq6KRqEs0F", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "fd471165-b7aa-4ff0-b447-9abbe8ba169f" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "Generating Questions: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1781/1781 [05:27<00:00, 5.44it/s]\n" + ] + } + ], + "source": [ + "training_questions, training_relevant_contexts = await create_questions(training_split_documents, 2)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "_FSTG0bb7w73" + }, + "source": [ + "We'll use the function to generate training, validation, and test data." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "id": "eIZm4CqGVzBx", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "5adf3d16-015b-4c5a-fb6e-bd38b71b5b68" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "Generating Questions: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 240/240 [00:04<00:00, 49.19it/s]\n" + ] + } + ], + "source": [ + "val_questions, val_relevant_contexts = await create_questions(val_split_documents, 2)" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "id": "o6qUHg9sV2_y", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "dd5d2885-fc36-40db-cea1-e549247c4cf1" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "Generating Questions: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 240/240 [00:10<00:00, 22.09it/s]\n" + ] + } + ], + "source": [ + "test_questions, test_relevant_contexts = await create_questions(test_split_documents, 2)" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": { + "id": "wqRuTct_XkWf", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "ac36d2b1-304f-4544-a075-c0ce3b2c91c6" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "{'0900b2ab-f4fb-48b0-8fbe-c4a9ab0bb383': 'What is the href link associated with the \"People\" section in the desktop sub-navigation? ',\n", + " '91359428-54f3-4e22-bc4a-d6b942bc07ae': 'What is the id of the div element present in the provided context?',\n", + " '553139ef-e493-49a2-b7ff-427277c71f47': 'What realization does the narrator experience while dreaming about their parents? ',\n", + " '447d05a1-56bf-467a-bece-239dd873bdf7': 'How does the narrator feel about the possibility of life after death?',\n", + " '7143f1a6-cf2a-42b7-8495-cb723a4ed313': 'Which web browsers have Wayback Machine extensions available according to the context? ',\n", + " '76bd5fe8-8c94-40c6-b1cc-c905ff1a43bb': 'Where can you find the Wayback Machine extension for Microsoft Edge?',\n", + " '13dff714-3f99-48e3-b982-6a8995f1c06b': 'What is the purpose of the baloney detection kit in scientific training? ',\n", + " 'e86bb6b3-422a-46f1-ac47-91b51e61a635': 'What does skeptical thinking help us to construct and understand?',\n", + " 'bcb5042a-8260-4ea1-ab42-3f3d5f6af46f': 'What argument does the tobacco industry use to deny a causal relationship between smoking and cancer? ',\n", + " '6bf48947-5978-4874-97e8-f22638656628': 'Why does science insist on control experiments when studying correlations like those between smoking and cancer or violent TV programs and violent behavior?',\n", + " 'c1a1a2f2-d546-4354-b56f-a43f3c1e8817': 'What percentage of the public believes that AI will lead to better outcomes for patients in health and medicine? ',\n", + " '36cc439b-7f93-4176-a94a-327b52c94a6c': 'In what specific medical application do 65% of Americans support the use of AI?',\n", + " '7efe38d6-4411-49aa-b2ea-911406d45dd6': \"How might exposure to frequent acts of violence on television affect young children's perception of aggression and cruelty? \",\n", + " '747cf78a-7338-4842-8b55-1a63fe37041b': \"What concerns are raised about the impact of watching violent programs on children's memories and behavior?\",\n", + " 'a26aeafe-4ed4-4825-886b-999a8088dbec': 'What are the two possible ways mentioned for paying them? ',\n", + " '88571be3-c9f2-44e0-8b40-5d7d2193e5be': 'Why must death or any other calamity not be considered if the speaker does not do rightly?',\n", + " 'ce4e5bb5-2a5f-4bda-9abb-e584c7ea8ad6': 'What percentage of gross profits must be paid as a royalty fee for the use of Project Gutenbergβ„’ works? ',\n", + " '8c04e14e-ae1b-4cf0-81b0-16027ec97d2d': 'Within how many days must royalty payments be made following the preparation of periodic tax returns?',\n", + " '752a2b6f-fc78-43b5-8850-369db31e1322': 'What are the three categories listed in the media sub-navigation menu? ',\n", + " '71589493-d640-40d5-ad13-1416bafbe2f3': 'Which category link includes the data-event-click-tracking attribute value \"TopNav|SportsVideosVideo\"?',\n", + " 'e8201de6-b6b9-44d1-b7f9-03fe1161f4c3': 'What sentence must appear prominently whenever any copy of a Project Gutenbergβ„’ work is accessed or distributed? ',\n", + " '5f1c943d-5af4-4d31-ba6e-d7ceb24062b2': 'In what situations must the sentence with active links to the full Project Gutenbergβ„’ License be displayed?',\n", + " 'bcc402dd-d277-4535-b172-a0d4a25eb5ce': 'What is the label of the link found within the media-subnav element? ',\n", + " '687b4b72-14f9-4191-bd93-3234559a4693': 'Which menu is associated with the media-subnav that contains the \"Video\" heading?',\n", + " 'a50316c5-b890-4ff8-ad84-9da7065f84b6': 'What types of AI uses tend to receive higher levels of support from Americans? ',\n", + " '1b98748e-f5da-46c7-afe9-ed4f303152e4': 'How do Americans generally feel about AI performing household chores?',\n", + " 'c513db58-415a-404e-9ee9-a6914183626b': 'What role did the ancients believe gods and the belief in punishment after death played in society? ',\n", + " '0b95ac39-b3c8-4d49-ba84-622734612483': 'How does the argument from adverse consequences attempt to justify the existence of God or the guilt of a defendant?',\n", + " '3cbff909-8585-4826-a380-f3453ec2137c': \"What is the author's stance on the evidence required to prove life after death? \",\n", + " '81b5f8aa-20c7-4822-93d0-23c39af3ed52': 'How does the author compare the comforting nature of facts versus fantasies regarding life after death?',\n", + " '937df0de-d2b8-47a7-bd6d-5dcd307c5e3b': 'What are the two main sections linked in the \"more-slider\" navigation menu? ',\n", + " '701008b6-ee59-4aac-8a6d-b091ad32e108': 'Where does the \"Volunteer\" link in the \"more-slider\" navigation menu direct users?',\n", + " '0b85b6b4-0cb6-4be6-9d0d-ecdb9b8993e1': 'What type of input element is used within the search menu for selecting the search option? ',\n", + " '144a6de7-adf9-454c-856f-d9b1c0a5ad33': 'What attribute indicates whether the search menu is expanded or hidden?',\n", + " '7e3265dd-4d63-433d-a104-4cf73d2f95e8': 'Which web browsers have Wayback Machine extensions available according to the context? ',\n", + " '36ed3f70-22fc-4b3c-8225-48e51cf68286': 'Where can you find the Wayback Machine extension for Firefox?',\n", + " 'edbcc2d5-e79f-44b2-b285-3891dbd1e7dd': 'What percentage of Americans would not want to apply for a job with an employer who uses AI to help with hiring decisions? ',\n", + " 'c722b86c-53fd-4694-b947-e2bc0bbe44b8': 'How do Americans generally feel about AI being used by health care providers to help care for them?',\n", + " 'e76733ee-15ed-49f7-b01d-4366ddebe48a': 'What are the terms under which you can copy, give away, or re-use this eBook? ',\n", + " '19e9670b-785b-41ee-8d9e-7abf27e88993': 'What should you do if you are located outside the United States before using this eBook?',\n", + " '89e4175c-2f1d-4f90-b18b-e52e35c543b0': 'What should you do if you have a different opinion? ',\n", + " '5f16e4a4-d6e9-4957-87a5-297fdfd626a2': 'What will happen if you remain of the same mind as before?',\n", + " 'b66deaf1-0a66-4091-b130-ec98693d5011': \"What criticism is being made about the person's approach to raising children? \",\n", + " 'b0010715-930a-4abc-b3fa-fd1cd1019d01': 'How does the speaker feel about themselves and their friends in relation to the situation described?',\n", + " 'fdc68a7b-9ffd-477a-9900-46882d529bf7': 'What reasons does Socrates give for needing to be guided by reason rather than zeal? ',\n", + " '740ed911-9fab-4636-83b3-381480efe610': 'Why does Socrates refuse to repudiate the principles he has always honored and revered?',\n", + " '9b0f9eb3-2229-488e-afed-6abc88e406df': 'What are the different categories of software links mentioned in the media subnav? ',\n", + " 'f36d0b2d-12b5-4d2c-87be-b32fb3a89b88': 'Which URL corresponds to the \"CD-ROM Software Library\" link?',\n", + " '3fccc7f2-5650-406e-9611-2fb7d9c60d4e': 'What are the three types of software categories listed in the media sub-navigation?',\n", + " '97b3d569-2016-4623-a0ae-966d655abaea': 'Which software category link includes the data-event-click-tracking attribute \"TopNav|APKSoftware\"?',\n", + " 'f50ab521-dbdc-4210-98e5-1132087dffd1': 'What are the key numerical values and units mentioned in the given context? ',\n", + " 'a31f2d2a-3495-43d2-a20c-9d7e5967ccac': 'How do the different numerical sequences in the context relate to each other in terms of structure or pattern?',\n", + " '54b00822-98c2-4de2-a51f-334804d22225': 'Why does Socrates believe that only the opinion of good men is worth considering? ',\n", + " '1b28d9bf-e209-4ce2-ad2b-052b224183c8': \"According to Socrates, why can't the opinion of the many cause the greatest good or evil?\",\n", + " '9db84222-8694-4d5b-b9c7-aec113ed1177': 'What option is available for searching archived web sites in the navigation menu? ',\n", + " '5e04b6aa-88f3-497a-a64c-2b456f978f49': 'Where does the \"Advanced Search\" link in the navigation menu direct the user?',\n", + " '4397f817-4a33-4903-a1c6-c799dd693456': 'What concern does Crito express about the consequences of Socrates escaping from prison? ',\n", + " '6ca4918d-e9e9-4d9b-9a61-c35436dcf229': \"How does Socrates respond to Crito's fear regarding the risks to their friends if he escapes?\",\n", + " '96c002d7-9193-4abb-8779-2c7011883618': 'What plan does Crito propose to Socrates to prevent his impending death? ',\n", + " 'd73f6e34-5abf-4bef-a43b-c78aa6296892': \"How does Crito justify the urgency and necessity of Socrates' escape?\",\n", + " '89b90904-fba7-4937-9f93-3bfc7174b6be': 'According to T. H. Huxley, what is the foundation of morality? ',\n", + " '99bcab72-56e1-46c2-838f-d6af28e952db': 'How do the ideas of Clement, Hume, Paine, and Huxley extend beyond religion to apply to commercial civilization?',\n", + " 'c930ffc9-1f82-441e-8c91-1d23cb71dca5': 'How does a quantum computer differ from a regular computer in exploring possible solutions? ',\n", + " '2121fbb0-f6c0-405f-bf96-530ec199e382': 'What analogy is used to describe the relationship between the economy and interest rates?',\n", + " 'bbcf2042-1dbe-4f38-9364-a13be91dec9d': 'What are the two main actions a user can take from the signed-out dropdown menu?',\n", + " 'd5ab4e0f-a147-43b6-b36c-d887931a1c64': 'What are the URLs associated with the \"Sign up for free\" and \"Log in\" options?',\n", + " 'b1dd8145-1356-48ff-8750-08858eb56c0a': 'What are the numerical values and variables mentioned in the given context related to?',\n", + " '541d6b55-c87b-4dfb-beb4-48e44e1c1514': 'How do the different numerical sequences in the context interact or relate to each other?',\n", + " 'ed591224-ff9f-4ea7-856d-6ad9736d87ca': 'What are the consequences mentioned if he breaks agreements and returns evil for evil? ',\n", + " 'f48cddeb-391c-48b7-99d1-3bfd5cec983d': 'According to the passage, what should he prioritize before thinking of life and children?',\n", + " '952324ee-9a64-4d9f-a97a-6b42ffb85852': 'What happens if a disclaimer or limitation in the agreement violates the law of the applicable state? ',\n", + " '33cda2ed-946a-42a7-852f-9c30f2f22500': 'Does the invalidity of any provision in the agreement affect the enforceability of the remaining provisions?',\n", + " '979d6378-7b6a-4eaa-aea5-72e631433d06': 'Why is he in Thessaly? ',\n", + " '3ab9e434-4749-4962-ac99-3e00abcce4fa': 'Will true friends care for someone equally whether they are alive or dead?',\n", + " 'fd40cada-5687-4733-b5e2-b7ffbabbaa2b': 'What does the phrase \"incidental damages\" refer to in this context? ',\n", + " 'ae359772-feee-425b-b75e-bca4a6f84bfa': 'How does giving notice of the possibility of incidental damages affect liability?',\n", + " '340ce79a-4792-45fb-96dc-4d74e26e8568': 'What role do euphemisms play in political language according to the context provided? ',\n", + " '3c8f409c-010a-42b6-b502-ae247050218f': 'How can the \"baloney detection kit\" be both misused and beneficial in evaluating arguments?',\n", + " '5b8f11f3-a468-4d5f-87b4-0123d9b6685d': 'What must you agree to comply with when using or distributing a work associated with Project Gutenberg? ',\n", + " '0887146a-edd2-454e-b662-0dc0131cc2da': 'Where can the Full Project Gutenberg License be found?',\n", + " '3f58d1a3-9d75-44df-b5be-eacc1cf561a5': 'What is the role of the element in the provided context? ',\n", + " 'aa1fa272-a00f-49c2-8510-bc2798a3eea2': 'How is the navigation structure organized within the component?',\n", + " 'b48df97d-17e9-4327-aafe-f768892d34c5': 'What is the height and width specified in the SVG element?',\n", + " '31f801ac-3091-45d2-bcb5-39d2c6ac7d50': 'Which XML namespace is used in the SVG tag?',\n", + " 'c41df411-b6ab-4221-8947-0385d62184be': 'Within how many days must electronic work be discovered and reported to you after receipt? ',\n", + " 'c47e4dc4-6c9f-4efa-9f71-f69a230f250f': 'What must you comply with in order to freely distribute Project Gutenbergβ„’ works?',\n", + " '21c5f20a-4609-49eb-ba0d-ddb7b52b50b9': \"What argument does the law present to Socrates regarding his obligation to abide by the state's sentence? \",\n", + " 'f016be98-fc3b-4a8a-98f2-410eb618d601': \"How does the law justify its role in Socrates' existence and upbringing?\",\n", + " '23af5c05-5f42-4c43-ad2e-9be731340ff9': 'What are the two main categories listed in the media sub-navigation menu? ',\n", + " 'a5d278c6-58d7-4ac1-a598-b773448fa0eb': 'Which section includes a link related to the topic \"Understanding 9/11\"?',\n", + " 'd78cc6ae-8eb1-4cd8-b17b-6c7655cbc53e': 'What reasons does Crito give to persuade Socrates not to worry about the cost of escaping from prison? ',\n", + " '0c98db29-fc86-47c0-8e12-46cd2345e26d': 'Who are the individuals mentioned by Crito that are willing to help Socrates escape, and what have they done to show their support?',\n", + " 'ab6f9f26-17fa-44f8-bdf2-fe47aea6d334': 'Why is it important to allow inveterate skeptics to follow your reasoning and duplicate your experiments? ',\n", + " 'abc919d9-8adc-4f8d-902c-bd82fb2b4f08': \"According to the context, why can't argumentation alone suffice for the discovery of new knowledge?\",\n", + " 'b2ff8242-d1d7-423b-babe-e1b873ba399f': 'What are some notable features of Thessaly?',\n", + " '81ef58dd-0662-42c0-9aeb-f0d693f239c6': 'How does Thessaly compare to other places mentioned?',\n", + " '776b2e07-6b8e-4e3c-827b-e7e078608d7c': 'Under what conditions can you charge a fee for access to or distribution of Project Gutenbergβ„’ works according to paragraph 1.E.7? ',\n", + " '8bad69c7-9be8-4e3a-bd73-f3a865b52372': 'What does paragraph 1.E.8 specify about charging a reasonable fee for copies or access to Project Gutenbergβ„’ electronic works?',\n", + " '349b3246-4d67-4f68-b6b8-7a0a56602315': 'Why does the speaker believe Socrates should not betray his own life?',\n", + " '6b1148ba-4412-466d-a93b-c90f8f523d3a': 'What reasons does the speaker give for Socrates to leave Athens and go to Thessaly?',\n", + " '15def61a-dc4c-4dc0-a4c1-4b2b1a29b70a': 'What percentage of Americans believe AI-powered driverless vehicles should be held to a higher testing standard than other passenger vehicles? ',\n", + " '8be3d8fb-5126-48b6-9484-9ac8e76ab4ee': 'How do Americans feel about the pace of AI adoption in health care, according to the context?',\n", + " 'a457cb72-92a4-4e67-a152-6da71ddc6fba': 'What was the original argument about the opinions of men that is being reconsidered in the context? ',\n", + " '3d5a4c75-e8c0-4617-8c25-a93f16374995': 'How does the speaker propose to evaluate whether the previous argument remains valid under their current circumstances?',\n", + " '28a32580-4b08-4a36-b15b-8214890a4fed': 'What type of graphical element is described in the provided context?',\n", + " '5a0f2e13-9e71-4fba-8ceb-dc12899679f6': 'Which attributes are specified for the path element in the given context?',\n", + " '25fce1ee-503a-49dd-85b7-cc276586c415': 'What is the name of the iOS app mentioned in the context? ',\n", + " '1365d6dc-0f3c-4ccc-8f2b-6462a42c8eed': 'Where can the Wayback Machine iOS app be downloaded from according to the context?',\n", + " '08565463-92b5-4143-8fc4-9d7adfdde80d': 'How does the example of President Dwight Eisenhower illustrate a common misunderstanding of statistics? ',\n", + " 'dc4a3c4a-1c8d-426b-9384-72bae984ac2e': 'What inconsistency is highlighted by the contrasting attitudes toward military threats and scientific projections on environmental dangers?',\n", + " '25c58d8c-6925-4db4-86c3-1d72396a716d': 'What is the tax status of the Project Gutenberg Literary Archive Foundation? ',\n", + " 'f08ffbce-3816-4a9b-b516-abfb5aac8f46': 'Where is the business office of the Project Gutenberg Literary Archive Foundation located?',\n", + " '565c39a9-44de-40c5-9224-0ee25e944b7c': 'What consequences does the speaker suggest might come from violating sacred laws out of a desire for a little more life? ',\n", + " 'aff5e59c-7c82-4e36-bdec-432fdb9f3241': 'How does the speaker question the value of living abroad in Thessaly in relation to justice, virtue, and raising children?',\n", + " 'ade00a9d-4a18-42fa-88b9-8cdd6d8b03f7': 'Why are control experiments important when testing the effectiveness of a new medicine? ',\n", + " 'e365a8db-a8b1-4a2b-a142-5656ddf8063f': 'How can you determine whether an acupressure bracelet or a pill is responsible for relieving seasickness?',\n", + " 'e26f0f8d-7870-453d-93b6-94ccf1ca8c44': 'What is the most commonly cited reason for concern among people who are more worried than excited about AI? ',\n", + " '40b2147e-88a9-4b0f-9831-347c28f82ffa': 'How do Americans generally feel about the use of personal information by companies employing AI?',\n", + " 'b30b0c64-8c93-4fce-9be6-33460bfbf74c': 'What are the three categories listed in the media sub-navigation menu? ',\n", + " '4c1e60bd-cc83-49e6-94b1-0650bf257cce': 'Which category link includes the data-event-click-tracking attribute value \"TopNav|VideogameVideosVideo\"?',\n", + " '7fd6ed8b-4398-42eb-bb3f-9cf16ce2c702': 'What argument does Socrates present regarding the power and authority of the laws in a state? ',\n", + " '3f7935ac-3914-4a7c-88c8-b69ebe72585d': 'How does Socrates suggest one should respond to accusations of undermining the laws and the state?',\n", + " '18a90423-a360-453c-9d0e-daf88d930a0e': 'What urgent action is being urged to be done \"this very night\" according to the speaker? ',\n", + " '3560da1c-e9ce-4996-a312-4a7dc538aff2': 'How does the speaker describe the consequences if the action is not taken promptly?',\n", + " '6061d56d-6461-439f-a823-d0655f14b057': 'What are the three categories of films listed in the media sub-navigation? ',\n", + " '8082ef73-8f00-4045-93a0-acb95de8a495': 'Which category link includes the data-event-click-tracking attribute value \"TopNav|EphemeralFilmsVideo\"?',\n", + " 'be4039e8-6745-48eb-ac7a-4637f1fb1ff3': 'What types of computers are supported by the electronic works distributed by Project Gutenbergβ„’? ',\n", + " '34ee71f6-cc76-4d37-8a22-1d1df20fdbf8': 'How does Project Gutenbergβ„’ sustain its operations and continue distributing electronic works?',\n", + " '2bbcb968-285e-4ee5-a371-106c8f9bcbff': 'What is the purpose of the Project Gutenberg Literary Archive Foundation created in 2001? ',\n", + " '4f64201b-8be2-4ef4-91f6-f8aadb1b29dc': 'Where can one find more information about how to support the Project Gutenberg Literary Archive Foundation?',\n", + " '08066f14-b8e6-4b1d-b5f0-6285616db43a': \"What value is assigned to the 'cache_bust' property in the vs[i] object? \",\n", + " '42a9dfa6-5fef-4e97-b0f8-30a5954975a1': \"Which server name is specified in the 'server_name' property of the vs[i] object?\",\n", + " '0e1d5e07-653a-4ba6-9056-9491967bbe3c': 'What were the reasons behind the charge that Socrates was not a good citizen? ',\n", + " '86aaef1e-830b-4a9d-9354-c79a80b57c6d': 'How did Plato respond to the accusations against Socrates regarding his citizenship?',\n", + " '0729e320-3343-40e9-af5a-49c3e163c95f': 'What is the link associated with the \"American Libraries\" section in the navigation menu? ',\n", + " '89e68787-9dc1-466d-b620-efc018d2905a': 'Under the \"Featured\" category, which text collection is listed in the provided context?',\n", + " '9a86602d-f07e-4e02-ab15-46eec2e38124': 'What percentage of American adults believe they have made contact with the dead, according to the context? ',\n", + " '8c7f92c9-f1fb-4e9c-bc7e-9da0904df0fb': 'Why does the author express skepticism towards mediums who claim to channel spirits?',\n", + " '803adf9c-16f5-4d5d-b5a2-a8c188be475f': 'What file and line number does the session_start event occur in according to the context? ',\n", + " '2980269f-5f40-4837-a009-dfebbcc3012d': 'Which function is called immediately after main_wrap in the bug dump trace?',\n", + " 'f9846830-b170-4934-9bf2-3d70f5cd86f7': 'What percentage of Americans familiar with chatbots like ChatGPT are more concerned that the government will not go far enough in regulating their use? ',\n", + " 'a3ba3a50-78ed-488b-bd4c-3d54b3e4dbfc': 'How do Americans generally feel about regulation and oversight of emerging AI technologies such as chatbots and driverless vehicles?',\n", + " 'bd9d54e1-f02c-4b86-913f-7c8333a57492': 'What are the three video categories listed in the media sub-navigation menu? ',\n", + " '4097d6a3-d34a-425b-bc5a-34c25ebe0ee1': 'Which video category link includes the data-event-click-tracking attribute \"TopNav|PrelingerArchivesVideo\"?',\n", + " '19d02dd0-4102-41ac-b3d3-2a1a09e263b8': 'What is the advantage of considering multiple working hypotheses when trying to explain something? ',\n", + " '47458f24-8e2f-41d5-8c1a-ef7bf65f7c37': 'Why should you avoid becoming overly attached to your own hypothesis?',\n", + " '3814b2e0-21d9-49e0-b427-04464cbaabd5': 'What is the href link associated with the \"Youth Media\" menu item? ',\n", + " '52d5f4ca-be31-4f90-a1f3-f6fb7f5776f5': 'How many pages are archived according to the wayback-search element?',\n", + " 'fb22329b-c547-4396-9c7a-86ed86e779d6': 'What are the three main navigation links mentioned in the provided context? ',\n", + " 'be6c0e28-f5c8-4e38-932b-7439a0a28693': 'Where does the \"Blog\" link in the navigation menu direct users to?',\n", + " '648e89ca-c63f-4fe9-a022-4be5b76fb59a': 'What are the two main topics listed in the media sub-navigation links? ',\n", + " '079574a0-5660-46b0-8d77-7b35a68a47eb': 'Which link in the media sub-navigation is associated with the \"TopNav|OccupyWallStreetVideo\" click tracking event?',\n", + " 'a40ddbac-f4c1-494b-99af-b5cdccc75b1b': 'What is the main concern expressed about the care and education of the children if the parent is absent? ',\n", + " '6bda8835-837e-4054-bb9b-6c03617dffed': \"How does the speaker view the role of friends in caring for the children in the parent's absence?\",\n", + " '39d6cc09-edf4-4173-9a0d-9f0bd32950e4': 'What parties are you agreeing to indemnify and hold harmless under the indemnity clause of the Project Gutenbergβ„’ agreement? ',\n", + " '60744c0b-94fe-4046-89d8-300d697fc13a': 'What actions or occurrences can trigger your liability under the indemnity provision in the Project Gutenbergβ„’ agreement?',\n", + " '64572b86-d252-460b-9070-ed4229e9ba6f': 'What are the numerical values and variables mentioned in the given context? ',\n", + " 'a32b51cb-6bf8-4c94-98d7-0177f9d58cb2': 'How are the different numerical sequences and symbols structured within the provided text?',\n", + " '103d5e58-2bd9-46da-96d2-2b7f3e42cae8': 'What is the significance of the voice Socrates hears, and how does it influence his actions? ',\n", + " '67b162a2-bb09-4ef8-a116-fc0addf8e46e': 'How does Socrates respond to Crito’s silence, and what does this reveal about his mindset?',\n", + " '2624da0e-db62-4d6d-8d82-001c11c3763d': 'According to Socrates, whose opinion should a person follow when deciding how to act and live? ',\n", + " '0e8381f8-78aa-46c8-9593-359e1261479c': 'What consequence does Socrates say will happen to a person who disobeys the opinion of the one with understanding and instead follows the opinion of the many?',\n", + " 'caf33238-4fbd-4658-bf05-3ee7daa9d353': 'What steps must you take to receive a refund if you discover a defect in the electronic work within 90 days of receiving it? ',\n", + " '7d09dad5-11c8-4b83-8629-2461da36e1e3': 'What options does the provider have if you report a defect in the electronic work you received?',\n", + " 'f8caf7a1-8434-4465-9d5f-50eabcc87919': 'What questions are raised about the clothing and manufacturing methods of the people mentioned in the context? ',\n", + " 'f20e925a-3c73-40a4-a3b7-c15b04ee1444': 'What skepticism is expressed regarding the claims about Ramtha and the information provided by non-human entities?',\n", + " '10fed1a3-4a0c-4190-8fe6-5fe3063f5b12': 'What is the starting point of the petabox process as indicated in the context? ',\n", + " '53bddc22-5dbb-41aa-bf69-2fdba976df64': 'Which function calls are involved in the redis_read process according to the provided trace?',\n", + " 'fa5c3a0b-c9a2-48c1-bc01-00cf4a7c39af': 'What is more important than liking the conclusion when evaluating an argument? ',\n", + " '41aadc5f-4b38-452f-aafa-2374f54f80f9': 'Why do arguments from authority carry little weight in scientific reasoning?',\n", + " 'de74a4cb-7845-4bbb-be85-8fb008e0894a': 'According to David Hume, why do men make a merit of implicit faith in doctrinaire religions? ',\n", + " '7ca6c52f-574a-4395-93dd-6debe06e1848': 'What moral consequences of infidelity does Tom Paine highlight in The Age of Reason?',\n", + " '68823012-d535-49a0-9dc9-5ba1d4ae9e95': 'What types of \"Defects\" might be present in Project Gutenbergβ„’ electronic works despite the efforts of volunteers and employees? ',\n", + " '871ce486-5ca9-44ba-81bf-2a9177fbdbd7': 'What steps do Project Gutenbergβ„’ volunteers and employees take to create the Project Gutenbergβ„’ collection?',\n", + " '31adc542-3087-491f-8033-ae1ac02346fe': 'What is the sequence of function calls leading up to the execution of the pipeline in the provided context? ',\n", + " 'b73a0d1b-4bdd-4851-b844-8c757512d189': 'At which file and line number does the redis_read finish event occur according to the context?',\n", + " '63545b2f-85ef-466f-8ed9-c7480ffdb91d': 'What does Crito mean by saying \"Very good, Socrates\"? ',\n", + " '9ed6b92c-2eeb-4dc7-a933-f32876313959': 'In what situation might Crito say \"Very good, Socrates\"?',\n", + " '6b32e99f-ece1-4e8c-b6e1-3f6853a14b48': 'What does the phrase \"they cannot make a man wise or foolish\" suggest about the nature of evil? ',\n", + " '3c4da9ce-461d-4058-9f99-ea56fe5630a5': 'How does the speaker describe evil in his own words?',\n", + " '691eb914-5a0f-4e6b-9cd2-7b583303e408': 'What is meant by observational selection and how does it relate to the concept of counting hits while forgetting misses? ',\n", + " '70b58e9d-c036-4929-b339-73b49c120e51': 'How do the statistics of small numbers contribute to misconceptions or faulty reasoning, as illustrated by the examples given?',\n", + " '819dee2f-2aca-4303-ad3f-75eb045d4087': 'According to the passage, why is the individual considered both a child and a slave to those who nurtured and educated him? ',\n", + " '861710ba-15a8-493e-9f5f-f921413b6543': 'What argument is made about the right to retaliate against a father or master who has caused harm?',\n", + " '48c86092-38a1-481c-9fab-1a8e628187a1': 'What type of data or code is represented in the given context?',\n", + " '0dd076fc-fb52-4108-8c76-f7203ca5cdfe': 'How might the numerical values in the context relate to graphical or path elements?',\n", + " 'e534e73e-fe49-49ab-a9ce-e21d5da94ddc': 'What is the argument for not setting aside the law mentioned in the context? ',\n", + " 'd92cace0-f5b4-4c1e-8779-236f998702a2': 'How does the speaker propose responding to the claim that the state has given an unjust sentence?',\n", + " '0c9d75e1-c6e8-4747-89f2-881aa2071a34': 'What methods do \"psychic surgeons\" in the Philippines use to convince cancer victims of their healing abilities? ',\n", + " '86d6f6e1-0239-431a-9bc0-3fec47b9b79c': 'Why do police consult ESP experts despite their predictions being no better than common sense?',\n", + " 'fea16e0c-73f8-4661-b382-4b6d7c1cefdc': 'Are unsolicited donations from states where solicitation requirements have not been met accepted? ',\n", + " 'a1149589-4c37-4a42-8611-6706ebdfee0d': 'What types of donation methods does Project Gutenberg accept?',\n", + " 'c35ab4b4-7d34-4e2e-b677-f550760ac086': 'What can you do if the second copy is also defective? ',\n", + " '1f67e4b8-66a2-4d19-9443-be980b6b45eb': 'How should you request a refund if the product remains defective after the second copy?',\n", + " '0aa9ffaa-b834-40e3-8416-eb225214eb08': 'What are the different sections available in the navigation menu based on the provided context? ',\n", + " '07064fc2-c07e-47eb-87a7-904e92191316': 'Which URL would you visit to access the blog according to the navigation links shown?',\n", + " 'd2a982e6-c81e-4a6d-9b87-1b811b2965c5': 'What is the significance of the tag in an SVG file? ',\n", + " '27bf6f5b-4d28-49e6-aa23-5292f6e7d6d4': 'How does the tag affect the rendering of SVG graphics?',\n", + " 'd88c8ff2-d984-4936-ae92-d2c2e49a1ec6': 'Why does Socrates believe that the opinion of the many should not influence his decision to escape? ',\n", + " 'f3a60a20-d0a3-4d37-b843-24d866013e5b': 'What reasons does Socrates give for valuing a just and honorable life over concerns about reputation or harm to his children?',\n", + " 'd6965041-eeed-4a81-9f38-32c6546d18c6': \"What aspect of Socrates' character does the Crito primarily aim to portray? \",\n", + " 'bf2a8e8b-1afc-4598-9e0d-b26d97d3aa5e': 'How is Socrates depicted in relation to the laws of the state in the Crito?',\n", + " 'dd8b95b8-67b5-458f-ae81-d737fa348cf6': 'What reasons does Socrates give for abiding by the covenants and agreements he made with the state? ',\n", + " '64e742c0-afe2-4d75-9ccb-14a594dd3beb': 'Why does Socrates mention Lacedaemon and Crete in his argument about staying in Athens?',\n", + " '8d24f023-2ec9-4f1f-a5cd-f3c50a64c81d': 'What are the numerical values and variables mentioned in the given context? ',\n", + " 'b15ef971-d54c-4c22-9b2c-4fc3bba29b8c': 'How do the different numerical sequences relate to each other within the context provided?',\n", + " '498228a8-006e-4d0e-abd0-df4e5ab3ca12': 'What are the three software categories listed in the media sub-navigation menu? ',\n", + " '5a0e1f6c-e2df-455a-8b2c-1342dde8fe88': 'Which software category link includes \"ZX Spectrum\" in its URL?',\n", + " 'f0ebd306-1cc9-4845-9628-f56163c73664': 'What is the purpose of the input field labeled \"Search the Wayback Machine\"? ',\n", + " 'b44363cb-e080-45e8-b20a-056ef021cbb3': 'What does the SVG element with the title \"Search icon\" represent in the context?',\n", + " '1f17967f-a097-4486-a907-fc5b9167596f': 'What are some reasons an item might be taken down according to the provided context? ',\n", + " 'a24bbac0-b2e0-4b4e-acde-7b3286828bd7': 'What issue is preventing the requested item from being displayed on the page?',\n", + " '9b080c0a-08d4-445e-9f04-a26a8f0907c0': 'What tactic do the aspirin commercials use to promote their product over competing products? ',\n", + " 'edd232d6-11d0-4e83-a273-95415d7bb14f': 'Why might the claims in these aspirin commercials be considered misleading or incomplete?',\n", + " '2f4f614f-9a2f-4607-95af-265571d4ad43': 'What restrictions are placed on the use of the \"Project Gutenberg\" trademark according to the agreement? ',\n", + " '7ed3f6b3-713b-41d4-99dc-027736bedebc': 'Where can you find information about what you can do with Project Gutenberg electronic works without fully complying with the agreement?',\n", + " '6ffcc666-bf3b-41d2-abf0-906163be14e1': \"What is the date range specified in the hidden input field's value attribute? \",\n", + " '13894c46-eafe-4654-8351-da6ad7fe4926': 'What is the class name of the hidden input element in the provided context?',\n", + " '214de7c5-6a1e-451b-920a-9c5ce380c6af': 'Why do many Americans hesitate to apply for jobs that use AI in the hiring process? ',\n", + " 'a93f40eb-ae4e-4ec3-8471-cb519b0fb121': 'How do most Americans believe increased reliance on AI in health care would affect patient-provider relationships?',\n", + " '265c92d0-820f-49b4-9962-ea89c9dca37b': 'How do U.S. adults who recognize racial and ethnic bias in health and medicine generally feel about the impact of AI on this issue? ',\n", + " '74392859-6b56-402c-82a6-801c58ec85f2': \"What is the public opinion among those who see bias in hiring regarding AI's effect on racial or ethnic bias in job applications?\",\n", + " '3bfaafaa-e557-4bc3-ad2b-a224fd9c4a2c': 'What literary devices are highlighted in the dialogue according to the provided context? ',\n", + " '5457f5fe-2351-4d98-8b33-b132d2b59d14': 'Who are the main characters involved in the dialogue, and where does the scene take place?',\n", + " '09225f8a-75a0-48a8-91dd-abda29237966': 'What is the significance of the values and coordinates mentioned in the context, such as \"5.1294029-6.0570166\" and \"-73.7580006-5.70016816\"?',\n", + " 'e3bd5737-0bee-4fb0-9645-c02cc36bfc80': 'How do the numerical sequences and measurements relate to the overall structure or purpose described in the context?',\n", + " 'c25e4174-9d4e-48d0-ad72-9167f67ebb71': 'What was the corporate objective stated in the 1971 internal report of the Brown and Williamson Tobacco Corporation regarding public beliefs about cigarette smoking? ',\n", + " '7f390cc3-3400-4722-9564-c013729959b1': 'How did the Brown and Williamson Tobacco Corporation describe the attacks against cigarettes in their 1971 internal report?',\n", + " '597f3825-fa1c-45e8-963a-cbe4ddff345f': 'What are the different sections available in the desktop sub-navigation menu?',\n", + " '9155f0c8-15a0-42ba-897a-8fef1ab3615b': 'Where can you find information about volunteer positions based on the navigation links?',\n", + " '0a1f8583-e9dd-4d87-a2e8-d47bfadf4d2a': 'What unethical practice did senior executives commit in 1994 regarding research on inhalable cigarette particles? ',\n", + " '0a692158-4e19-436b-be96-e87b6c79db2b': 'How does the tobacco industry reportedly handle research findings that show their products to be more dangerous than publicly declared?',\n", + " '244f0a17-c033-4a1b-8ce7-3e96730f8643': 'What logical fallacy is illustrated by the statement that UFOs must exist because there is no compelling evidence that they do not visit Earth? ',\n", + " '81c0e868-16a1-4caa-9376-3927aaf55250': 'How does the phrase \"absence of evidence is not evidence of absence\" relate to the arguments about intelligent life elsewhere in the Universe?',\n", + " 'd9a5dea5-032d-4df1-b365-4e368f658bfb': 'What does the term \"non sequitur\" mean, and can you provide an example of this fallacy? ',\n", + " '2d23c0b0-6826-4b2a-b6ef-48c548a6f8aa': 'How is the fallacy \"post hoc, ergo propter hoc\" illustrated in the example involving Jaime Cardinal Sin?',\n", + " 'fef09259-6d6f-4c77-9f1b-4c4a7416f1b9': 'What rights does the Project Gutenberg Literary Archive Foundation claim over the individual works in its collection that are in the public domain in the United States? ',\n", + " 'c7364030-6fc9-4f18-8bda-a6b902f5be19': 'Under what condition does the Project Gutenberg Literary Archive Foundation allow copying, distributing, performing, displaying, or creating derivative works of unprotected individual works in the United States?',\n", + " 'af0efaf8-a840-42b6-85c8-fe20846743f9': 'How does the author express their feelings about the loss of their parents? ',\n", + " '757480fa-9c57-448a-aeef-1d6e031ab71d': 'What does the author wish for in terms of communicating with their deceased parents?',\n", + " 'a8165ad5-1f65-4471-90c9-1777b6910bd8': 'What emotions often make victims susceptible to deception or \"baloney\" according to the context? ',\n", + " 'caac5efb-5dda-4af3-8b82-7e24c3c9ad68': 'Why can the acceptance of baloney be more dangerous than just losing money?',\n", + " 'a9912f5a-244c-4e6d-840e-61760c8b576a': 'What is the primary mission of Project Gutenbergβ„’ as described in the context? ',\n", + " 'de6e9eee-2e93-4961-860c-d70282541dda': 'Why are many small donations important to Project Gutenbergβ„’?',\n", + " '62257d57-bd43-40f1-a442-559668c99e17': 'What types of liability and damages does the Project Gutenberg Literary Archive Foundation disclaim under this agreement? ',\n", + " 'bd62e465-437a-4fdc-b608-2e19e57d2155': 'Under what condition does the agreement provide remedies for negligence, strict liability, breach of warranty, or breach of contract?',\n", + " 'd7e47de5-7639-44c3-823f-7a0e7c3d575f': 'What principles did the individuals agree upon regarding doing evil and returning evil for evil? ',\n", + " '8ffc7f48-ad18-4e22-9b03-c3ec2cbc2e46': \"Why is Crito unable or unwilling to reply when asked if Socrates' escape is consistent with maintaining these principles?\",\n", + " '6a5da1f2-7d0c-4779-8b6b-ef04d86f9ba9': 'Why does the narrator feel that their emotions might make them vulnerable to deception or manipulation? ',\n", + " '1a4d25f4-f436-494c-b5fb-108a85352690': 'What examples does the narrator use to question the credibility of channelers providing verifiable information?',\n", + " 'c47ca189-770e-44fd-9a67-f38ade4eaa4c': 'What must you do if you are redistributing a work with the phrase \"Project Gutenberg\" associated with it? ',\n", + " '346ad38a-5806-4b75-9663-cb3e3e7c340a': 'Under what conditions can you use the Project Gutenbergβ„’ trademark when providing access to a work?',\n", + " 'b8fbce2a-49fa-4b9e-b2b0-12e9082a5367': 'What is the ad hominem fallacy and how is it exemplified in the given context? ',\n", + " '2556c8f6-2a0f-4801-a891-5be531142a64': 'How does the argument from authority fallacy manifest in the example involving President Richard Nixon?',\n", + " '24970f2c-40c3-4b62-9876-195cfb320ee1': 'What must you keep attached to the work when sharing it without charge to comply with the Project Gutenbergβ„’ agreement? ',\n", + " 'a202b5b5-b61a-418c-ae7f-b5a8a7836a06': 'How can you easily comply with the terms of the Project Gutenbergβ„’ agreement when sharing the work?',\n", + " '2d72c3c6-a4f3-484a-bc0f-63501a053e6d': 'What function is called at line 79 in download.php according to the context? ',\n", + " 'd7faf830-0951-4757-97f8-2a6e994e2602': 'Which file and line number is associated with the exitItemNotFetched404 function?',\n", + " 'b233e1f6-67ab-4fbe-bead-c38eda9d51e4': 'What are the two main actions users can take according to the links provided in the context? ',\n", + " '67632c0b-d7d0-4472-b226-fcc161661ef3': 'Where do the \"Learn More\" and \"Build Collections\" links direct users to?',\n", + " '1fd532fd-0f96-4bba-b503-f017885a3e91': 'What platform is the Wayback Machine app available on according to the context? ',\n", + " '70f47292-fdc1-46d3-aaa8-405219dfb9b2': 'What section follows the mention of the Wayback Machine (Android) in the provided context?',\n", + " '281ec4b0-f30b-4ad3-be5e-62bef111b811': 'How do commercials featuring real scientists affect public perceptions of scientific objectivity according to the context? ',\n", + " '3b91002b-433f-4cdd-8ad1-8a7232a45fa1': 'What types of presentations are typically found at the Whole Life Expos mentioned in the context?',\n", + " '6e080ecd-85e0-4d82-a867-62304b3f1855': 'What steps does the Foundation take to ensure compliance with charity laws in all 50 states? ',\n", + " 'ee1279d8-fab7-438e-9bec-dc32af209542': \"Where can donors check the status of the Foundation's compliance before sending donations?\",\n", + " '5f45299a-6a67-464a-a9ba-cdb3b57e10ae': 'What reason does Socrates give for believing there will be a delay in the arrival of the ship? ',\n", + " '7ef903df-49ed-4806-b9a2-059de83223a4': 'According to Crito, when is Socrates supposed to die?',\n", + " 'f59b70be-238e-4010-9287-97fa12c59784': 'What are the names of the collections mentioned in the media sub-navigation?',\n", + " '83ca1a73-0d41-4ed6-a764-20c9bdad408d': 'What is the purpose of the links under the \"Top\" section in the media sub-navigation?',\n", + " '78538704-3038-486d-a617-891bb76c8180': 'What reasons are given for Socrates to either stay in well-ordered cities or flee to places like Thessaly? ',\n", + " 'b0c4d18a-7c8d-44e6-92b8-1424b639fcf4': \"How might Socrates' friends in Thessaly react to the story of his escape from prison?\",\n", + " '3971529f-69da-4311-8bdb-fe8d486d038f': 'What does the speaker imply about Socrates\\' connection to the city by saying \"you never stirred out of her\"?',\n", + " '343c2955-696a-40d5-a0a7-5d448abee959': 'Why does the speaker advise Socrates not to escape out of the city?',\n", + " 'b86b2d8c-3dfa-4039-b978-633600dd11a7': 'How do some jurors form their opinions during a trial according to retrospective studies? ',\n", + " '422c9c6e-8c00-4a05-82d0-5a33800095ac': 'Why is quantifying information important when explaining or evaluating competing hypotheses?',\n", + " '2455bbba-f5bf-4c73-9bd5-6af8fda40810': 'According to the passage, why is the country considered more valuable and sacred than one’s parents or ancestors? ',\n", + " 'fe298b7f-5c57-4588-8a55-9ed57829caf6': 'What attitude does the passage suggest one should have when punished by their country?',\n", + " 'bfdc69f3-2fbd-4998-8b37-bc70a20c9beb': 'Why does Crito say \"We cannot help it\" to Socrates? ',\n", + " '6e960a10-1952-424c-a96b-a97160b84a8f': 'What situation might Crito be referring to when he says \"We cannot help it\"?',\n", + " '4a351199-33c9-4573-b3a2-d9246280b20d': 'According to the speaker, what should Socrates prioritize above life and children?',\n", + " '8be64181-a526-4415-bb10-4a1f30aaf6b4': 'What consequences does the speaker warn Socrates will result from breaking the covenants and agreements?',\n", + " '3f36740c-0efd-4af7-8126-9bcf8cffc2f5': 'What must a person do according to the laws when their city and country give an order? ',\n", + " '9013ce49-d14b-46a1-be14-8a7aae40a808': \"How does the speaker compare doing violence to one's country with doing violence to one's parents?\",\n", + " 'f27b0213-4c9a-4ce0-9644-83b6114474ed': 'What is being implied about a person who runs away and breaks the compacts and agreements they made as a citizen? ',\n", + " '01b614ff-d005-4f71-8ac4-f74af8463560': 'What question is posed to Crito regarding the agreement to be governed?',\n", + " 'a69905a5-d8e4-425f-a4d0-fc40cdd94788': 'What is the significance of the numerical sequences and decimal values presented in the context? ',\n", + " 'd8e18a12-b431-41a8-855a-fd979bc3a148': 'How do the repeated decimal patterns relate to the overall structure or meaning of the given data?',\n", + " 'b58b18c2-3e00-4119-a283-da93fe74b6c5': 'According to the World Health Organization, how many people are expected to die annually from smoking by 2020? ',\n", + " '0938bd4f-fae7-4195-bf68-779110e8ef48': \"What factors have contributed to the tobacco industry's success in promoting smoking, especially among young women in the developing world?\",\n", + " '1f7cabed-bae8-4360-a012-d8fc0fc3594e': 'What reasons does Crito give for urging Socrates to escape? ',\n", + " '21ca1b52-9797-4056-ad1f-4171c24823be': 'How does Crito fear he will be perceived by others if Socrates does not escape?',\n", + " 'dc4b1248-4a87-4cb3-9150-c042f4f47cb9': \"What event does the script trigger if elements with the class '.more_search' are present in the document? \",\n", + " 'eedb71ee-e82e-4878-bd7b-3044d6e9bdf5': 'How does the script ensure tracking occurs even if JavaScript is disabled in the browser?',\n", + " '00100506-eacb-4df8-a2d0-625f57e11efc': 'What is an example of confusing correlation with causation as described in the context? ',\n", + " '349e1720-8ce9-4129-bd40-3ffa859d4028': 'How is the straw man fallacy illustrated in the given context?',\n", + " 'd77b2d2c-ad6e-467d-80fd-656a2b657303': 'What is the significance of the value \"-29.0339092-4.5105709\" in the given context? ',\n", + " '41d4bbb3-bd3e-41da-b419-2f0d28088c43': 'How does the attribute \"fill=\\\\\"#ab2e33\\\\\"\" relate to the elements described in the provided data?',\n", + " '10a3ad68-1d1f-42f8-a577-a9388ab88fc4': 'What evidence from studies on mice supports the causal link between tobacco smoke and cancer? ',\n", + " 'd725676d-e566-42df-a450-dc5733ef2cf8': 'How did the six major tobacco companies respond to the 1953 research showing that cigarette smoke substances cause malignancies in rodents?',\n", + " '5a9f9f2b-376e-415a-b94b-7c905c7d4b7d': 'What conditions must a user meet to receive a full refund for a Project Gutenbergβ„’ work according to Section 4? ',\n", + " '44c97260-4c3a-40d5-b583-b41311eeb011': 'What actions must a user take if they do not agree to the terms of the full Project Gutenbergβ„’ License and request a refund?',\n", + " '7c3c6022-106e-4461-9e08-c46073f452fb': 'What is the difference between short-term and long-term considerations as discussed in the context? ',\n", + " 'd86f5aba-8959-48b6-9468-1ff3c0f85b90': 'How does the slippery slope argument relate to the excluded middle fallacy according to the provided context?',\n", + " '325086fc-185f-4e4e-a199-781a02d063c5': 'What methods does the psychic use to locate new mineral deposits according to the context? ',\n", + " 'c6b05c06-0e5e-4d90-82f8-1d2f568fcfda': 'How do people react to the statues of Jesus or murals of Mary being spotted with moisture?',\n", + " '5a74ac55-d7fa-47c6-ad34-313d53d363e4': 'What are the three categories listed in the media sub-navigation menu? ',\n", + " '90583762-fc08-4a5a-a550-60fff9a86b52': 'Which category in the media sub-navigation menu links to the URL \"/details/fedlink\"?',\n", + " 'c8e88c54-f5cd-462d-9fec-99320c81b982': 'What is the significance of the repeated numerical values such as \"1.363719-1.3551363\" in the given context? ',\n", + " 'a7f9f11d-c0a5-489e-a681-9c3d308375a0': 'How do the numerical sequences relate to the changes in vertical and horizontal measurements mentioned in the text?',\n", + " 'ed289161-61ac-4955-94ba-face8f1c2fb6': 'What is the name of the menu shown in the media-subnav section? ',\n", + " '791a8d2d-362a-4c33-b31e-a33ddaa34e07': 'Which website is linked under the \"Texts\" menu in the media-subnav?',\n", + " '992ecebc-8f39-4fd6-8639-6c234f459e69': 'What action did the Du Pont Corporation take after research in 1974 showed that their Freon product attacks the ozone layer? ',\n", + " '00e00b08-4cca-4c7f-9f4b-ec6b7af5c167': 'Why might major corporations protest independent scientific findings that suggest hazards related to their products?',\n", + " '5c8947fa-7e39-42b5-8a61-21d891814d34': 'According to Socrates and Crito, is it ever just to do wrong or injure someone in return for being wronged? ',\n", + " 'd73f0006-05e4-427a-af42-11540a6b02c0': 'How do Socrates and Crito view the morality of doing evil in response to evil?',\n", + " '762de16f-e974-4582-9793-bae2eb2cf13d': 'According to Socrates, whose opinion should be regarded when determining what is just and unjust? ',\n", + " '7a146d2d-d5ef-4580-97fe-e2f96f8a84a0': 'What does Socrates suggest is more important to value than life itself?',\n", + " 'cdba82e5-8a5f-4f2f-b971-ca79f903935e': \"What ideal does Plato aim to exhibit through Socrates' behavior in his last hours? \",\n", + " 'cde011e5-3152-43f6-8719-5114dc758e1f': 'How does Socrates view the role of reason in guiding actions, even when its conclusions may be fatal?',\n", + " '63da28c0-ff8b-4170-9804-c0b00abc080e': 'What is the purpose of the \"media-subnav\" element in the given context? ',\n", + " '8927ce97-995f-43f0-9b00-1721f2608382': 'How is the \"more-slider\" component structured within the provided HTML snippet?',\n", + " '95a66922-f1f2-45ee-9b17-e8b76bd96bcf': 'What factors does the Consolation of Haldane depend on regarding the nature of our universe? ',\n", + " '75567532-563d-4453-b8df-b4727c063615': 'Why might someone with a deep longing for life after death be drawn to study fields like cosmology and quantum gravity?',\n", + " 'b6c65bc0-992e-4b7f-bae5-0a81e31ea255': 'How does the concept of the Divine Mystery of the Trinity challenge the understanding of God as Father, Son, and Holy Ghost in the same Person? ',\n", + " '10943a2b-92c4-4c9b-abb2-1cea995ed44c': 'What are the implications of invoking Free Will and mysterious divine actions when explaining the coexistence of religious teachings on compassion with historical acts of cruelty?',\n", + " '7cad3607-b5f3-45e5-a33d-85bd6c0aaf41': 'What consequences does the speaker suggest will happen to your friends if you transgress and err in this way? ',\n", + " '66ea3b65-eba6-4133-bd84-231226c6270a': 'How does the speaker describe the reaction of neighboring cities like Thebes or Megara if you seek refuge there after committing such transgressions?',\n", + " 'd4ef2520-e5b2-479f-828f-9e34e1507fbf': \"What is Socrates' first principle regarding injury, retaliation, and warding off evil? \",\n", + " 'e8cc6283-a52a-4944-9eee-d5a4d909e3e4': 'How does Socrates describe the general acceptance of the opinion that one should not retaliate or render evil for evil?',\n", + " 'c867ba49-99a1-4d94-8c6b-893a82940cd5': 'What are the different navigation options available in the \"more-slider\" menu based on the provided context? ',\n", + " '7c26bc84-99dd-45e9-b19a-2f303b00651d': 'Which data-event-click-tracking values are associated with the \"Help\" and \"Donate\" links in the navigation?',\n", + " '97d2f2a3-d2eb-41f2-a5b5-e63600af6035': 'What does it mean when a work is not protected by U.S. copyright law according to the context? ',\n", + " '3428407c-b94a-4d4d-80ed-4bdbb289ce04': 'What restrictions apply to the use of the Project Gutenberg trademark when charging for an eBook?',\n", + " '7cd0f3ff-773c-454a-b63f-22b4b88020cd': 'What must you prominently display when copying, displaying, performing, distributing, or redistributing this electronic work? ',\n", + " 'a37a1233-3b7d-46e3-b8c6-bb1ff05f4dc7': 'According to the context, what should the displayed sentence include to comply with the Project Gutenbergβ„’ License?',\n", + " 'f4276eb8-1c32-4dff-9a7b-4e46bb8b532f': 'What does Crito think about the matter being discussed? ',\n", + " '390f2836-0471-46b1-85ae-b52dc2aee857': 'How does Crito express his opinion in the conversation?',\n", + " 'f1eab5fe-9c01-4b3c-a106-ceeb0ac0feb0': 'How many web pages does the Wayback Machine allow you to search through?',\n", + " 'b0c006b9-4871-4249-9aae-ddb83989cd82': 'Where can you find more information about defining web pages, web sites, and web captures according to the provided context?',\n", + " 'ee1b64fb-9020-4e9b-96af-9909e73d64be': 'According to Socrates, what happens to the body when it is acted upon by health and disease?',\n", + " 'a3986f56-5dc3-4284-b96d-3447ecb7b98a': 'How does Socrates compare the importance of the body to the part of man associated with justice and injustice?',\n", + " '687e3ac9-0f9c-4cc7-9ec6-c01ca029b684': 'How is artificial intelligence increasingly shaping the way Americans live? ',\n", + " '6783a186-8324-4aef-ba23-6ac42fe05ff1': 'What are some examples of tasks that artificial intelligence can perform according to the context?',\n", + " 'cf039b63-37c2-412d-bd85-eb6699dc36e3': 'What evidence would support the claim that Ramtha is truly 35,000 years old? ',\n", + " 'e8e97689-d411-46b0-bd52-4b949a5b76d2': 'What were the living conditions and social structures like 35,000 years ago where Ramtha supposedly lived?',\n", + " '0214389c-73d4-42bb-9e34-a472e1ed1707': 'What did J.B.S. Haldane imagine about the far future of the Universe when the stars have darkened? ',\n", + " 'f40dd0ed-98f7-4bf5-ba36-04f1fced352c': 'According to Haldane, what would happen over immense periods of time due to statistical fluctuations in the density of cold, thin gas in an infinitely old Universe?',\n", + " 'c7272a42-4dfc-4f4a-86e0-401d6d907e33': 'Why does the individual prefer death over exile according to the context? ',\n", + " '0a7d3839-4c6b-43dd-9ede-9332dbada63a': 'What potential consequences might the individual face if he takes his children to Thessaly or leaves them behind?',\n", + " '1b875d66-528e-4373-b80c-e3a1016be952': 'What is the purpose of the link with the class \"donate\" in the provided context? ',\n", + " '267456a9-4b4d-40ae-8172-e2e118b80a36': 'How is the donate icon visually described in the SVG element?',\n", + " '7eb92e9c-a3d5-4c9b-978c-db8c1a91e2a4': 'What misconceptions about crystals and their functions are mentioned in the context? ',\n", + " '82f660e6-f8a0-4065-956d-6430b09a9f4a': 'What types of content and opportunities are described as being available at the Whole Life Expo?',\n", + " '0bab67c7-5cc3-444a-84c3-097f122e2ad1': 'What type of warranties are explicitly excluded in the provided work according to paragraph 1.F.4? ',\n", + " '7d9006a8-33d6-49c8-9c07-8431f47eb679': 'Under what condition is the work provided, as stated in paragraph 1.F.4?',\n", + " '8353d568-2dcf-4987-83ac-0e4db456eebd': 'What reasons does Shelley give for believing that Socrates \"did well to die,\" and how do they differ from Plato\\'s portrayal? ',\n", + " '97d94415-1914-4837-a615-7bab22538773': 'Why might the selection of Crito as the person to propose escape to Socrates be considered an artistic choice by Plato?',\n", + " '12b0fd8c-c354-40d4-b6e4-3ebd3b346d25': 'What is the significance of the numerical values such as 0.84639831 and 1.53184695 in the given context? ',\n", + " '354967f3-2c32-43d8-8ef4-1332a03ef933': 'How does the fill attribute \"#211e1e\" relate to the visual representation described in the context?',\n", + " 'f186cec1-e7d2-467f-8dcb-8e6b95716fae': \"According to Carl Sagan, how do a person's will and affections influence their understanding of science? \",\n", + " '5ad9556c-6244-4d4c-9d7d-6d6821a9d0f2': 'What reasons does the passage give for why people might reject certain scientific truths or experiences?',\n", + " '4ee8cc27-8eb5-4236-b352-a6d2b9caafa8': 'According to Socrates, what is a good life equivalent to? ',\n", + " '2b16a324-b0ba-4037-9c8a-a68c9c971f9b': 'Does Crito agree that a good life is just and honorable?',\n", + " '93a3dcc3-4e44-44a3-8157-83ef63df57f1': \"How does the level of education affect adults' familiarity with AI according to the context? \",\n", + " 'bd98a79c-612d-4cf5-b86e-86ad7cfeb6a3': 'What percentage of U.S. adults had heard of ChatGPT as of March 2023?',\n", + " '5a5a9740-8582-44a1-93de-2686073176c7': \"What did the woman in Socrates' vision tell him to do? \",\n", + " 'b4184d29-fa77-4246-813e-f5ee016a787b': \"How does Crito react to Socrates' description of the vision?\",\n", + " '223da833-464e-43ac-875c-e3ed22fd721b': 'Who does J. Z. Knight claim to be in contact with, and what is notable about this entity? ',\n", + " '980ce94d-8363-46a1-b6ab-ba05005db20f': 'According to the context, what possibility does spiritualism or channeling suggest about death and communication with loved ones?',\n", + " '7c317ccc-108c-4c34-aa3a-4be58bcaf121': 'How can comparing those who experienced remission of symptoms with those who received a new drug help determine correlation? ',\n", + " '5ff97d28-cef4-4f9e-8b86-006565cc5325': \"Why is it important for the officer in charge of a police lineup to be unaware of the prime suspect's identity?\",\n", + " 'dbfc5f13-3642-4d66-aac2-66b6c0b83f5c': 'What are the three software categories listed in the media sub-navigation menu? ',\n", + " '5bdc0daa-cd3b-4ca4-ae6f-f1310e884033': 'Which software category link includes the data-event-click-tracking attribute \"TopNav|SharewareCD-ROMsSoftware\"?',\n", + " '5136fd17-273c-46a0-b16b-641214df95e0': 'What reasons do the laws give for why Socrates would be doing them an injury by attempting to disobey them? ',\n", + " '0b0dfa6e-e43c-424c-932c-f141310dc8ee': 'According to the laws, what options are available to an Athenian citizen who does not like the city after reaching adulthood?',\n", + " '48180484-eca8-48c7-807d-3c210a57f269': 'What is described as coming \"like the wind\" and being a \"Life Force\" larger than the energies in your body? ',\n", + " 'db86dcfd-d811-4d69-9e31-b51867d11efe': 'According to the context, what do people mainly pay attention to in relation to old-time religion and life after death?',\n", + " '7fce6441-0603-4821-b33c-746590693921': 'According to the speaker, why should some opinions be valued while others are not? ',\n", + " '32252a64-0249-4960-82d1-88c021fbe044': 'What reason does the speaker give for Crito being disinterested and not liable to be deceived by his current circumstances?',\n", + " '735816ec-79f0-4029-9a5a-675bfcb35d62': 'What must you obtain if you wish to charge a fee or distribute a Project Gutenbergβ„’ electronic work on different terms than those set forth in the agreement? ',\n", + " 'baeca4b9-eb2e-4909-a4d2-f9915f1bb1fd': 'Who manages the Project Gutenbergβ„’ trademark and must be contacted for permission to distribute works under different terms?',\n", + " '9bcd233d-535e-43bd-aea8-e580f5982e84': 'How can someone make donations to the Project Gutenberg Literary Archive Foundation? ',\n", + " 'dbb8bd92-73a8-4305-bd8b-587c2dd1be5b': 'What are the ways to help produce new eBooks for Project Gutenberg?',\n", + " '97e722b9-2167-4f3f-8dd6-d396da3fdadc': 'According to Socrates, whose opinions should be regarded as good and whose as evil? ',\n", + " 'd118bb48-96d9-4c0b-8402-df3d22acfbcf': 'In the analogy of the pupil practicing gymnastics, whose praise and blame should the pupil attend to?',\n", + " 'c246d4dd-d2df-4cdb-a2c4-8c24a72dc9ae': 'What is the purpose of the \"Archive-It Subscription\" section in the context provided? ',\n", + " 'a6b0a307-4971-446c-b6db-1e6c97399a34': 'Where does the \"Explore the Collections\" link direct users to?',\n", + " 'f9eb898d-8ab6-4403-85fa-ebcc041c459c': 'According to Socrates, whose opinion should we value more when considering questions of justice and injusticeβ€”the many or the one man who has understanding? ',\n", + " '0ac57091-f6d5-473d-94d7-bbde7ce2baf9': 'What does Socrates suggest happens to the principle within us that is improved by justice and deteriorated by injustice if we desert the man who understands these concepts?',\n", + " '68047f28-aadc-4881-a301-0c352f62557e': 'What reasons does the speaker give to suggest that the person addressed was once satisfied with their state and government? ',\n", + " '83560fa8-46ec-4393-bb6d-409f891feab3': \"How does the speaker contrast the person's past preference for death over exile with their current actions toward the laws?\",\n", + " '0aae9dc6-3b2f-413c-ac66-cf32e84f487d': 'How might we test the claim that Ramtha speaks independently without the assistance of Ms. Knight’s mouth? ',\n", + " 'cb222e99-71c0-4944-b20c-56a18c4c637a': 'If Ramtha were available for questioning, how could we verify whether he truly lived 35,000 years ago?',\n", + " '8cc36baf-b5fc-4a26-91fb-dae3cdd9be76': 'What type of data or information is represented by the numerical sequences and symbols in the given context? ',\n", + " 'bcfd553c-4b33-435c-9499-d5ebd70d47a3': 'How might the values and characters in the context relate to graphical or path data in a digital format?',\n", + " 'ccfd94ba-b00f-4f59-995b-430b737bb6e1': \"According to Fermi's argument, what is the probability of winning five consecutive battles purely by chance?\",\n", + " '9d4520ec-503f-458b-88e7-dfef6ad69bef': 'What logical fallacy is described as considering only two extremes without acknowledging intermediate possibilities?',\n", + " '9945e710-28fd-46fd-b280-47ddc6abd6d5': 'What does Socrates ask Crito to do if he cannot refute his argument? ',\n", + " '7cc74b08-24f8-476f-aff7-7291ed44b551': 'Why does Socrates say he may not be persuaded to escape from prison?',\n", + " '0354361b-6bbc-4ace-b649-abb91b89e717': 'How might presidents use \"weasel words\" to justify military actions without a formal declaration of war by Congress? ',\n", + " '2d73818c-7095-41f5-b5b7-17e02c701b23': 'In what ways can the control of foreign policy and war conduct by the President influence their chances of reelection?',\n", + " '017372f0-5325-4f1f-aaa1-961ca08735e8': 'What function is called to send a pageview event when the page loads according to the script?',\n", + " '546fb047-a0be-4fe4-b587-5a09fa103b06': 'How does the script handle undefined or empty data packets in the variable `vs`?',\n", + " '7683486c-16cd-4774-8d7b-34f1578b3344': 'What percentage of Americans say they have heard at least a little about artificial intelligence according to the August 2023 survey? ',\n", + " '03619ca5-9d64-4f6c-9315-dcdef342d31f': 'How many U.S. adults correctly identified all six examples of AI in everyday life in the December 2022 survey?',\n", + " '14693807-dbf9-4f80-9b80-168a47c2c01b': 'What are the three categories listed in the media sub-navigation menu? ',\n", + " '8e41e518-7455-4c2b-81d3-8b1fd612e2ae': 'Which category link includes the data-event-click-tracking attribute value \"TopNav|Arts&MusicVideo\"?',\n", + " '10b6f237-bc6a-4da8-a2b6-8a0880e80917': 'According to the context, what must your use and distribution comply with if an individual Project Gutenbergβ„’ electronic work is posted with the permission of the copyright holder?',\n", + " '08de4576-1217-4633-b586-cfedbf8210ed': 'What is prohibited regarding the Project Gutenbergβ„’ License terms in relation to the work or any associated files?',\n", + " 'a08f1e80-8e83-46ef-bf0a-54b43de59f23': 'How did Clement of Alexandria view the telling of fabulous stories to children in his time? ',\n", + " '41b0392f-24f4-4bf4-a975-92be7ec5ec88': 'What reasons are given in the context for why modern adults retract myths like Santa Claus and the Easter Bunny from children?',\n", + " 'bd198cce-5070-45f8-94df-9ab5f9dcfc72': \"What is Socrates' main criterion for deciding whether he ought to escape from prison? \",\n", + " '3d2be7b7-86bf-4389-bf34-0a15ed5d2911': 'How does Socrates view the common concerns about money, reputation, and family duties in relation to his decision to escape?',\n", + " '6e9cef4f-519e-4629-a537-c116172b2278': 'What is the purpose of the form described in the context? ',\n", + " '5d69128e-6f9e-410b-bb8a-084496c02392': 'What input is required from the user to use the \"Save Page Now\" feature?',\n", + " 'dbb5b17c-98c7-456d-847b-4c72539428bb': 'Who are the \"brethren\" mentioned in the context, and why will the laws in the world below receive you as an enemy? ',\n", + " 'f08e36e0-2eac-4ccd-b824-51d87f918caf': 'Why does the speaker advise to listen to \"us\" instead of Crito?',\n", + " 'c564d614-e11d-495b-8939-23de1bcc067a': 'What demographic groups show higher familiarity with ChatGPT according to the context? ',\n", + " '8016de87-47a3-4de6-9604-dd4103d945f8': 'What percentage of U.S. teens ages 13 to 17 have used ChatGPT to help with their schoolwork?',\n", + " 'a534d4d6-13bc-4389-a727-bc2a9e43ae0c': 'What is the role of the `getItem` function in the `download.php` file at line 308? ',\n", + " '5c5878b7-b589-4328-bc4b-6b4dfb5cce87': 'How does the `AdapterPipeline.inc` file contribute to the metadata reading process based on the functions `pipeline` and `fetch`?',\n", + " '0a3d3d91-ef9f-439a-83be-f0bd09d5faf6': \"What reasons do Socrates' accusers give for targeting him above all other Athenians? \",\n", + " '07d23d6a-8119-41bf-be6c-84a2b30cdd46': \"How does Socrates' constant residence in Athens serve as evidence of his relationship with the city according to the passage?\",\n", + " '226b2243-6e14-4863-9bec-ca8781d17ac0': 'What are the names of the libraries listed in the media sub-navigation? ',\n", + " 'b8711578-5e81-412d-af06-c83c1c743d86': 'Which library link includes the data-event-click-tracking attribute value \"TopNav|ProjectGutenbergTexts\"?',\n", + " '82310728-6b4c-4068-9668-45eb59f95331': 'What is the significance of the repeated numerical sequences and decimal values in the given context? ',\n", + " '5f653cff-32a9-4a18-969a-746758eec240': 'How do the changes in numerical values relate to the structural or functional aspects described in the context?',\n", + " '95c74c6d-e938-4f0b-9d20-409c5428fb44': 'How does nature\\'s process of \"ratcheting up\" relate to the idea of saving what works and discarding what doesn’t?',\n", + " '93adc22e-c303-4d61-9b9b-1b63e1a462a1': 'What concerns are raised about the accuracy and timing of the \"prophecy\" regarding the assassination attempt on President Reagan?',\n", + " '1f8f9f85-a708-4257-bc7b-ec2351ea2cdb': 'Why did Crito choose not to awaken Socrates immediately upon his arrival? ',\n", + " 'd89b78ef-b4b4-4a1a-b978-80927f6b6487': \"How does Crito describe Socrates' demeanor despite the trouble he is facing?\",\n", + " '6a704e34-1288-4967-ab3e-e1f49e071052': 'What criteria did Enrico Fermi use to define a \"great general\" during his conversation with U.S. flag officers? ',\n", + " '3d6b6bb6-8f77-4c52-981f-d4a4b9c6e8f3': 'According to the discussion in the context, what percentage of American generals were considered \"great\"?',\n", + " '3dde3324-4768-4325-8bb3-d46f201645b6': 'What consolation does the author find lacking in the idea of an identical Earth reappearing infinitely? ',\n", + " '9aebe243-3115-4c1f-a898-46977943c8ca': 'How does Haldane’s picture change the author’s understanding of infinity and memory?',\n", + " 'bdbb76cb-4dfd-433d-8be3-2a727776caa4': 'Why does Socrates believe that a man of his age should not be repining at the approach of death? ',\n", + " '22c4f9f1-fd8a-48c0-a9d3-9573d3cee60b': 'What message does Crito bring to Socrates, and why is it particularly painful for him?',\n", + " '978b8e8a-95e2-4be4-a986-d19dd167a7e2': 'Why is it important to conduct experiments in a \"double-blind\" manner according to the context? ',\n", + " '28767ceb-c0fe-4de4-939a-b414811142ab': \"How might knowledge of which patients received a new drug influence physicians' evaluations in a medical experiment?\",\n", + " 'd54f8f42-cd73-49bb-8476-68e217d0c792': 'What are the three categories listed in the media sub-navigation menu? ',\n", + " '960a2866-9dee-4525-806a-afdf23d3ae2e': 'Which category link includes the data-event-click-tracking attribute \"TopNav|BiodiversityHeritageLibraryTexts\"?',\n", + " '997f6c82-f7a5-451f-845d-f89361662369': 'What is the significance of the numerical values and coordinates in the provided path data?',\n", + " '98f206f8-6079-4c7a-8704-b7d7d098c7f5': 'How does the path element\\'s \"fill-color\" and \"style-scope wayback-search\" classes affect its appearance or function?',\n", + " '4421cf9e-4610-4add-883a-a4a4fddb3652': 'According to the passage, what are the three reasons given for why a person who disobeys the state is considered \"thrice wrong\"?',\n", + " '707792ab-4088-406c-b3f3-e944cc518385': 'What choice does the state offer to those who question its commands, as described in the context?',\n", + " '70087d5b-1bbb-4f1a-bed9-2d73f73ebb22': 'What percentage of teens find it acceptable to use ChatGPT for researching new things? ',\n", + " 'b2e9017a-1bce-48ed-a1b9-75479b94fdb4': 'According to the December 2022 survey, what proportion of Americans believe AI will have a major impact on workers in general?',\n", + " '9e548e5f-e31f-4de4-a527-699c17501f56': 'What are some examples of health risks mentioned in the context related to common medications like aspirin and acetaminophen? ',\n", + " '32a826c8-7da3-41af-8409-d735c211d2d2': 'How does the context describe the impact of commercial culture and paid product endorsements on consumers?',\n", + " '96146e37-8a50-4bf5-b38f-f84a999b659a': 'According to Socrates, is doing wrong always considered evil and dishonorable? ',\n", + " 'fd6c6284-5bb0-4384-9bc1-0601c94c5f0d': 'Does Crito agree with Socrates that injustice is always an evil and dishonor to the person who commits it?',\n", + " '78004db8-deda-46d2-abca-2811a73e09bf': 'What argument do the Laws of Athens present to Socrates regarding his obligation to obey them? ',\n", + " '210c787b-0742-40ed-9c04-264fb12d0c11': 'How does Socrates justify his continued residence and acceptance of the Laws of Athens despite any grievances he may have?',\n", + " 'bfb68d1e-141f-45d7-8533-461c0e582b09': 'Who was the originator of the Project Gutenbergβ„’ concept of a library of electronic works? ',\n", + " 'e635517e-c9fb-45bb-9cca-86eb7b8b8872': 'How does Project Gutenbergβ„’ ensure that the eBooks they produce are not protected by copyright in the U.S.?',\n", + " '0f8e88de-bf53-483a-b1f9-b8cd5793046b': 'What is the significance of the values and coordinates mentioned in the context, such as \"6.9117365\" and \"-36.64339281\"?',\n", + " '8bef38e1-3eb3-425d-87fa-92b5a6e4e0ac': 'How do the repeated references to measurements and adjustments, like \"v39.5210721h-5.7365952\" and \"c6.6510318-5.555347,\" relate to the overall structure described in the context?',\n", + " '87f2ebd3-f848-49b6-bd73-50638d8cb59a': 'What percentage of Americans are more concerned than excited about AI in daily life according to Pew Research Center surveys? ',\n", + " '6104402a-901e-4c42-9880-d8eb62b24413': \"How do Americans' attitudes toward AI vary depending on the context of its use?\",\n", + " '70c3891f-49e8-43c4-b1a8-e79b77467d4a': 'What does Occam’s Razor suggest when comparing two hypotheses that explain the data equally well? ',\n", + " '718d8fd0-1703-4159-9707-baa29fca21c5': 'Why is it important for a hypothesis to be falsifiable according to the given context?',\n", + " '2898722d-8190-486b-a2b5-166b2a27490d': 'According to Socrates, should a man do what he admits to be right or betray what is right? ',\n", + " 'da1bf5c8-8a4a-4821-afd4-56b8c9d25488': 'What dilemma does Socrates present about leaving the prison against the will of the Athenians?',\n", + " '2aac97a0-a46b-4288-9ce8-bf8eb7ebcd85': 'What are the different types of content you can search for using the provided radio button options? ',\n", + " 'b781b9da-0dd5-4f0f-8891-293e5000ac1e': 'Which input value corresponds to searching TV news captions in the given context?',\n", + " '088a320f-e45b-4e3a-88b6-667ccb0296f1': 'Who is the author of the eBook titled \"Crito\"? ',\n", + " '32c259cf-92c0-4ff8-a8b8-3ab79fbe5060': 'Under what conditions can you use, copy, or distribute the Project Gutenberg eBook of \"Crito\"?',\n", + " '67de76e6-106f-46ea-968a-5b4048acf404': 'What numerical values are mentioned at the beginning of the context, and how do they relate to each other?',\n", + " 'ce7e0a89-42f2-4fe0-bc7c-90dc91b3069c': 'How does the context describe changes in values or states over time, particularly with references to \"s\" and \"v\" notations?',\n", + " 'bdec125c-8ce8-446c-a9eb-d748e64751a7': 'What formats are allowed for converting and distributing a Project Gutenbergβ„’ work according to the given context? ',\n", + " '064a414d-db2b-47e0-a8e3-2ca67fa2d0b6': 'What must be provided at no additional cost if a Project Gutenbergβ„’ work is distributed in a format other than β€œPlain Vanilla ASCII” or the official format?',\n", + " '12cd3973-7268-4819-ae0e-cfb51da0fd0d': 'According to the context, whose copyright laws govern what you can do with the work you have downloaded? ',\n", + " 'd8ca5637-653b-4cf0-9e0e-93b3169eb191': 'What should you do if you are located outside the United States before using a Project Gutenbergβ„’ work?',\n", + " 'cc4fc1bd-70fd-495e-b5b9-17732d11e5a4': 'What is the significance of advertising low \"tar\" cigarettes according to the context? ',\n", + " '525976b8-f13e-49e4-a7a3-b492088cef7a': 'How does the rhetoric mentioned compare to that of the tobacco industry?',\n", + " '0d976ce3-89ca-4c45-9ba3-045146ef9c0e': 'What are some of the permitted uses of the Project Gutenberg eBooks according to the trademark license? ',\n", + " '400c76d3-48f4-4e95-9427-176856af766e': 'Under what condition is redistribution of Project Gutenberg eBooks restricted?',\n", + " 'c1118c70-e00f-4d76-b7b6-6f060b7298e7': 'What must you do if you do not agree to abide by all the terms of the Project Gutenbergβ„’ license agreement? ',\n", + " '0352421c-6298-473b-a688-2b03f2b3ed25': 'How can you obtain a refund if you paid a fee for a Project Gutenbergβ„’ electronic work but do not agree to the terms of the license?'}" + ] + }, + "metadata": {}, + "execution_count": 23 + } + ], + "source": [ + "# Lets see some questions!\n", + "test_questions" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "K_jYOnAI43zK" + }, + "source": [ + "### Reformatting and Saving Datasets\n", + "\n", + "Now, we can save our datasets for later use!\n", + "We save these generated datasets, along with the original document chunks (the \"corpus\"), into JSON files: training_dataset.jsonl, val_dataset.jsonl, and test_dataset.jsonl.\n", + "\n", + "\n", + "\n", + " We're saving the train_dataset, val_dataset, and test_dataset dictionaries as JSON files (using json.dump) for several key reasons:\n", + "\n", + "Human Readability: JSON (JavaScript Object Notation) is a text-based format that is relatively easy for humans to read and understand, especially compared to binary formats. This makes it simple to inspect the generated questions and contexts if needed.\n", + "\n", + "Interoperability: JSON is a language-independent data format. While we're using Python now, saving in JSON means the data could be easily loaded and used by programs written in other languages if necessary in the future.\n", + "Ease of Use in Python: Python's built-in json library makes it trivial to serialize (write) Python dictionaries and lists to a JSON file (json.dump) and deserialize (read) them back into Python objects (json.load) later in the notebook or in a different script.\n", + "\n", + "Structured Data: JSON naturally represents the kind of nested structures we have (dictionaries containing other dictionaries and lists), mapping well to our Python objects.\n", + "Essentially, it's a standard, convenient, and readable way to store structured data like our question-answer pairs and corpus for later use." + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": { + "id": "iF6IFFq9VsNu" + }, + "outputs": [], + "source": [ + "import json\n", + "\n", + "# training-split-documents is the original document chunks. It is a LangChain Document object. Convert it to a\n", + "# dictionary with the document id as the key and the page content as the value.\n", + "training_corpus = {train_item.metadata[\"id\"] : train_item.page_content for train_item in training_split_documents}\n", + "\n", + "# training_questions is a dictionary with the question id as the key and the question as the value.\n", + "# training_relevant_contexts is a dictionary with the question id as the key and the document id as the value.\n", + "# training_corpus is a dictionary with the document id as the key and the page content as the value.\n", + "train_dataset = {\n", + " \"questions\" : training_questions,\n", + " \"relevant_contexts\" : training_relevant_contexts,\n", + " \"corpus\" : training_corpus\n", + "}\n", + "\n", + "# Save the training dataset to a JSON file\n", + "with open(\"training_dataset.jsonl\", \"w\") as f:\n", + " json.dump(train_dataset, f)" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": { + "id": "PqF9WaueV-V8" + }, + "outputs": [], + "source": [ + "val_corpus = {val_item.metadata[\"id\"] : val_item.page_content for val_item in val_split_documents}\n", + "\n", + "val_dataset = {\n", + " \"questions\" : val_questions,\n", + " \"relevant_contexts\" : val_relevant_contexts,\n", + " \"corpus\" : val_corpus\n", + "}\n", + "\n", + "with open(\"val_dataset.jsonl\", \"w\") as f:\n", + " json.dump(val_dataset, f)" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": { + "id": "0DSQ7WMnWAu6" + }, + "outputs": [], + "source": [ + "train_corpus = {test_item.metadata[\"id\"] : test_item.page_content for test_item in test_split_documents}\n", + "\n", + "test_dataset = {\n", + " \"questions\" : test_questions,\n", + " \"relevant_contexts\" : test_relevant_contexts,\n", + " \"corpus\" : train_corpus\n", + "}\n", + "\n", + "with open(\"test_dataset.jsonl\", \"w\") as f:\n", + " json.dump(test_dataset, f)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "vAwklqQCgVi-" + }, + "source": [ + "## Task 4: Fine-tuning `snowflake-arctic-embed-l`\n", + "\n", + "Now that we have a dataset, let's grab a `sentence-transformers` Embeddings model!\n", + "\n", + "We'll be using Snowflake's [`snowflake-arctic-embed-l`](https://huggingface.co/Snowflake/snowflake-arctic-embed-l) as a base embeddings model.\n", + "\n", + "So, we're essentially taking a knowledgeable generalist and turning it into a specialist for our particular task and data. This way, we use less data and compute, we leverage existing knowledge of a pre trained model, and we do focused learning.\n", + "\n", + "It is a well performing embeddings model by itself, but there's a lot of very specific domain terms and vocabulary in our courpus - so lets fine-tune it and see what that can do for us!\n", + "\n", + ">> NOTE: Skip installing dependencies if you are running this notebook locally." + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": { + "id": "AXzVHP3v1Cno", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "6f9128b6-36f5-463c-ebee-2fdc27333f2a" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m491.5/491.5 kB\u001b[0m \u001b[31m20.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m42.3/42.3 MB\u001b[0m \u001b[31m59.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m193.6/193.6 kB\u001b[0m \u001b[31m19.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n", + "gcsfs 2025.3.2 requires fsspec==2025.3.2, but you have fsspec 2025.3.0 which is incompatible.\n", + "pylibcudf-cu12 25.2.1 requires pyarrow<20.0.0a0,>=14.0.0; platform_machine == \"x86_64\", but you have pyarrow 20.0.0 which is incompatible.\n", + "cudf-cu12 25.2.1 requires pyarrow<20.0.0a0,>=14.0.0; platform_machine == \"x86_64\", but you have pyarrow 20.0.0 which is incompatible.\u001b[0m\u001b[31m\n", + "\u001b[0m" + ] + } + ], + "source": [ + "!pip install -qU sentence_transformers datasets pyarrow" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "id": "G-PGsQB7Xo6V", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 369, + "referenced_widgets": [ + "b6301e443fd84844991a50f6950a34d3", + "108a14d3e60c48979e767e71286b1f9f", + "1769f34120624f7f99a886f774cbcd7e", + "0c7de7a830164795897d1e134fdf3524", + "1bde6edcfbc64cabbda95a18ee65bfef", + "828d57002ab94e90b6a5b8f404765cb3", + "e88cad91f2a8499989dad61394725350", + "80e211d62556464dbb71de7dee22f845", + "c64275e3c65d4599b8c00eaf5400ee6a", + "0360b811cd314f94aa4e2d4472434a4b", + "ffd849455079460a81f045505ea04d75", + "cb6b64a5fef5428897527bcf1e231e7f", + "dd86e848207f418aaa6de7e06f6502cd", + "0fd9e2d62cf242578da61c79366b9e35", + "695a33a4bab94fd2a70aab044b0356e0", + "bb57cdd9477641caa1335c7cb464d5e9", + "b8d4e32967ef4a5bb4ff8ebff39b244e", + "0dd4478d22bf4734b97db1e78a5ca548", + "ed61b2a09c444747bf61501c045f5acf", + "4279ec0cb1fe4c899f0f6ccf7ed8a8b4", + "af6b15f7163947ecb713157084c20dfc", + "c839234d06d44aa8973ebfa1f4e0a100", + "a485f31a2abc49498b5c59e3523fc9be", + "699b42c314cd437b95cda77f7e7384d4", + "7dbb2e58cf874ad5bf11809d7a9d1258", + "651e0533fa8344bc849904ad867c1922", + "766e15fa5aed44ad86a693eb43095c13", + "de9f359a37b3450d857fa24f32aacfe4", + "6f5911d1d54540a380b2c9822c026ba1", + "a993da3455df4454add4623af76d644b", + "d8394e5f45894a7387f2544aa7eef6c5", + "d371178a535c4ff8a777d531caeabe7d", + "2dbe6450b9284a9fbfa3c80eeb18e36f", + "a57f01a0a0084ea1a0d6d0071c2cab3e", + "efbef2a276c54f288392498f2c9be5dd", + "8661a966d644457f8fb9afacbe5c1974", + "9589b453269e4a4abe514050134ff3a0", + "ee2e792c16774e33976fbf6b613c904a", + "16d5892b57b841458c5149de98c3f4ab", + "ae8158ffe4234d4885e45936e0fa53f8", + "2cc996696fe54861b6587b8c1bebb911", + "e9c541335cec4e019fc6f03e3581e886", + "b58d32b86f1c43d5b8c0ac2ad9c4a670", + "e499f8b0fce64b1ab5e3f7dc0bf4228b", + "503d81fc173a404d94087c33d9158a41", + "026bed8e4fe8484f9e443d6982c67fe2", + "644d9e8fb743490ea6f0d2c6c8be7a74", + "7d0dffb18e554de6a7130df37f8edcaf", + "d17b4c123cd54122b69c2a615fb01bbc", + "83f3d5d59e7d4fe48616fcfa7d96818d", + "c329a098e4074edb814f9b3982f6c7d4", + "1a78ad30d346495984e02b46c37d51f1", + "d6dc2f432caf4607afe065384421b53c", + "40cfca37a32d4a668a2755c0e455127f", + "20b62a0ee2a441a995b9143f1b62b7da", + "cfa7400579474370a5351e5b4cc2eebe", + "45940be0514e45e98839f67746ada83b", + "8f6b27ae83754c309c0be766f475aee9", + "18e1ade2b72d4b42ae7ebb4139e4892c", + "099f1441619245c793fefe89101db7c3", + "528cd3e59bcc46bbb3ed2db02d30c20d", + "8bd9f734a7a24a77b67ae691ecdb6882", + "72272bf41f2548d9a19134adc38e4645", + "8cfd6ee1ab2240b6b2f4cf56650781e1", + "7564568b8cca457c86f7dd81cf86f3f9", + "7cf5122a6599450a844bd164fdff5d8c", + "95daedff61324647bf2e597df23f4b8c", + "fca84d17cad449de943c3374710f9bbc", + "085e6f70bf984f0896e140b636cf34bf", + "e2e6db6f7ac84bbf9084afafbfab0056", + "bf821f44ed94401a86a822f4b8a46894", + "0575a67c1fb04d5ebd814baa07994817", + "21ef965a52904d128bc7426088f9083f", + "3c8dbcbe555949c9ab9df3c3ceaedede", + "7e280a7580ce49328d2e0e2e11d9ae56", + "26ba9f4e69f941e09c2827e6fe0fd2d3", + "74137c8f95ff4af79826f8921683820d", + "976eba6063754ce1abc0e9c26b4e7dcd", + "1c29d94d2dcb443ab875280651ca4c6c", + "1876b833ddd148c1b11f92763ef4dedc", + "db20e7a134a8431eb6854a7167a27c46", + "c95ebfbacab14e43b1885c5b02e91285", + "e81544773c7f4a64adca17f543281abc", + "1eb2314e77b44aa1b3d324cf097169a3", + "ccd97ed6ad20478fa4ee67011a3d37db", + "11fb745d929b471ba0c00bd70e22cfc7", + "cb8d61ae7dca484488f3ea43e983a6fb", + "c9372109bc004e06a2c47908da32e536", + "f257141a6f18401dad79a6efd148124f", + "3869f8481e2947b888fd2b829e607601", + "6396369ab3d34f08adffebcd457ae47b", + "9774c7359a634c639c180fc2b316839f", + "53eed3f0eb644e3ab2617d60c91eacc6", + "b3314162932f491da150cd7cf07cd2d1", + "12db42dd32c448d4a5d4f0d1c6406eea", + "cede1a8f47294b73872192c997a72fba", + "d2441d036c3341be95cc0b1a43264073", + "67d852327a7a48b686b48aee7c262c53", + "31b65d68e1084b0c86fc1f3d4f6baab1", + "7cca7d66ca114cc6a28d6f462eef9a8e", + "f59f485740b24966be6f76d31c5aae83", + "1c460d52be2a4d1fb8e6ececf35048f7", + "6a96db6392894eb899a161cd0c480bc8", + "ddbd04ecee6c4c739933216ac901a28f", + "38391ebc2e2a42ac8a99214e78131464", + "ea1018a84a5842329bb3a5ec91100a2d", + "13f077624ab74511bbec8cfd409f1f7d", + "c2bb7ebd3328454d94d854b5b1dec4ed", + "6f98b0fcfa8d4ea28f0f903d34f5427a", + "7ca4c152e1cc47fe99bb440d5565e44d", + "4ae9037538584adaa96e80415239a8fa", + "0ce565970c1d40328ac54a157609902f", + "0f2ddc6bafcd4066a9945fa0f1c517e2", + "eb9e77674003464eae6fd0bbea9d1361", + "3eebface9bcf4bd48a98a3961a474d15", + "5790b2e22323416a80d56da7a78e3afb", + "d243906ef5254edf8f1a1f015998d0f2", + "ca938b533b934559829d7a05b95f54bc", + "690cda34446c43d7b77437c4a6807a8b", + "0ce8cda0337e4250be7c37f2216eb4c1", + "56af82497ec64575a65ad47efd13fbe3" + ] + }, + "outputId": "645a6385-cf81-412c-d9f4-fd6ff131263c" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "modules.json: 0%| | 0.00/349 [00:00 NOTE: PyTorch (`torch`) is a popular machine learning library - while we don't go very deep into PyTorch it's an incredibly powerful and interesting library! Please read more about it [here](https://pytorch.org/tutorials/beginner/basics/intro.html)!" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": { + "id": "B-WbpuUWYFJr" + }, + "outputs": [], + "source": [ + "from torch.utils.data import DataLoader\n", + "from torch.utils.data import Dataset\n", + "from sentence_transformers import InputExample" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "AJtPPlck8HBE" + }, + "source": [ + "We're using a toy batch size here to reflect the limited number of examples we have. We have 78 training documents, and two questions each so all of 156 ques/doc pairs or examples.\n", + "\n", + "> NOTE: It is typical to use a much larger batch size (~64+), hardware permitting." + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": { + "id": "8Lokhy6KYHAv" + }, + "outputs": [], + "source": [ + "BATCH_SIZE = 8" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "b-6DT8hc8PmT" + }, + "source": [ + "Let's move our dataset into the expected format for training.\n", + "\n", + "Remember we have query_ids, doc_ids, and query text and doc text. Now, we want query_text, doc_text examples. We go from query_id to doc_id to doc_text\n", + "\n", + "The loss function (which we'll see next) will use these pairs to calculate how \"far apart\" the query and text embeddings currently are and generate gradients to push them closer together.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": { + "id": "JJk37zQsYJ4P" + }, + "outputs": [], + "source": [ + "corpus = train_dataset['corpus']\n", + "queries = train_dataset['questions']\n", + "relevant_docs = train_dataset['relevant_contexts']\n", + "\n", + "examples = []\n", + "for query_id, query in queries.items():\n", + " doc_id = relevant_docs[query_id][0]\n", + " text = corpus[doc_id]\n", + " example = InputExample(texts=[query, text])\n", + " examples.append(example)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "OjFx7KHI8TL0" + }, + "source": [ + "Now we can create a `torch` `DataLoader`!\n", + "\n", + "**Shuffling**: The DataLoader, by default, shuffles the training examples at the beginning of each epoch (each full pass through the data). This is crucial to prevent the model from learning any patterns based on the order in which examples happen to appear in the dataset. If the data wasn't shuffled, the model might inadvertently learn biases related to the sequence, which could hurt its ability to generalize to new, unseen data. Shuffling ensures it sees examples in a random order each time.\n", + "\n", + "**Batching:** (our size=10)\n", + "\n", + "Efficiency: 10 at a time in parallel. And Gradient Stability: We compute error and direction every batch size and update gradient. Smoother and more relaible training.\n", + "\n", + "**Within One Epoch**:\n", + "\n", + " - Get the next 10 examples.\n", + " - Forward pass: Compute embeddings for these\n", + " - Calculate loss. A score to see how well the model did. Specifically, it measures if the related query-context pairs are closer together in embedding space than unrelated pairs within that same batch.\n", + " - Calculate gradients: Based on loss, how much each parameter contributed to the loss. (backward pass) - these are gradients.\n", + " - Update parametes (gradient adjustment): An \"optimizer\" uses these gradients to slightly adjust the model's parameters (weights) to try and reduce the loss next time. This \"adjustment\" happens after processing each batch.\n", + " - Repeat: Steps 1-5 are repeated for the next batch, and the next, until all the training examples have been seen once.\n", + "\n", + " \n", + "**Gradient Stability:** Imagine updating the model based on just one query-context pair. That single example might be weird or unrepresentative, causing the parameter update (gradient adjustment) to be jerky or point in a slightly wrong direction. By calculating the loss and gradients over a batch (10 examples), the \"weirdness\" of individual examples tends to average out. The resulting gradient provides a more stable, reliable estimate of the direction the parameters should move to improve performance on average across those 10 examples. This usually leads to smoother, more consistent training.\n", + "\n", + "\n", + "**Epochs and Validation:**\n", + "An epoch is defined as one complete pass through the entire training dataset. Since we have 156 training examples and a batch size of 10, one epoch consists of ceil(156 / 10) = 16 batches (15 batches of 10, and one final batch of 6).\n", + "\n", + "**Validation** typically happens after each epoch (as specified by evaluation_strategy=\"epoch\" which is often the default, or evaluation_steps=50 as explicitly set later in cell 60). The model is put into evaluation mode (no gradients calculated, no parameters updated), and its performance is measured on the separate validation set. This gives us an idea of how well the model is generalizing to data it hasn't been trained on during that epoch.\n", + "\n", + "So, to summarize: Learning (loss calculation, gradient adjustment) happens per batch. An epoch is a full pass over all batches. Validation is a separate check, usually done between epochs, to monitor generalization." + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "metadata": { + "id": "tiizmeIqZ_-w" + }, + "outputs": [], + "source": [ + "loader = DataLoader(\n", + " examples, batch_size=BATCH_SIZE\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "_vA8rzlX8XbT" + }, + "source": [ + "**Moving on!**\n", + "Next up, we'll prepare our loss function!\n", + "\n", + "Loss is an important part of training, fine-tuning, and more. If you want a deep dive on loss - you can check out our [event on loss!](https://www.youtube.com/watch?v=iB8FWR9aD5Q&t=8s).\n", + "\n", + "The core loss we're using today is called `MultipleNegativesRankingLoss` - you can find more information [here](https://github.com/UKPLab/sentence-transformers/blob/master/sentence_transformers/losses/MultipleNegativesRankingLoss.py).\n", + "\n", + "This is \"wrapped\" in `MatryoshkaLoss`, which you can read the implementation of [here](https://github.com/UKPLab/sentence-transformers/blob/master/sentence_transformers/losses/MatryoshkaLoss.py)." + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": { + "id": "Uga4nnBqlVeh" + }, + "outputs": [], + "source": [ + "from sentence_transformers.losses import MatryoshkaLoss, MultipleNegativesRankingLoss\n", + "\n", + "matryoshka_dimensions = [768, 512, 256, 128, 64]\n", + "inner_train_loss = MultipleNegativesRankingLoss(model)\n", + "train_loss = MatryoshkaLoss(\n", + " model, inner_train_loss, matryoshka_dims=matryoshka_dimensions\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "aJG4fOm66PHI" + }, + "source": [ + "##### πŸ—οΈ Activity #2:\n", + "\n", + "Both of these losses sound \"cool\", but what are they - exactly - under the hood?\n", + "\n", + "Why are these losses specifically doing? Please write a short summary of each loss.\n", + "\n", + "\n", + "Okay, here's a summary of the two loss functions, incorporating our discussion:\n", + "\n", + "#### Loss Functions Explained\n", + "\n", + "#### 1. MultipleNegativesRankingLoss (MNRL)\n", + "\n", + "* **Core Goal:** The primary objective of MNRL is to fine-tune the embedding model so that the vector embedding of a query is semantically closer to the embedding of its relevant context (the \"positive\" pair) than it is to the embeddings of irrelevant contexts (the \"negative\" pairs). In simpler terms, it pushes related items together and unrelated items apart in the embedding space.\n", + "* **Mechanism - In-Batch Negatives:** This loss function cleverly avoids the need to explicitly provide negative examples. When processing a batch of (query, positive_context) pairs, it uses an \"in-batch negative\" strategy. For a specific query (Query_A) in the batch:\n", + " * Its corresponding PositiveContext_A is treated as the single positive example.\n", + " * All *other* contexts present in that same batch (PositiveContext_B, PositiveContext_C, etc.) are implicitly treated as *negative* examples for Query_A.\n", + "* **Training Signal:** The loss is calculated based on how well the model ranks the similarity score of the positive pair (sim(Query_A, PositiveContext_A)) compared to the similarity scores of the negative pairs (sim(Query_A, PositiveContext_B), sim(Query_A, PositiveContext_C), etc.). The goal is to maximize the positive similarity relative to the negative similarities within that batch.\n", + "* **Batch Size Dependency:** The effectiveness of this strategy relies on the batch size. A larger batch provides more (and potentially harder) negative examples for each query, generally leading to more robust training.\n", + "\n", + "#### 2. MatryoshkaLoss\n", + "\n", + "* **Core Goal:** This loss function aims to train embeddings that are not only effective at their full dimensionality but also perform well when truncated to shorter lengths (like Russian nesting dolls). The primary motivation is **efficiency** in downstream applications like RAG – shorter embeddings require less storage, faster retrieval computations, and less bandwidth.\n", + "* **Mechanism - Learning Hierarchical Structure:** MatryoshkaLoss achieves this by incentivizing the model *during training* to learn a hierarchical representation within the embedding vector. It encourages the model to pack the most crucial, coarse-grained semantic information into the initial dimensions of the vector, adding progressively finer-grained details in subsequent dimensions.\n", + "* **Training Process:**\n", + " 1. It \"wraps\" an inner loss function (in our case, MultipleNegativesRankingLoss).\n", + " 2. For each batch, it calculates the inner loss multiple times: once using the full-dimension embeddings (e.g., 768), and then again using only the first N dimensions for each specified shorter length (e.g., first 512, first 256, first 128, first 64).\n", + " 3. These individual loss values (calculated at different dimensionalities) are combined, often using a weighted average. This combined loss reflects how well the embedding performs *at multiple levels of truncation*.\n", + " 4. The model's parameters are updated based on the gradient of this *combined* loss.\n", + " 5. By penalizing poor performance at shorter lengths *during the training loop*, this mechanism forces the model to organize information hierarchically, ensuring the truncated versions remain meaningful.\n", + "* **Outcome:** The result is an embedding model where the initial dimensions capture the most vital information. This *enables* practitioners, after training, to evaluate performance at different truncation levels (e.g., 768 vs. 512 vs. 256) and choose the best trade-off between accuracy and efficiency (storage/speed) for their specific RAG application. The MatryoshkaLoss during training is what makes this post-training choice possible and meaningful.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "IVi0k92rXkWg" + }, + "source": [ + "**A note on creating negative context examples.**\n", + " We created two queries for each context. So, it might incorrectly pick a context as negative for a query when it isn't. Gemini says this is not such a big deal:\n", + "\n", + "Primary Goal: The loss function's main goal is to ensure sim(q1, c) is higher than sim(q1, other_context) for all other contexts in the batch. Even if one of those \"other contexts\" happens to be the same text c (but paired with q2), the loss still pushes to maximize the similarity for the direct (q1, c) pairing relative to everything else.\n", + "\n", + "Different Queries: While the context c is the same text, q1 and q2 are (hopefully) different questions. The model learns to associate the specific semantics of q1 with c and the specific semantics of q2 with c. Treating c (paired with q2) as a negative for q1 encourages the model to differentiate why c is relevant specifically to q1 compared to other potential queries (like q2).\n", + "\n", + "Batch Size/Probability: With shuffling and reasonable batch sizes, the chances of both pairs derived from the exact same context landing in the same batch decrease, though it can certainly happen.\n", + "\n", + "In practice, this nuance of the in-batch negative strategy usually doesn't prevent the model from learning effectively, especially since the positive pairing signal is strong and consistent across batches." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "QKxRuXfH844c" + }, + "source": [ + "Now we can set-up our evaluator.\n", + "\n", + "> NOTE: Due to the formatting of our dataset - this is all we have to do!" + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": { + "id": "f0hAFwUyaHQG" + }, + "outputs": [], + "source": [ + "from sentence_transformers.evaluation import InformationRetrievalEvaluator\n", + "\n", + "corpus = val_dataset['corpus']\n", + "queries = val_dataset['questions']\n", + "relevant_docs = val_dataset['relevant_contexts']\n", + "\n", + "evaluator = InformationRetrievalEvaluator(queries, corpus, relevant_docs)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "MYfap_ct8-bU" + }, + "source": [ + "We'll train this model for 5 epochs, though you could increase this number if we had a significant amount more data." + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "metadata": { + "id": "svZG0pBHiQr6" + }, + "outputs": [], + "source": [ + "EPOCHS = 10" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "wxitWoNX9DwW" + }, + "source": [ + "It's training time!\n", + "\n", + "> NOTE: We're manually defining a warm-up period here - this is just to provide a smooth ramp into our training!" + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "metadata": { + "id": "u6Q7o02cXkWh", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 75 + }, + "outputId": "59d30625-5c0a-46c1-ad96-eef23f79552f" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "Changes to your `wandb` environment variables will be ignored because your `wandb` session has already started. For more information on how to modify your settings with `wandb.init()` arguments, please refer to the W&B docs." + ] + }, + "metadata": {} + }, + { + "output_type": "execute_result", + "data": { + "text/html": [ + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "execution_count": 60 + } + ], + "source": [ + "import wandb\n", + "wandb.init(mode=\"disabled\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "d_bZN8YEXkWh" + }, + "source": [ + "> NOTE: You may not see direct improvement during the training cycles - this is absolutely expected. We will verify performance later in the notebook." + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "metadata": { + "id": "aDhUHZY-iR09", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 + }, + "outputId": "59a10b8e-53a8-4222-e668-fc24ba024b2b" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n", + "
\n", + " \n", + " \n", + " [4460/4460 26:35, Epoch 10/10]\n", + "
\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
StepTraining LossValidation LossCosine Accuracy@1Cosine Accuracy@3Cosine Accuracy@5Cosine Accuracy@10Cosine Precision@1Cosine Precision@3Cosine Precision@5Cosine Precision@10Cosine Recall@1Cosine Recall@3Cosine Recall@5Cosine Recall@10Cosine Ndcg@10Cosine Mrr@10Cosine Map@100
50No logNo log0.5812500.7375000.7875000.8541670.5812500.2458330.1575000.0854170.5812500.7375000.7875000.8541670.7154750.6713580.678750
100No logNo log0.5854170.7375000.7854170.8541670.5854170.2458330.1570830.0854170.5854170.7375000.7854170.8541670.7171230.6735330.681246
150No logNo log0.5916670.7416670.7979170.8708330.5916670.2472220.1595830.0870830.5916670.7416670.7979170.8708330.7278490.6824770.689214
200No logNo log0.5895830.7416670.8125000.8729170.5895830.2472220.1625000.0872920.5895830.7416670.8125000.8729170.7269660.6806690.687202
250No logNo log0.5520830.7437500.8062500.8687500.5520830.2479170.1612500.0868750.5520830.7437500.8062500.8687500.7111830.6605960.666916
300No logNo log0.5666670.7583330.8166670.8791670.5666670.2527780.1633330.0879170.5666670.7583330.8166670.8791670.7238160.6739290.680497
350No logNo log0.5812500.7604170.8104170.8666670.5812500.2534720.1620830.0866670.5812500.7604170.8104170.8666670.7255070.6802270.687683
400No logNo log0.5500000.7541670.8020830.8687500.5500000.2513890.1604170.0868750.5500000.7541670.8020830.8687500.7119780.6615200.668568
446No logNo log0.5625000.7416670.8104170.8687500.5625000.2472220.1620830.0868750.5625000.7416670.8104170.8687500.7163710.6674560.674576
450No logNo log0.5645830.7416670.8125000.8708330.5645830.2472220.1625000.0870830.5645830.7416670.8125000.8708330.7169680.6676710.674605
5000.411300No log0.5833330.7562500.8145830.8750000.5833330.2520830.1629170.0875000.5833330.7562500.8145830.8750000.7293250.6825550.689423
5500.411300No log0.5416670.7333330.7979170.8625000.5416670.2444440.1595830.0862500.5416670.7333330.7979170.8625000.7031950.6520310.659616
6000.411300No log0.5687500.7354170.7916670.8604170.5687500.2451390.1583330.0860420.5687500.7354170.7916670.8604170.7130260.6661080.673959
6500.411300No log0.5375000.7187500.7833330.8562500.5375000.2395830.1566670.0856250.5375000.7187500.7833330.8562500.6966900.6457420.653585
7000.411300No log0.5583330.7312500.7958330.8708330.5583330.2437500.1591670.0870830.5583330.7312500.7958330.8708330.7132120.6631360.669668
7500.411300No log0.5583330.7312500.7875000.8666670.5583330.2437500.1575000.0866670.5583330.7312500.7875000.8666670.7089410.6588190.666458
8000.411300No log0.5645830.7250000.7979170.8687500.5645830.2416670.1595830.0868750.5645830.7250000.7979170.8687500.7106140.6605760.668073
8500.411300No log0.5750000.7229170.7895830.8645830.5750000.2409720.1579170.0864580.5750000.7229170.7895830.8645830.7147610.6673360.675218
8920.411300No log0.5604170.7145830.7875000.8770830.5604170.2381940.1575000.0877080.5604170.7145830.7875000.8770830.7117770.6598680.666605
9000.411300No log0.5479170.7208330.7895830.8770830.5479170.2402780.1579170.0877080.5479170.7208330.7895830.8770830.7072040.6535730.660255
9500.411300No log0.5520830.7375000.7979170.8729170.5520830.2458330.1595830.0872920.5520830.7375000.7979170.8729170.7110910.6595200.666047
10000.249300No log0.5666670.7250000.7958330.8833330.5666670.2416670.1591670.0883330.5666670.7250000.7958330.8833330.7178160.6658630.671928
10500.249300No log0.5479170.7166670.7937500.8812500.5479170.2388890.1587500.0881250.5479170.7166670.7937500.8812500.7102220.6562560.662691
11000.249300No log0.5437500.7354170.7916670.8770830.5437500.2451390.1583330.0877080.5437500.7354170.7916670.8770830.7090270.6556960.662875
11500.249300No log0.5500000.7270830.7875000.8750000.5500000.2423610.1575000.0875000.5500000.7270830.7875000.8750000.7098410.6575290.664717
12000.249300No log0.5708330.7250000.7895830.8645830.5708330.2416670.1579170.0864580.5708330.7250000.7895830.8645830.7136130.6659340.673769
12500.249300No log0.5437500.7312500.7958330.8729170.5437500.2437500.1591670.0872920.5437500.7312500.7958330.8729170.7059170.6529030.659984
13000.249300No log0.5479170.7354170.7979170.8854170.5479170.2451390.1595830.0885420.5479170.7354170.7979170.8854170.7131260.6585950.665058
13380.249300No log0.5604170.7333330.7937500.8770830.5604170.2444440.1587500.0877080.5604170.7333330.7937500.8770830.7166350.6656800.672952
13500.249300No log0.5562500.7250000.7895830.8645830.5562500.2416670.1579170.0864580.5562500.7250000.7895830.8645830.7076120.6576840.666011
14000.249300No log0.5604170.7250000.7812500.8750000.5604170.2416670.1562500.0875000.5604170.7250000.7812500.8750000.7101410.6584030.665706
14500.249300No log0.5562500.7270830.7791670.8750000.5562500.2423610.1558330.0875000.5562500.7270830.7791670.8750000.7094530.6572610.664491
15000.117300No log0.5354170.7208330.7750000.8750000.5354170.2402780.1550000.0875000.5354170.7208330.7750000.8750000.6990170.6436860.650962
15500.117300No log0.5354170.7333330.7812500.8729170.5354170.2444440.1562500.0872920.5354170.7333330.7812500.8729170.7015890.6472590.654535
16000.117300No log0.5000000.7020830.7708330.8666670.5000000.2340280.1541670.0866670.5000000.7020830.7708330.8666670.6801700.6210050.628848
16500.117300No log0.5250000.7208330.7791670.8583330.5250000.2402780.1558330.0858330.5250000.7208330.7791670.8583330.6916790.6384660.647026
17000.117300No log0.5437500.7312500.7791670.8625000.5437500.2437500.1558330.0862500.5437500.7312500.7791670.8625000.7027910.6519430.660411
17500.117300No log0.5270830.7208330.7625000.8770830.5270830.2402780.1525000.0877080.5270830.7208330.7625000.8770830.6976360.6411060.647865
17840.117300No log0.5416670.7166670.7708330.8687500.5416670.2388890.1541670.0868750.5416670.7166670.7708330.8687500.7015160.6486890.655936
18000.117300No log0.5250000.7166670.7729170.8687500.5250000.2388890.1545830.0868750.5250000.7166670.7729170.8687500.6949270.6398570.646763
18500.117300No log0.5270830.7145830.7666670.8583330.5270830.2381940.1533330.0858330.5270830.7145830.7666670.8583330.6894060.6359080.643928
19000.117300No log0.5395830.7104170.7687500.8583330.5395830.2368060.1537500.0858330.5395830.7104170.7687500.8583330.6926500.6403210.647986
19500.117300No log0.5291670.7104170.7854170.8625000.5291670.2368060.1570830.0862500.5291670.7104170.7854170.8625000.6929460.6390480.646445
20000.052000No log0.5291670.7229170.7833330.8729170.5291670.2409720.1566670.0872920.5291670.7229170.7833330.8729170.6983720.6430360.649898
20500.052000No log0.5458330.7354170.7833330.8750000.5458330.2451390.1566670.0875000.5458330.7354170.7833330.8750000.7067910.6536020.660305
21000.052000No log0.5541670.7416670.7833330.8687500.5541670.2472220.1566670.0868750.5541670.7416670.7833330.8687500.7100830.6597110.666906
21500.052000No log0.5291670.7250000.7791670.8666670.5291670.2416670.1558330.0866670.5291670.7250000.7791670.8666670.6962350.6420790.649176
22000.052000No log0.5354170.7208330.7791670.8604170.5354170.2402780.1558330.0860420.5354170.7208330.7791670.8604170.6968060.6446840.652309
22300.052000No log0.5479170.7333330.7833330.8750000.5479170.2444440.1566670.0875000.5479170.7333330.7833330.8750000.7083070.6554910.662010
22500.052000No log0.5354170.7229170.7895830.8729170.5354170.2409720.1579170.0872920.5354170.7229170.7895830.8729170.7025760.6483660.655110
23000.052000No log0.5375000.7187500.7833330.8750000.5375000.2395830.1566670.0875000.5375000.7187500.7833330.8750000.7023270.6476830.654344
23500.052000No log0.5479170.7104170.7812500.8687500.5479170.2368060.1562500.0868750.5479170.7104170.7812500.8687500.7032080.6509600.658328
24000.052000No log0.5458330.7208330.7854170.8770830.5458330.2402780.1570830.0877080.5458330.7208330.7854170.8770830.7087490.6555790.662159
24500.052000No log0.5583330.7291670.7937500.8875000.5583330.2430560.1587500.0887500.5583330.7291670.7937500.8875000.7170750.6634860.669323
25000.029100No log0.5458330.7250000.7875000.8687500.5458330.2416670.1575000.0868750.5458330.7250000.7875000.8687500.7055650.6536190.660961
25500.029100No log0.5291670.7208330.7875000.8666670.5291670.2402780.1575000.0866670.5291670.7208330.7875000.8666670.6969120.6428630.650055
26000.029100No log0.5354170.7291670.7854170.8687500.5354170.2430560.1570830.0868750.5354170.7291670.7854170.8687500.6999860.6463910.653660
26500.029100No log0.5395830.7250000.7812500.8645830.5395830.2416670.1562500.0864580.5395830.7250000.7812500.8645830.7012060.6493110.656942
26760.029100No log0.5437500.7291670.7854170.8708330.5437500.2430560.1570830.0870830.5437500.7291670.7854170.8708330.7067800.6547020.661849
27000.029100No log0.5458330.7270830.7833330.8729170.5458330.2423610.1566670.0872920.5458330.7270830.7833330.8729170.7084470.6563310.663414
27500.029100No log0.5270830.7250000.7833330.8770830.5270830.2416670.1566670.0877080.5270830.7250000.7833330.8770830.7002440.6442200.650985
28000.029100No log0.5375000.7229170.7750000.8708330.5375000.2409720.1550000.0870830.5375000.7229170.7750000.8708330.7014690.6478410.655123
28500.029100No log0.5375000.7166670.7770830.8645830.5375000.2388890.1554170.0864580.5375000.7166670.7770830.8645830.6974480.6444900.652025
29000.029100No log0.5229170.7125000.7729170.8687500.5229170.2375000.1545830.0868750.5229170.7125000.7729170.8687500.6903000.6340350.641066
29500.029100No log0.5312500.7166670.7750000.8666670.5312500.2388890.1550000.0866670.5312500.7166670.7750000.8666670.6942530.6397920.646711
30000.030400No log0.5312500.7166670.7729170.8645830.5312500.2388890.1545830.0864580.5312500.7166670.7729170.8645830.6938370.6399030.647082
30500.030400No log0.5375000.7145830.7750000.8687500.5375000.2381940.1550000.0868750.5375000.7145830.7750000.8687500.6987920.6450700.652320
31000.030400No log0.5416670.7145830.7750000.8666670.5416670.2381940.1550000.0866670.5416670.7145830.7750000.8666670.7001200.6474500.654701
31220.030400No log0.5437500.7166670.7812500.8645830.5437500.2388890.1562500.0864580.5437500.7166670.7812500.8645830.7004470.6484390.655902
31500.030400No log0.5395830.7208330.7833330.8708330.5395830.2402780.1566670.0870830.5395830.7208330.7833330.8708330.7003740.6465320.653362
32000.030400No log0.5312500.7270830.7750000.8666670.5312500.2423610.1550000.0866670.5312500.7270830.7750000.8666670.6949900.6405290.647798
32500.030400No log0.5416670.7187500.7833330.8583330.5416670.2395830.1566670.0858330.5416670.7187500.7833330.8583330.6975130.6463080.654538
33000.030400No log0.5312500.7104170.7729170.8625000.5312500.2368060.1545830.0862500.5312500.7104170.7729170.8625000.6926720.6389340.646634
33500.030400No log0.5375000.7208330.7812500.8645830.5375000.2402780.1562500.0864580.5375000.7208330.7812500.8645830.6973940.6443830.652033
34000.030400No log0.5312500.7166670.7708330.8604170.5312500.2388890.1541670.0860420.5312500.7166670.7708330.8604170.6927970.6396430.647619
34500.030400No log0.5333330.7166670.7729170.8645830.5333330.2388890.1545830.0864580.5333330.7166670.7729170.8645830.6963490.6430840.650747
35000.020700No log0.5291670.7208330.7708330.8604170.5291670.2402780.1541670.0860420.5291670.7208330.7708330.8604170.6929510.6397340.647725
35500.020700No log0.5333330.7166670.7708330.8625000.5333330.2388890.1541670.0862500.5333330.7166670.7708330.8625000.6937760.6403990.648185
35680.020700No log0.5354170.7250000.7750000.8604170.5354170.2416670.1550000.0860420.5354170.7250000.7750000.8604170.6951940.6427140.650630
36000.020700No log0.5312500.7208330.7708330.8604170.5312500.2402780.1541670.0860420.5312500.7208330.7708330.8604170.6935570.6405320.648436
36500.020700No log0.5500000.7270830.7833330.8625000.5500000.2423610.1566670.0862500.5500000.7270830.7833330.8625000.7045280.6542790.661906
37000.020700No log0.5333330.7187500.7750000.8625000.5333330.2395830.1550000.0862500.5333330.7187500.7750000.8625000.6964280.6435300.651187
37500.020700No log0.5291670.7166670.7791670.8604170.5291670.2388890.1558330.0860420.5291670.7166670.7791670.8604170.6940390.6409850.648870
38000.020700No log0.5395830.7187500.7791670.8645830.5395830.2395830.1558330.0864580.5395830.7187500.7791670.8645830.6989240.6464410.653950
38500.020700No log0.5458330.7187500.7791670.8604170.5458330.2395830.1558330.0860420.5458330.7187500.7791670.8604170.6990910.6480010.655906
39000.020700No log0.5437500.7187500.7770830.8604170.5437500.2395830.1554170.0860420.5437500.7187500.7770830.8604170.6991560.6479600.655809
39500.020700No log0.5437500.7187500.7791670.8604170.5437500.2395830.1558330.0860420.5437500.7187500.7791670.8604170.6997480.6487040.656758
40000.022800No log0.5458330.7166670.7791670.8604170.5458330.2388890.1558330.0860420.5458330.7166670.7791670.8604170.7004270.6496240.657619
40140.022800No log0.5437500.7166670.7770830.8604170.5437500.2388890.1554170.0860420.5437500.7166670.7770830.8604170.6992680.6480990.656181
40500.022800No log0.5479170.7187500.7833330.8645830.5479170.2395830.1566670.0864580.5479170.7187500.7833330.8645830.7029940.6517820.659478
41000.022800No log0.5479170.7250000.7833330.8645830.5479170.2416670.1566670.0864580.5479170.7250000.7833330.8645830.7039590.6529270.660659
41500.022800No log0.5500000.7187500.7854170.8729170.5500000.2395830.1570830.0872920.5500000.7187500.7854170.8729170.7071320.6548020.661834
42000.022800No log0.5479170.7187500.7854170.8729170.5479170.2395830.1570830.0872920.5479170.7187500.7854170.8729170.7058500.6531080.660147
42500.022800No log0.5479170.7229170.7854170.8687500.5479170.2409720.1570830.0868750.5479170.7229170.7854170.8687500.7048670.6529720.660331
43000.022800No log0.5458330.7229170.7875000.8666670.5458330.2409720.1575000.0866670.5458330.7229170.7875000.8666670.7037400.6519780.659578
43500.022800No log0.5458330.7229170.7854170.8666670.5458330.2409720.1570830.0866670.5458330.7229170.7854170.8666670.7037260.6520150.659592
44000.022800No log0.5458330.7229170.7854170.8666670.5458330.2409720.1570830.0866670.5458330.7229170.7854170.8666670.7037730.6520640.659627
44500.022800No log0.5458330.7250000.7833330.8666670.5458330.2416670.1566670.0866670.5458330.7250000.7833330.8666670.7035810.6518210.659381
44600.022800No log0.5458330.7250000.7833330.8666670.5458330.2416670.1566670.0866670.5458330.7250000.7833330.8666670.7035810.6518210.659381

" + ] + }, + "metadata": {} + } + ], + "source": [ + "warmup_steps = int(len(loader) * EPOCHS * 0.1)\n", + "\n", + "model.fit(\n", + " train_objectives=[(loader, train_loss)],\n", + " epochs=EPOCHS,\n", + " warmup_steps=warmup_steps,\n", + " output_path='finetuned_arctic_ft',\n", + " show_progress_bar=True,\n", + " evaluator=evaluator,\n", + " evaluation_steps=50\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "metadata": { + "id": "b3iwclvyRD8L", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 17, + "referenced_widgets": [ + "c1bdc2bee22a4028a1622f2fa207369f", + "f93934fcb1f74359a786503cb2771a53", + "e76ee26c1e0540a6855b4c2a09143084", + "740ec743ea6742549d54eb1d1cbd2e20", + "482613b5c1bb4b53b4052d050dc78638", + "589bf123f49b45f5b1ad5968e8a6abea", + "b68f3642a0d54b99b10f7a63c793f1ac", + "4d12dd2fd5964d34ab9e83d0ff01570e", + "480ecbe8908644ff957102c25064025e", + "3487ade73bd8403186ad7af1b4eb503a", + "62edd19b390943aab76f8f717dc3de66", + "e3fdb79108064ecdab361420f97a8e4e", + "a05930373152409c956ce7a33cbd489b", + "a99da24492b049dc91c6c87c73047f35", + "416d61f2e90f4d70abc732f15e9b289d", + "f177c091937f406488825e8569de3390", + "8d72c745b53747ddaafb0fc1bfbf799f", + "340204007ee847e78a882b2c204a730f", + "2e4ea07138c6451aa8bf003ee195cd0d", + "30fc4c21dfea4c76b52ca610759ba2bd" + ] + }, + "outputId": "f0fce637-c9dc-4cc1-829f-de4376b71d81" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "VBox(children=(HTML(value='

NOTE: We're assuming 1 correct document in a \"hit\"." + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "metadata": { + "id": "0713_3cowX4q" + }, + "outputs": [], + "source": [ + "def evaluate_openai(\n", + " dataset,\n", + " embed_model,\n", + " top_k=5,\n", + " verbose=False,\n", + "):\n", + " corpus = dataset['corpus']\n", + " questions = dataset['questions']\n", + " relevant_docs = dataset['relevant_contexts']\n", + " documents = [Document(page_content=content, metadata={\"id\": doc_id}) for doc_id, content in corpus.items()]\n", + " vectorstore = FAISS.from_documents(documents, embed_model)\n", + "\n", + " retriever = vectorstore.as_retriever(search_kwargs={\"k\": top_k})\n", + "\n", + " eval_results = []\n", + " for id, question in tqdm.tqdm(questions.items()):\n", + " retrieved_nodes = retriever.invoke(question)\n", + " retrieved_ids = [node.metadata[\"id\"] for node in retrieved_nodes]\n", + " expected_id = relevant_docs[id][0]\n", + " is_hit = expected_id in retrieved_ids\n", + " eval_results.append({\"id\": id, \"question\": question, \"expected_id\": expected_id, \"is_hit\": is_hit})\n", + "\n", + " return eval_results" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "hOr49m4O9lxY" + }, + "source": [ + "All that's left to do is evaluate, we'll evaluate our model against:\n", + "\n", + "1. OpenAI's closed source `text-embedding-3-small`\n", + "2. The base non-fine-tuned version of `Snowflake/snowflake-arctic-embed-l`.\n", + "\n", + "Let's see how it stacks up!" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ijaeYpf593IW" + }, + "source": [ + "### `text-embedding-3-small`" + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "metadata": { + "id": "kyY3PztaxnU3", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "7be168cb-2932-4819-de13-e64e007d8658" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 480/480 [03:10<00:00, 2.52it/s]\n" + ] + } + ], + "source": [ + "te3_openai = OpenAIEmbeddings(model=\"text-embedding-3-small\")\n", + "te3_results = evaluate_openai(test_dataset, te3_openai)" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "metadata": { + "id": "kkyW90TCxx_i" + }, + "outputs": [], + "source": [ + "te3_results_df = pd.DataFrame(te3_results)" + ] + }, + { + "cell_type": "code", + "execution_count": 71, + "metadata": { + "id": "MscVRdNCylJ-", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "c21d2a79-7966-4dc6-fce5-ed5849243569" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "np.float64(0.90625)" + ] + }, + "metadata": {}, + "execution_count": 71 + } + ], + "source": [ + "te3_hit_rate = te3_results_df[\"is_hit\"].mean()\n", + "te3_hit_rate" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4Ra-mh0L96dQ" + }, + "source": [ + "### `Snowflake/snowflake-arctic-embed-l` (base)" + ] + }, + { + "cell_type": "code", + "execution_count": 72, + "metadata": { + "id": "OEskxwvFypHe", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "86a77f76-9be8-40d6-938c-0d8abb7ca33d" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 480/480 [00:10<00:00, 46.80it/s]\n" + ] + } + ], + "source": [ + "from langchain_huggingface import HuggingFaceEmbeddings\n", + "\n", + "huggingface_embeddings = HuggingFaceEmbeddings(model_name=\"Snowflake/snowflake-arctic-embed-l\")\n", + "arctic_embed_m_results = evaluate_openai(test_dataset, huggingface_embeddings)" + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "metadata": { + "id": "KlKgiXTWzMTg" + }, + "outputs": [], + "source": [ + "arctic_embed_m_results_df = pd.DataFrame(arctic_embed_m_results)" + ] + }, + { + "cell_type": "code", + "execution_count": 74, + "metadata": { + "id": "zV5vJWrJzOhc", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "2b530138-dc21-4375-d34f-9f624cdd4dd1" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "np.float64(0.6145833333333334)" + ] + }, + "metadata": {}, + "execution_count": 74 + } + ], + "source": [ + "arctic_embed_m_hit_rate = arctic_embed_m_results_df[\"is_hit\"].mean()\n", + "arctic_embed_m_hit_rate" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "lcR3-0s19_lu" + }, + "source": [ + "### `Snowflake/snowflake-arctic-embed-l` (fine-tuned)" + ] + }, + { + "cell_type": "code", + "execution_count": 75, + "metadata": { + "id": "Ilse1LduzP1i", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "2e585653-641c-4b0c-a0a2-e2170d25d7ee" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "Some weights of BertModel were not initialized from the model checkpoint at finetuned_arctic_ft and are newly initialized: ['pooler.dense.bias', 'pooler.dense.weight']\n", + "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n", + "100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 480/480 [00:10<00:00, 47.01it/s]\n" + ] + } + ], + "source": [ + "finetune_embeddings = HuggingFaceEmbeddings(model_name=\"finetuned_arctic_ft\")\n", + "finetune_results = evaluate_openai(test_dataset, finetune_embeddings)" + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "metadata": { + "id": "xxhZPqkNzZlh" + }, + "outputs": [], + "source": [ + "finetune_results_df = pd.DataFrame(finetune_results)" + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "metadata": { + "id": "4thAK2BXzaj6", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "ac33765b-75cb-46ef-cc57-c992f883867c" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "np.float64(0.9104166666666667)" + ] + }, + "metadata": {}, + "execution_count": 77 + } + ], + "source": [ + "finetune_hit_rate = finetune_results_df[\"is_hit\"].mean()\n", + "finetune_hit_rate" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "iegFM209mBk3" + }, + "source": [ + "## Task 1: Vibe Checking the RAG Pipeline\n", + "\n", + "We're going to use our RAG pipeline to vibe check on some common phrases now that we've modified it!" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Xzg0AA5krgR4" + }, + "source": [ + "### Creating New Chunks\n", + "\n", + "In order to try and evaluate our system more fairly, let's create new chunks that we will use to create our Vector Store." + ] + }, + { + "cell_type": "code", + "execution_count": 78, + "metadata": { + "id": "KwQ2_LqNr0Tw", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "997e9154-c3ca-4d4a-cdff-db58222cfa8e" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 18/18 [00:00<00:00, 1555.81it/s]\n" + ] + } + ], + "source": [ + "from langchain_huggingface import HuggingFaceEmbeddings\n", + "from sentence_transformers import SentenceTransformer\n", + "\n", + "\n", + "text_splitter = RecursiveCharacterTextSplitter(\n", + " chunk_size = 600,\n", + " chunk_overlap = 50,\n", + " length_function = len\n", + ")\n", + "\n", + "training_documents = text_splitter.split_documents(text_loader.load())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "gIdxahHXpP-c" + }, + "source": [ + "### Base Chain\n", + "\n", + "We'll start by constructing our base chain, which will use the untrained retrieval model." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "bOsxIXpNpWC2" + }, + "source": [ + "#### R - Retrieval" + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "metadata": { + "id": "azIGIKYfmNCT" + }, + "outputs": [], + "source": [ + "from langchain_community.vectorstores import FAISS\n", + "\n", + "huggingface_embeddings = HuggingFaceEmbeddings(model_name=\"Snowflake/snowflake-arctic-embed-l\")\n", + "base_vectorstore = FAISS.from_documents(training_documents, huggingface_embeddings)\n", + "base_retriever = base_vectorstore.as_retriever(search_kwargs={\"k\": 6})" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "l-1nVZ0KpX5N" + }, + "source": [ + "#### A - Augmented" + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "metadata": { + "id": "G10Fr-aKojeA" + }, + "outputs": [], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "RAG_PROMPT = \"\"\"\\\n", + "Given a provided context and a question, you must answer the question. If you do not know the answer, you must state that you do not know.\n", + "\n", + "Context:\n", + "{context}\n", + "\n", + "Question:\n", + "{question}\n", + "\n", + "Answer:\n", + "\"\"\"\n", + "\n", + "rag_prompt_template = ChatPromptTemplate.from_template(RAG_PROMPT)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Euq6RQEopZvD" + }, + "source": [ + "#### G - Generation" + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "metadata": { + "id": "5-mfbbrypMHG" + }, + "outputs": [], + "source": [ + "rag_llm = ChatOpenAI(\n", + " model=\"gpt-4.1-nano\",\n", + " temperature=0\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "wQ2p4mnUpbYY" + }, + "source": [ + "#### RAG - LCEL RAG Pipeline" + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "metadata": { + "id": "ssuR-LaboyGq" + }, + "outputs": [], + "source": [ + "from operator import itemgetter\n", + "from langchain_core.output_parsers import StrOutputParser\n", + "from langchain_core.runnables import RunnablePassthrough, RunnableParallel\n", + "\n", + "base_rag_chain = (\n", + " {\"context\": itemgetter(\"question\") | base_retriever, \"question\": itemgetter(\"question\")}\n", + " | RunnablePassthrough.assign(context=itemgetter(\"context\"))\n", + " | {\"response\": rag_prompt_template | rag_llm | StrOutputParser(), \"context\": itemgetter(\"context\")}\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "metadata": { + "id": "emm6WbB9pfKt", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "3ae93884-bae6-4f95-ce19-8dc2cf51a853" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "'The provided context does not include a definition or explanation of what an agent is.'" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 83 + } + ], + "source": [ + "base_rag_chain.invoke({\"question\" : \"What is an agent?\"})[\"response\"]" + ] + }, + { + "cell_type": "code", + "execution_count": 84, + "metadata": { + "id": "mUOrd0OBprAq", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "513cb214-81c1-41ac-a9b9-b1608e522144" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "'I do not know.'" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 84 + } + ], + "source": [ + "base_rag_chain.invoke({\"question\" : \"Who has produced better models than GPT-3?\"})[\"response\"]" + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "metadata": { + "id": "OnfuFl59py7I", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "b548a92f-6690-4dca-a347-73f723491250" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "'I do not know.'" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 85 + } + ], + "source": [ + "base_rag_chain.invoke({\"question\" : \"What is the laziest time of the year for AI?\"})[\"response\"]" + ] + }, + { + "cell_type": "code", + "execution_count": 86, + "metadata": { + "id": "-NmqwHBDqTZ8", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "cc50dda7-caaa-45df-b8f6-3e06d0a035b4" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "'I do not know.'" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 86 + } + ], + "source": [ + "base_rag_chain.invoke({\"question\" : \"What is the largest model that Simon has run on his phone?\"})[\"response\"]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "SqNS0UJAp3lC" + }, + "source": [ + "### Fine-tuned Embedding Model\n", + "\n", + "Now let's rebuild our RAG chain with the Fine-tuned model - the only component we need to change is our `FAISS` vectorstore!" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "el_4axkbKxsR" + }, + "source": [ + "**To avoid re-running the fine tuning when I reload this into Google Colab, I am getting the fine tuned model I pushed to HuggingFace**" + ] + }, + { + "cell_type": "code", + "execution_count": 87, + "metadata": { + "id": "ihO7tP6mqATy", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "fdbdb4b7-355b-4cd5-bb09-32470cab7352" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "Some weights of BertModel were not initialized from the model checkpoint at finetuned_arctic_ft and are newly initialized: ['pooler.dense.bias', 'pooler.dense.weight']\n", + "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n" + ] + } + ], + "source": [ + "# If you just ran the fine tuning, you can use the following line to load the fine tuned model.\n", + "finetune_embeddings = HuggingFaceEmbeddings(model_name=\"finetuned_arctic_ft\")\n", + "\n", + "# Otherwise, you can use the following line to load the fine tuned model from HuggingFace. From your old runs.\n", + "# finetune_embeddings = HuggingFaceEmbeddings(model_name=\"geetach/legal-ft-450c1026-6554-476b-96f1-34f426f777c8\")\n", + "\n", + "finetune_vectorstore = FAISS.from_documents(training_documents, finetune_embeddings)\n", + "finetune_retriever = finetune_vectorstore.as_retriever(search_kwargs={\"k\": 6})" + ] + }, + { + "cell_type": "code", + "execution_count": 88, + "metadata": { + "id": "1_cIFvWzqKGY" + }, + "outputs": [], + "source": [ + "finetune_rag_chain = (\n", + " {\"context\": itemgetter(\"question\") | finetune_retriever, \"question\": itemgetter(\"question\")}\n", + " | RunnablePassthrough.assign(context=itemgetter(\"context\"))\n", + " | {\"response\": rag_prompt_template | rag_llm | StrOutputParser(), \"context\": itemgetter(\"context\")}\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 89, + "metadata": { + "id": "OJmRHJF2qNgj", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "39d18699-85d9-4f72-e3a2-16ee43301d18" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "'The provided context does not include a definition or explanation of what an Agent is. Therefore, I do not know.'" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 89 + } + ], + "source": [ + "finetune_rag_chain.invoke({\"question\" : \"What is an Agent?\"})[\"response\"]" + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "metadata": { + "id": "EnK-c2ugqPPh", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "ce62a1ce-54c9-4363-9bb7-231e8900cd81" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "'I do not know.'" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 90 + } + ], + "source": [ + "finetune_rag_chain.invoke({\"question\" : \"Who has produced better models than GPT-3?\"})[\"response\"]" + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "metadata": { + "id": "83hssg1AWozc", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "e8b8f225-70ef-4e18-a6d5-b5330ffcffa2" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "'I do not know.'" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 91 + } + ], + "source": [ + "finetune_rag_chain.invoke({\"question\" : \"What is the laziest time of the year for AI?\"})[\"response\"]" + ] + }, + { + "cell_type": "code", + "execution_count": 92, + "metadata": { + "id": "rsHmGeFbqRET", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "375de15c-b1c6-4949-cef0-dd7f591d9d31" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "'I do not know.'" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 92 + } + ], + "source": [ + "finetune_rag_chain.invoke({\"question\" : \"What is the largest model that Simon has run on his phone?\"})[\"response\"]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "jDgD8seY_I3W" + }, + "source": [ + "#### ❓Question #2:\n", + "\n", + "Which LCEL RAG Chain do you think answered the questions better, and why?\n", + "\n", + " The fine tuned rag chain!! Here is why:\n", + "\n", + " **Specificity and Contextual Relevance**\n", + "\n", + " - For questions like \"What is an agent?\" or \"What is the laziest time of the year for AI?\", the fine-tuned chain is more likely to retrieve the specific passages from Simon Willison's blogs discussing these topics in his specific context (e.g., LLM agents, the ChatGPT \"lazy\" period).\n", + " - The base model, relying on more general embeddings, might retrieve less relevant chunks or provide more generic answers that miss the nuances present in the source documents.\n", + " - Similarly, for specific factual questions like \"What is the largest model that Simon has run on his phone?\", the fine-tuned embeddings are much more likely to correctly identify and retrieve the exact text containing this information, whereas the base model might struggle.\n", + "\n", + " **More on evaluation**\n", + " - If you look at the hit rate, we can see that the fine tuned retireiver outperformed the base retriever. - its better at finding the correct chunk for a given synhtetic question.\n", + " -- Later, I did a langsmith evaluation of these two rag chains and the fine tuned rag chain achieved better scores for both correctness and helpfulness (screenshot at the end of this notebook!)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "WCbq1sZArIx4" + }, + "source": [ + "## Task 2: RAGAS Evaluation\n", + "\n", + "It's great to have some idea of how our system is doing based on vibe-checks, but let's use RAGAS to provide more insight info. on how things are improving!\n", + "\n", + "> NOTE: Please recreate *exactly* the RAGAS process we used to evaluate RAG, baselining with the default retriever, and then comparing the new retriever. The includes the Synthetic Data Generation steps." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Rd_CmvttXkWj" + }, + "source": [ + "Ok, so we already have the base_rag_chain and finetune_rag_chain. We want to now generate test data via ragas and then evaluate the two rag chains." + ] + }, + { + "cell_type": "code", + "execution_count": 93, + "metadata": { + "id": "jq880DtHk9pX", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "2a26b2db-6585-435e-8cd8-9d136e2134e5" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/175.7 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m175.7/175.7 kB\u001b[0m \u001b[31m11.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/45.5 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.5/45.5 kB\u001b[0m \u001b[31m3.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/71.1 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m71.1/71.1 kB\u001b[0m \u001b[31m7.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting unstructured\n", + " Downloading unstructured-0.17.2-py3-none-any.whl.metadata (24 kB)\n", + "Requirement already satisfied: chardet in /usr/local/lib/python3.11/dist-packages (from unstructured) (5.2.0)\n", + "Collecting filetype (from unstructured)\n", + " Downloading filetype-1.2.0-py2.py3-none-any.whl.metadata (6.5 kB)\n", + "Collecting python-magic (from unstructured)\n", + " Downloading python_magic-0.4.27-py2.py3-none-any.whl.metadata (5.8 kB)\n", + "Requirement already satisfied: lxml in /usr/local/lib/python3.11/dist-packages (from unstructured) (5.4.0)\n", + "Requirement already satisfied: nltk in /usr/local/lib/python3.11/dist-packages (from unstructured) (3.9.1)\n", + "Requirement already satisfied: requests in /usr/local/lib/python3.11/dist-packages (from unstructured) (2.32.3)\n", + "Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.11/dist-packages (from unstructured) (4.13.4)\n", + "Collecting emoji (from unstructured)\n", + " Downloading emoji-2.14.1-py3-none-any.whl.metadata (5.7 kB)\n", + "Requirement already satisfied: dataclasses-json in /usr/local/lib/python3.11/dist-packages (from unstructured) (0.6.7)\n", + "Collecting python-iso639 (from unstructured)\n", + " Downloading python_iso639-2025.2.18-py3-none-any.whl.metadata (14 kB)\n", + "Collecting langdetect (from unstructured)\n", + " Downloading langdetect-1.0.9.tar.gz (981 kB)\n", + "\u001b[2K \u001b[90m━━��━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m981.5/981.5 kB\u001b[0m \u001b[31m39.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + "Requirement already satisfied: numpy in /usr/local/lib/python3.11/dist-packages (from unstructured) (2.0.2)\n", + "Collecting rapidfuzz (from unstructured)\n", + " Downloading rapidfuzz-3.13.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)\n", + "Collecting backoff (from unstructured)\n", + " Downloading backoff-2.2.1-py3-none-any.whl.metadata (14 kB)\n", + "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.11/dist-packages (from unstructured) (4.13.2)\n", + "Collecting unstructured-client (from unstructured)\n", + " Downloading unstructured_client-0.34.0-py3-none-any.whl.metadata (21 kB)\n", + "Requirement already satisfied: wrapt in /usr/local/lib/python3.11/dist-packages (from unstructured) (1.17.2)\n", + "Requirement already satisfied: tqdm in /usr/local/lib/python3.11/dist-packages (from unstructured) (4.67.1)\n", + "Requirement already satisfied: psutil in /usr/local/lib/python3.11/dist-packages (from unstructured) (5.9.5)\n", + "Collecting python-oxmsg (from unstructured)\n", + " Downloading python_oxmsg-0.0.2-py3-none-any.whl.metadata (5.0 kB)\n", + "Requirement already satisfied: html5lib in /usr/local/lib/python3.11/dist-packages (from unstructured) (1.1)\n", + "Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.11/dist-packages (from beautifulsoup4->unstructured) (2.7)\n", + "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /usr/local/lib/python3.11/dist-packages (from dataclasses-json->unstructured) (3.26.1)\n", + "Requirement already satisfied: typing-inspect<1,>=0.4.0 in /usr/local/lib/python3.11/dist-packages (from dataclasses-json->unstructured) (0.9.0)\n", + "Requirement already satisfied: six>=1.9 in /usr/local/lib/python3.11/dist-packages (from html5lib->unstructured) (1.17.0)\n", + "Requirement already satisfied: webencodings in /usr/local/lib/python3.11/dist-packages (from html5lib->unstructured) (0.5.1)\n", + "Requirement already satisfied: click in /usr/local/lib/python3.11/dist-packages (from nltk->unstructured) (8.1.8)\n", + "Requirement already satisfied: joblib in /usr/local/lib/python3.11/dist-packages (from nltk->unstructured) (1.5.0)\n", + "Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.11/dist-packages (from nltk->unstructured) (2024.11.6)\n", + "Collecting olefile (from python-oxmsg->unstructured)\n", + " Downloading olefile-0.47-py2.py3-none-any.whl.metadata (9.7 kB)\n", + "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests->unstructured) (3.4.2)\n", + "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests->unstructured) (3.10)\n", + "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests->unstructured) (2.4.0)\n", + "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests->unstructured) (2025.4.26)\n", + "Collecting aiofiles>=24.1.0 (from unstructured-client->unstructured)\n", + " Downloading aiofiles-24.1.0-py3-none-any.whl.metadata (10 kB)\n", + "Requirement already satisfied: cryptography>=3.1 in /usr/local/lib/python3.11/dist-packages (from unstructured-client->unstructured) (43.0.3)\n", + "Collecting eval-type-backport>=0.2.0 (from unstructured-client->unstructured)\n", + " Downloading eval_type_backport-0.2.2-py3-none-any.whl.metadata (2.2 kB)\n", + "Requirement already satisfied: httpx>=0.27.0 in /usr/local/lib/python3.11/dist-packages (from unstructured-client->unstructured) (0.28.1)\n", + "Requirement already satisfied: nest-asyncio>=1.6.0 in /usr/local/lib/python3.11/dist-packages (from unstructured-client->unstructured) (1.6.0)\n", + "Requirement already satisfied: pydantic>=2.11.2 in /usr/local/lib/python3.11/dist-packages (from unstructured-client->unstructured) (2.11.4)\n", + "Collecting pypdf>=4.0 (from unstructured-client->unstructured)\n", + " Downloading pypdf-5.5.0-py3-none-any.whl.metadata (7.2 kB)\n", + "Requirement already satisfied: requests-toolbelt>=1.0.0 in /usr/local/lib/python3.11/dist-packages (from unstructured-client->unstructured) (1.0.0)\n", + "Requirement already satisfied: typing-inspection>=0.4.0 in /usr/local/lib/python3.11/dist-packages (from unstructured-client->unstructured) (0.4.0)\n", + "Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.11/dist-packages (from cryptography>=3.1->unstructured-client->unstructured) (1.17.1)\n", + "Requirement already satisfied: anyio in /usr/local/lib/python3.11/dist-packages (from httpx>=0.27.0->unstructured-client->unstructured) (4.9.0)\n", + "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.11/dist-packages (from httpx>=0.27.0->unstructured-client->unstructured) (1.0.9)\n", + "Requirement already satisfied: h11>=0.16 in /usr/local/lib/python3.11/dist-packages (from httpcore==1.*->httpx>=0.27.0->unstructured-client->unstructured) (0.16.0)\n", + "Requirement already satisfied: packaging>=17.0 in /usr/local/lib/python3.11/dist-packages (from marshmallow<4.0.0,>=3.18.0->dataclasses-json->unstructured) (24.2)\n", + "Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.11/dist-packages (from pydantic>=2.11.2->unstructured-client->unstructured) (0.7.0)\n", + "Requirement already satisfied: pydantic-core==2.33.2 in /usr/local/lib/python3.11/dist-packages (from pydantic>=2.11.2->unstructured-client->unstructured) (2.33.2)\n", + "Requirement already satisfied: mypy-extensions>=0.3.0 in /usr/local/lib/python3.11/dist-packages (from typing-inspect<1,>=0.4.0->dataclasses-json->unstructured) (1.1.0)\n", + "Requirement already satisfied: pycparser in /usr/local/lib/python3.11/dist-packages (from cffi>=1.12->cryptography>=3.1->unstructured-client->unstructured) (2.22)\n", + "Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.11/dist-packages (from anyio->httpx>=0.27.0->unstructured-client->unstructured) (1.3.1)\n", + "Downloading unstructured-0.17.2-py3-none-any.whl (1.8 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m84.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading backoff-2.2.1-py3-none-any.whl (15 kB)\n", + "Downloading emoji-2.14.1-py3-none-any.whl (590 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m590.6/590.6 kB\u001b[0m \u001b[31m50.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading filetype-1.2.0-py2.py3-none-any.whl (19 kB)\n", + "Downloading python_iso639-2025.2.18-py3-none-any.whl (167 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m167.6/167.6 kB\u001b[0m \u001b[31m17.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading python_magic-0.4.27-py2.py3-none-any.whl (13 kB)\n", + "Downloading python_oxmsg-0.0.2-py3-none-any.whl (31 kB)\n", + "Downloading rapidfuzz-3.13.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m105.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading unstructured_client-0.34.0-py3-none-any.whl (189 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m189.4/189.4 kB\u001b[0m \u001b[31m18.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading aiofiles-24.1.0-py3-none-any.whl (15 kB)\n", + "Downloading eval_type_backport-0.2.2-py3-none-any.whl (5.8 kB)\n", + "Downloading pypdf-5.5.0-py3-none-any.whl (303 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m303.4/303.4 kB\u001b[0m \u001b[31m28.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading olefile-0.47-py2.py3-none-any.whl (114 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m114.6/114.6 kB\u001b[0m \u001b[31m12.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hBuilding wheels for collected packages: langdetect\n", + " Building wheel for langdetect (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + " Created wheel for langdetect: filename=langdetect-1.0.9-py3-none-any.whl size=993223 sha256=1620ee6f30556e3dc45ebc206abb0af215614a4c165b45aeb9a970f5f9ca99fc\n", + " Stored in directory: /root/.cache/pip/wheels/0a/f2/b2/e5ca405801e05eb7c8ed5b3b4bcf1fcabcd6272c167640072e\n", + "Successfully built langdetect\n", + "Installing collected packages: filetype, rapidfuzz, python-magic, python-iso639, pypdf, olefile, langdetect, eval-type-backport, emoji, backoff, aiofiles, python-oxmsg, unstructured-client, unstructured\n", + "Successfully installed aiofiles-24.1.0 backoff-2.2.1 emoji-2.14.1 eval-type-backport-0.2.2 filetype-1.2.0 langdetect-1.0.9 olefile-0.47 pypdf-5.5.0 python-iso639-2025.2.18 python-magic-0.4.27 python-oxmsg-0.0.2 rapidfuzz-3.13.0 unstructured-0.17.2 unstructured-client-0.34.0\n" + ] + } + ], + "source": [ + "!pip install -qU ragas==0.2.10\n", + "!pip install unstructured" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "qhJsO4qVXkWj" + }, + "source": [ + "Load our data!" + ] + }, + { + "cell_type": "code", + "execution_count": 98, + "metadata": { + "id": "xzAwAd5sXkWj" + }, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import DirectoryLoader, TextLoader\n", + "\n", + "path = \"data/\"\n", + "# Updated loader with glob pattern and TextLoader class\n", + "loader = DirectoryLoader(path, glob=\"*.txt\", loader_cls=TextLoader)\n", + "docs = loader.load()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "JodPMkxQXkWj" + }, + "source": [ + "**Generate synthetic test data using ragas.** We provide an llm, an embedding model, and testset_size, and our documents and RAGAS does it's magic via knowledge graphs!\n", + "\n", + "As a reminder, this will create about 10 rows of test data with the following columns:\\\n", + "**user_input\treference_contexts\treference\tsynthesizer_name**\\\n", + "User_input is the query, reference_context is the ideal context for this query, reference is the ideal response for this query and synthesizer name is the query synthesizer used. It will be one of SingleHopSpecific, MultiHopSpecific, and MultiHopAbstract." + ] + }, + { + "cell_type": "code", + "execution_count": 99, + "metadata": { + "id": "37iz3MB6XkWj", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 287, + "referenced_widgets": [ + "68d08d2c70a44ae68cfc2a5885e83cb1", + "d426d1252b1c443eaa1d7079cbca2b54", + "b3968843072b4551b7ec6c807b673b72", + "c6c8e0fdade24e83a133b4f98477774a", + "b7f5d55603814fee86de87cfe4c1d433", + "da141c357a764dcab7441532e0b98c43", + "0c024e9045904cad81fe7d2f11b97bf9", + "ecaf62a0b90a4d39bca075753e495508", + "43e42e2ff62f44d58423249242effa17", + "8d5370219dfa4f4193210d10cfa3565e", + "9b6991490078438995b17c476c39e2ad", + "cf446e2820fc48178a98809bd9f82bc5", + "67c62bd1c70d43489b841b0981e00511", + "1aeba09816e247b3afaa45c878d41899", + "7d24f2ba4f694bbc8b43756e877712aa", + "abbf513aa08d44cb94aeba577414c560", + "ce2ab19759e248c2b73760921797d576", + "6bc39ff9a8604c4b8cae1453bdf55fc7", + "22cf143a0e234510889df3a3d17318ed", + "38722dbe9db44ce5a3337bc7d7e7afdf", + "154fb399e08a467f8fbad618e386ad91", + "ef2d28f5756947f3ab40b358ef851e62", + "451465ccf1ad469b9a0a0e2a8a6f2d61", + "8b479049e375404aac1293a182daeefd", + "bb68fee5c65447ac9403495f855542c2", + "1e9f1f8e3ef94a1eb1434ae328abc94f", + "42a46644be3b42368b33a39594ab0e2b", + "13205a4bd8fc41e3b68df4452ceb2a8a", + "fa82cb2e42ab44cbb2c3666327841330", + "d3f16edb9d164e60a3a2078b815e368f", + "12f8887f3bfa4d78bdca80c4a5fdce84", + "2a282a4bbc0046848a55d103cf770da2", + "7737530a1337496388fdf6db04546b29", + "b130b62aaa0648d589e1db78cefa45dc", + "e60de60334104b6cb0e89f0460cb18af", + "71f21a487fb84576baba26c7d46d39e5", + "f9296de4f2f34224952c6486ea149e58", + "ae34ca91a0af4ded929efaa27860fe71", + "7e40823470fc4e439b66da7abb09dc90", + "1b16421167ab48b8a5917d72d656c7bf", + "f7e04af9a15847d5a948a894f610cc49", + "4327e516e08a4c73b72253077d7e357b", + "479f97a939d540d7b027414a51d431fb", + "5df46535a7cf42989003692bf06d4cd8", + "bc0b7e8d78ed4f5689444ea0a98ae561", + "77271c647e124d35bbd2b0b08ed40c65", + "f12f0c28c26c4663932c0aeb0ea209e2", + "448f78a96ab54fc4a22575c54617ca0c", + "2b186131460f4396b14ba5bc7e917533", + "41b54f704a7a4111beb0969226aa383d", + "b196c652848a4c7e9c7569f881c89b1f", + "797447801d634b5a8e08c5318b1353ed", + "506eb650f3ab4d3fbfc0c0235e53bc7d", + "bffa759cd8674d889e9c2f4ecf12d3e1", + "315bf7504aa44c958b809900d535d25b", + "09db23941a9c4904b21c714da581a43a", + "45ca2e30c7a249f891398109562d411c", + "3e507be792e14294a65311adf32f3acd", + "7876a02ea0324dd88219f6dd437f33ad", + "b304e464495a4694a138305c99b3c1bb", + "2d1bdefd99b04c9286a67a32beeb18be", + "c4ab6cd1de4245dcb646a23ed6e866f7", + "2c79cd256dd6412e823486d903c7c29f", + "3eaed3e9110b4e7791916cacfc3d9e98", + "fffe61a141cd4734ad3672f897377ba0", + "7183d9c0f4fe46d9a60da91d6251eec7", + "79098b28847d454a82b0f059acd9b1fd", + "785d5293fe434795aeb39c1543437713", + "89ae40bc94a44a2da9d087b8cd39fece", + "6240b2910907434aaca164448fec0180", + "6a3001ea45cc4611b9ada6ae91285d6f", + "8b0d39314515439a98836ac0c1184c06", + "52b39c401c5a4921ad7bf5f114903459", + "bdad31eb9e6b4590b3f92a2e200fccb5", + "72d74876ce7d4a1692ae5ae4d49544f0", + "0a14d94b59114e40ba6b592cea136702", + "4c47217a866d4da9a33f0f945af5e3cc", + "9a941919d4ad4670acee80d33543cf3b", + "b3dd78bc59194ce5971daa413a11ca7e", + "6ca416d7b6714ed9a026266db2d19e6d", + "c8119aa65b6640dc9c68e04ae66e6357", + "ee7f9374cd0b402ca1405dcd77d8a436", + "cfb3de7781fa4b69bada2708cbe6b658", + "33a6c456af8144feb34d20f63499dcca", + "8c982789718348048264095afb8d409c", + "018816f934b64d2c8ee4329455f6dabd", + "fd55665525c54123bf52156d132c5137", + "54db49340c3e4c3e93d04a282c11ac5b", + "58d03b48d18c4865996d21d1cd03c1ae", + "fa9f67c70db14b738093c3afce6ac95e", + "19d5df5770114e03bff203815e01eee5", + "d29d0ae1b38c49d69b61cac820ea4083", + "ad7fd43b8f554a898def1886a2247351", + "c8e8c705b0c743cebb662347c85a7834", + "44e02a685c944750b7947fc040ea5bd5", + "05af8375ea734765b52e3c6cc36bf014", + "970f40cf6f4541a8a357a30198759241", + "4b8cf390496a46b78002530bbfb74eac", + "01234191c7a945c58515a207ebc42dde" + ] + }, + "outputId": "2a6479fc-7683-438b-bfe3-a3daa04c36f2" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "Applying HeadlinesExtractor: 0%| | 0/10 [00:00**Evaluate the UnFineTuned and FineTuned model with Ragas**\n", + "\n", + "**First, we create the two datasets, by adding the retrieved response and retrieved context** - we do this for both the rag chains, one which uses the base snowflake embedding model and one which uses the finetuned snowflake embedding model." + ] + }, + { + "cell_type": "code", + "execution_count": 100, + "metadata": { + "id": "_lEUkh3QXkWk", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 + }, + "outputId": "b6d580d3-0f97-4569-805f-18a3fed05d7b" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + " user_input \\\n", + "0 Who J.H. Stickney and what he do for Hans Ande... \n", + "1 How does Hans Andersen's storytelling ability ... \n", + "2 Who is Edna F. Hart in relation to Hans Anders... \n", + "3 What does the sun represent for the fir tree i... \n", + "4 In what ways does Socrates' defense in Plato's... \n", + "5 In Plato's Apology, how does Socrates' rhetori... \n", + "6 In what ways does Plato's 'Apology' reflect th... \n", + "7 In what ways does Plato's Apology reflect the ... \n", + "8 What do the gods require of a man to live a di... \n", + "9 How does the theme of spring manifest in the e... \n", + "10 How does the arrival of autumn affect Thumbeli... \n", + "11 Why did the duckling feel thankful for being u... \n", + "\n", + " retrieved_contexts \\\n", + "0 [carefully standardized, their place in perman... \n", + "1 [carefully standardized, their place in perman... \n", + "2 [carefully standardized, their place in perman... \n", + "3 [\"Rejoice in our love,\" said the air and the s... \n", + "4 [The β€œApology” or Platonic defence of Socrates... \n", + "5 [Yet some of the topics may have been actually... \n", + "6 [Yet some of the topics may have been actually... \n", + "7 [Yet some of the topics may have been actually... \n", + "8 [seest that those things, which for a man to h... \n", + "9 [sparrow flew, for she saw many others of her ... \n", + "10 [\"Yes, I will go with you,\" said Thumbelina; a... \n", + "11 [\"I believe I must go out into the world again... \n", + "\n", + " reference_contexts \\\n", + "0 [ο»ΏProject Gutenberg's Hans Andersen's Fairy Ta... \n", + "1 [PREFACE The Hans Andersen Fairy Tales will be... \n", + "2 [HANS ANDERSEN'S FAIRY TALES *** Produced by S... \n", + "3 [THE GREENIES 141 OLE-LUK-OIE, THE DREAM GOD 1... \n", + "4 [<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche... \n", + "5 [<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche... \n", + "6 [<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche... \n", + "7 [<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche... \n", + "8 [<1-hop>\\n\\nlatter years. That as often as I h... \n", + "9 [<1-hop>\\n\\nSee, here we are at the blacksmith... \n", + "10 [<1-hop>\\n\\nlie warmly in the cold earth. \"Far... \n", + "11 [<1-hop>\\n\\nWill you go with us and become a b... \n", + "\n", + " response \\\n", + "0 J. H. Stickney is the person who reprinted and... \n", + "1 Hans Andersen's storytelling ability contribut... \n", + "2 The provided context does not mention Edna F. ... \n", + "3 The sun represents a source of joy and vitalit... \n", + "4 The provided context does not contain specific... \n", + "5 The provided context does not explicitly detai... \n", + "6 The provided context suggests that Plato's 'Ap... \n", + "7 Plato's Apology reflects the themes of rhetori... \n", + "8 According to the teachings in the context, the... \n", + "9 I do not know. \n", + "10 The provided context does not include informat... \n", + "11 The provided context does not indicate that th... \n", + "\n", + " reference \\\n", + "0 J.H. Stickney is the editor of Hans Andersen's... \n", + "1 Hans Andersen's storytelling ability is charac... \n", + "2 Edna F. Hart is the illustrator of Hans Anders... \n", + "3 The sun represents warmth and joy for the fir ... \n", + "4 Socrates' defense in Plato's 'Apology' reflect... \n", + "5 In Plato's Apology, Socrates employs a unique ... \n", + "6 Plato's 'Apology' reflects the themes of rheto... \n", + "7 Plato's Apology reflects the themes of rhetori... \n", + "8 The gods require no more of any man than to ke... \n", + "9 In the provided fairy tale context, the theme ... \n", + "10 The arrival of autumn significantly impacts Th... \n", + "11 The duckling felt thankful for being ugly beca... \n", + "\n", + " synthesizer_name \n", + "0 single_hop_specifc_query_synthesizer \n", + "1 single_hop_specifc_query_synthesizer \n", + "2 single_hop_specifc_query_synthesizer \n", + "3 single_hop_specifc_query_synthesizer \n", + "4 multi_hop_abstract_query_synthesizer \n", + "5 multi_hop_abstract_query_synthesizer \n", + "6 multi_hop_abstract_query_synthesizer \n", + "7 multi_hop_abstract_query_synthesizer \n", + "8 multi_hop_specific_query_synthesizer \n", + "9 multi_hop_specific_query_synthesizer \n", + "10 multi_hop_specific_query_synthesizer \n", + "11 multi_hop_specific_query_synthesizer " + ], + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
user_inputretrieved_contextsreference_contextsresponsereferencesynthesizer_name
0Who J.H. Stickney and what he do for Hans Ande...[carefully standardized, their place in perman...[ο»ΏProject Gutenberg's Hans Andersen's Fairy Ta...J. H. Stickney is the person who reprinted and...J.H. Stickney is the editor of Hans Andersen's...single_hop_specifc_query_synthesizer
1How does Hans Andersen's storytelling ability ...[carefully standardized, their place in perman...[PREFACE The Hans Andersen Fairy Tales will be...Hans Andersen's storytelling ability contribut...Hans Andersen's storytelling ability is charac...single_hop_specifc_query_synthesizer
2Who is Edna F. Hart in relation to Hans Anders...[carefully standardized, their place in perman...[HANS ANDERSEN'S FAIRY TALES *** Produced by S...The provided context does not mention Edna F. ...Edna F. Hart is the illustrator of Hans Anders...single_hop_specifc_query_synthesizer
3What does the sun represent for the fir tree i...[\"Rejoice in our love,\" said the air and the s...[THE GREENIES 141 OLE-LUK-OIE, THE DREAM GOD 1...The sun represents a source of joy and vitalit...The sun represents warmth and joy for the fir ...single_hop_specifc_query_synthesizer
4In what ways does Socrates' defense in Plato's...[The β€œApology” or Platonic defence of Socrates...[<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche...The provided context does not contain specific...Socrates' defense in Plato's 'Apology' reflect...multi_hop_abstract_query_synthesizer
5In Plato's Apology, how does Socrates' rhetori...[Yet some of the topics may have been actually...[<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche...The provided context does not explicitly detai...In Plato's Apology, Socrates employs a unique ...multi_hop_abstract_query_synthesizer
6In what ways does Plato's 'Apology' reflect th...[Yet some of the topics may have been actually...[<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche...The provided context suggests that Plato's 'Ap...Plato's 'Apology' reflects the themes of rheto...multi_hop_abstract_query_synthesizer
7In what ways does Plato's Apology reflect the ...[Yet some of the topics may have been actually...[<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche...Plato's Apology reflects the themes of rhetori...Plato's Apology reflects the themes of rhetori...multi_hop_abstract_query_synthesizer
8What do the gods require of a man to live a di...[seest that those things, which for a man to h...[<1-hop>\\n\\nlatter years. That as often as I h...According to the teachings in the context, the...The gods require no more of any man than to ke...multi_hop_specific_query_synthesizer
9How does the theme of spring manifest in the e...[sparrow flew, for she saw many others of her ...[<1-hop>\\n\\nSee, here we are at the blacksmith...I do not know.In the provided fairy tale context, the theme ...multi_hop_specific_query_synthesizer
10How does the arrival of autumn affect Thumbeli...[\"Yes, I will go with you,\" said Thumbelina; a...[<1-hop>\\n\\nlie warmly in the cold earth. \"Far...The provided context does not include informat...The arrival of autumn significantly impacts Th...multi_hop_specific_query_synthesizer
11Why did the duckling feel thankful for being u...[\"I believe I must go out into the world again...[<1-hop>\\n\\nWill you go with us and become a b...The provided context does not indicate that th...The duckling felt thankful for being ugly beca...multi_hop_specific_query_synthesizer
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "\n", + "
\n", + "
\n" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "dataframe", + "summary": "{\n \"name\": \"sdg_dataset\",\n \"rows\": 12,\n \"fields\": [\n {\n \"column\": \"user_input\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"How does the arrival of autumn affect Thumbelina's situation and her feelings towards the swallow in the story?\",\n \"How does the theme of spring manifest in the experiences of the characters in the provided fairy tale context?\",\n \"Who J.H. Stickney and what he do for Hans Andersen's Fairy Tales?\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"retrieved_contexts\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"reference_contexts\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"response\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"The provided context does not include information about the arrival of autumn or its effects on Thumbelina's situation and her feelings towards the swallow. Therefore, I do not know how autumn influences her circumstances or her emotions in the story.\",\n \"I do not know.\",\n \"J. H. Stickney is the person who reprinted and adapted the text of Hans Andersen's Fairy Tales. He based his edition on a comparison of various translations, made slight abridgments to longer stories, and arranged the stories to suit the reading level of children.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"reference\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"The arrival of autumn significantly impacts Thumbelina's situation as she prepares for her wedding to the mole, which she dreads. She feels unhappy at the thought of leaving the warm sun and her dear friend, the swallow, who has flown away to warmer climates. As autumn progresses, she reflects on her fondness for the swallow and wishes to see him again, highlighting her sadness and longing for the beauty of summer and the companionship of the bird. The coldness of autumn contrasts with her memories of warmth and joy, deepening her sense of isolation and despair as she faces a future with the disagreeable mole.\",\n \"In the provided fairy tale context, the theme of spring is vividly illustrated through the characters' experiences and the descriptions of their surroundings. The little girl expresses her delight in the beauty of spring, stating, \\\"It is beautiful here in the spring,\\\" as they stand in a grove of beech trees adorned with fresh green leaves and surrounded by blooming flowers like thyme and anemone. This imagery evokes a sense of renewal and joy associated with the season. Additionally, the narrative transitions through various seasons, but spring is highlighted as a time of fragrance and beauty, with the elder tree's sweet scent permeating their adventures. The boy, who later becomes a sailor, cherishes the memory of spring, as he recalls the little girl and the elder blossom she gave him, symbolizing the lasting impact of their experiences during this vibrant season. Thus, spring serves as a backdrop for their innocent play and the blossoming of their friendship, reinforcing the theme of beauty and nostalgia in the tale.\",\n \"J.H. Stickney is the editor of Hans Andersen's Fairy Tales, which is a collection of stories by Hans Christian Andersen.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"synthesizer_name\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 3,\n \"samples\": [\n \"single_hop_specifc_query_synthesizer\",\n \"multi_hop_abstract_query_synthesizer\",\n \"multi_hop_specific_query_synthesizer\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" + } + }, + "metadata": {}, + "execution_count": 100 + } + ], + "source": [ + "import time\n", + "for test_row in sdg_dataset:\n", + " response = base_rag_chain.invoke({\"question\" : test_row.eval_sample.user_input})\n", + " test_row.eval_sample.response = response[\"response\"]\n", + " test_row.eval_sample.retrieved_contexts = [context.page_content for context in response[\"context\"]]\n", + " time.sleep(5)\n", + "\n", + "time.sleep(5)\n", + "\n", + "sdg_dataset.to_pandas()" + ] + }, + { + "cell_type": "code", + "execution_count": 101, + "metadata": { + "id": "zVAutZ-8XkWk", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 + }, + "outputId": "513b4497-c432-4718-f819-c824d0a8376d" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + " user_input \\\n", + "0 Who J.H. Stickney and what he do for Hans Ande... \n", + "1 How does Hans Andersen's storytelling ability ... \n", + "2 Who is Edna F. Hart in relation to Hans Anders... \n", + "3 What does the sun represent for the fir tree i... \n", + "4 In what ways does Socrates' defense in Plato's... \n", + "5 In Plato's Apology, how does Socrates' rhetori... \n", + "6 In what ways does Plato's 'Apology' reflect th... \n", + "7 In what ways does Plato's Apology reflect the ... \n", + "8 What do the gods require of a man to live a di... \n", + "9 How does the theme of spring manifest in the e... \n", + "10 How does the arrival of autumn affect Thumbeli... \n", + "11 Why did the duckling feel thankful for being u... \n", + "\n", + " retrieved_contexts \\\n", + "0 [Language: English\\n\\nCharacter set encoding: ... \n", + "1 [The Hans Andersen Fairy Tales will be read in... \n", + "2 [Language: English\\n\\nCharacter set encoding: ... \n", + "3 [[Illustration: They danced merrily ... around... \n", + "4 [β€œFew persons will be found to wish that Socra... \n", + "5 [Yet some of the topics may have been actually... \n", + "6 [Yet some of the topics may have been actually... \n", + "7 [Yet some of the topics may have been actually... \n", + "8 [seest that those things, which for a man to h... \n", + "9 [[Illustration]\\n\\n\\n\\n\\nTHE CONCEITED APPLE B... \n", + "10 [Every morning when the sun rose and every eve... \n", + "11 [\"I believe I must go out into the world again... \n", + "\n", + " reference_contexts \\\n", + "0 [ο»ΏProject Gutenberg's Hans Andersen's Fairy Ta... \n", + "1 [PREFACE The Hans Andersen Fairy Tales will be... \n", + "2 [HANS ANDERSEN'S FAIRY TALES *** Produced by S... \n", + "3 [THE GREENIES 141 OLE-LUK-OIE, THE DREAM GOD 1... \n", + "4 [<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche... \n", + "5 [<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche... \n", + "6 [<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche... \n", + "7 [<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche... \n", + "8 [<1-hop>\\n\\nlatter years. That as often as I h... \n", + "9 [<1-hop>\\n\\nSee, here we are at the blacksmith... \n", + "10 [<1-hop>\\n\\nlie warmly in the cold earth. \"Far... \n", + "11 [<1-hop>\\n\\nWill you go with us and become a b... \n", + "\n", + " response \\\n", + "0 J.H. Stickney is the editor of Hans Andersen's... \n", + "1 Hans Andersen's storytelling ability contribut... \n", + "2 Edna F. Hart is the illustrator of Hans Anders... \n", + "3 The provided context does not explicitly state... \n", + "4 The provided context does not explicitly discu... \n", + "5 The provided context indicates that Plato’s Ap... \n", + "6 The provided context indicates that Plato's 'A... \n", + "7 Based on the provided context, Plato's Apology... \n", + "8 According to the teachings in the context, the... \n", + "9 The theme of spring manifests in the character... \n", + "10 The arrival of autumn makes Thumbelina's situa... \n", + "11 The provided context does not indicate that th... \n", + "\n", + " reference \\\n", + "0 J.H. Stickney is the editor of Hans Andersen's... \n", + "1 Hans Andersen's storytelling ability is charac... \n", + "2 Edna F. Hart is the illustrator of Hans Anders... \n", + "3 The sun represents warmth and joy for the fir ... \n", + "4 Socrates' defense in Plato's 'Apology' reflect... \n", + "5 In Plato's Apology, Socrates employs a unique ... \n", + "6 Plato's 'Apology' reflects the themes of rheto... \n", + "7 Plato's Apology reflects the themes of rhetori... \n", + "8 The gods require no more of any man than to ke... \n", + "9 In the provided fairy tale context, the theme ... \n", + "10 The arrival of autumn significantly impacts Th... \n", + "11 The duckling felt thankful for being ugly beca... \n", + "\n", + " synthesizer_name \n", + "0 single_hop_specifc_query_synthesizer \n", + "1 single_hop_specifc_query_synthesizer \n", + "2 single_hop_specifc_query_synthesizer \n", + "3 single_hop_specifc_query_synthesizer \n", + "4 multi_hop_abstract_query_synthesizer \n", + "5 multi_hop_abstract_query_synthesizer \n", + "6 multi_hop_abstract_query_synthesizer \n", + "7 multi_hop_abstract_query_synthesizer \n", + "8 multi_hop_specific_query_synthesizer \n", + "9 multi_hop_specific_query_synthesizer \n", + "10 multi_hop_specific_query_synthesizer \n", + "11 multi_hop_specific_query_synthesizer " + ], + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
user_inputretrieved_contextsreference_contextsresponsereferencesynthesizer_name
0Who J.H. Stickney and what he do for Hans Ande...[Language: English\\n\\nCharacter set encoding: ...[ο»ΏProject Gutenberg's Hans Andersen's Fairy Ta...J.H. Stickney is the editor of Hans Andersen's...J.H. Stickney is the editor of Hans Andersen's...single_hop_specifc_query_synthesizer
1How does Hans Andersen's storytelling ability ...[The Hans Andersen Fairy Tales will be read in...[PREFACE The Hans Andersen Fairy Tales will be...Hans Andersen's storytelling ability contribut...Hans Andersen's storytelling ability is charac...single_hop_specifc_query_synthesizer
2Who is Edna F. Hart in relation to Hans Anders...[Language: English\\n\\nCharacter set encoding: ...[HANS ANDERSEN'S FAIRY TALES *** Produced by S...Edna F. Hart is the illustrator of Hans Anders...Edna F. Hart is the illustrator of Hans Anders...single_hop_specifc_query_synthesizer
3What does the sun represent for the fir tree i...[[Illustration: They danced merrily ... around...[THE GREENIES 141 OLE-LUK-OIE, THE DREAM GOD 1...The provided context does not explicitly state...The sun represents warmth and joy for the fir ...single_hop_specifc_query_synthesizer
4In what ways does Socrates' defense in Plato's...[β€œFew persons will be found to wish that Socra...[<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche...The provided context does not explicitly discu...Socrates' defense in Plato's 'Apology' reflect...multi_hop_abstract_query_synthesizer
5In Plato's Apology, how does Socrates' rhetori...[Yet some of the topics may have been actually...[<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche...The provided context indicates that Plato’s Ap...In Plato's Apology, Socrates employs a unique ...multi_hop_abstract_query_synthesizer
6In what ways does Plato's 'Apology' reflect th...[Yet some of the topics may have been actually...[<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche...The provided context indicates that Plato's 'A...Plato's 'Apology' reflects the themes of rheto...multi_hop_abstract_query_synthesizer
7In what ways does Plato's Apology reflect the ...[Yet some of the topics may have been actually...[<1-hop>\\n\\nAPOLOGY *** Produced by Sue Assche...Based on the provided context, Plato's Apology...Plato's Apology reflects the themes of rhetori...multi_hop_abstract_query_synthesizer
8What do the gods require of a man to live a di...[seest that those things, which for a man to h...[<1-hop>\\n\\nlatter years. That as often as I h...According to the teachings in the context, the...The gods require no more of any man than to ke...multi_hop_specific_query_synthesizer
9How does the theme of spring manifest in the e...[[Illustration]\\n\\n\\n\\n\\nTHE CONCEITED APPLE B...[<1-hop>\\n\\nSee, here we are at the blacksmith...The theme of spring manifests in the character...In the provided fairy tale context, the theme ...multi_hop_specific_query_synthesizer
10How does the arrival of autumn affect Thumbeli...[Every morning when the sun rose and every eve...[<1-hop>\\n\\nlie warmly in the cold earth. \"Far...The arrival of autumn makes Thumbelina's situa...The arrival of autumn significantly impacts Th...multi_hop_specific_query_synthesizer
11Why did the duckling feel thankful for being u...[\"I believe I must go out into the world again...[<1-hop>\\n\\nWill you go with us and become a b...The provided context does not indicate that th...The duckling felt thankful for being ugly beca...multi_hop_specific_query_synthesizer
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "\n", + "
\n", + "
\n" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "dataframe", + "summary": "{\n \"name\": \"finetuned_sdg_dataset\",\n \"rows\": 12,\n \"fields\": [\n {\n \"column\": \"user_input\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"How does the arrival of autumn affect Thumbelina's situation and her feelings towards the swallow in the story?\",\n \"How does the theme of spring manifest in the experiences of the characters in the provided fairy tale context?\",\n \"Who J.H. Stickney and what he do for Hans Andersen's Fairy Tales?\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"retrieved_contexts\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"reference_contexts\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"response\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"The arrival of autumn makes Thumbelina's situation more poignant and sad. She has her outfit ready for her wedding, but she feels sorrowful because she does not want to marry the disagreeable mole and is longing to see her dear friend, the swallow, again. She weeps and feels unhappy about her circumstances, especially since she is only an inch tall and unable to freely go out into the sunshine. Her feelings towards the swallow are tender; she cares deeply for him and is distressed when he leaves, feeling very sad as she watches him fly away into the green woods. Overall, autumn heightens her sense of longing, sadness, and attachment to her friends.\",\n \"The theme of spring manifests in the characters' experiences through the renewal and beauty of nature that surrounds them. The characters observe the arrival of spring with blooming flowers, fresh green leaves, and the singing of birds, which symbolize new life and hope. For example, the little apple tree is covered with delicate pink blossoms just before they open, indicating the blossoming of spring. The children enjoy sitting in the garden on the roof, surrounded by blooming roses and the vibrant landscape, reflecting the joy and freshness of the season. Additionally, Gerda and Kay notice the green verdure and beautiful flowers as they walk home, and the maiden expresses a longing for spring to be everlasting in the fragrant beech groves. Overall, spring is depicted as a time of natural beauty, happiness, and renewal that deeply influences the characters' surroundings and feelings.\",\n \"J.H. Stickney is the editor of Hans Andersen's Fairy Tales.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"reference\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"The arrival of autumn significantly impacts Thumbelina's situation as she prepares for her wedding to the mole, which she dreads. She feels unhappy at the thought of leaving the warm sun and her dear friend, the swallow, who has flown away to warmer climates. As autumn progresses, she reflects on her fondness for the swallow and wishes to see him again, highlighting her sadness and longing for the beauty of summer and the companionship of the bird. The coldness of autumn contrasts with her memories of warmth and joy, deepening her sense of isolation and despair as she faces a future with the disagreeable mole.\",\n \"In the provided fairy tale context, the theme of spring is vividly illustrated through the characters' experiences and the descriptions of their surroundings. The little girl expresses her delight in the beauty of spring, stating, \\\"It is beautiful here in the spring,\\\" as they stand in a grove of beech trees adorned with fresh green leaves and surrounded by blooming flowers like thyme and anemone. This imagery evokes a sense of renewal and joy associated with the season. Additionally, the narrative transitions through various seasons, but spring is highlighted as a time of fragrance and beauty, with the elder tree's sweet scent permeating their adventures. The boy, who later becomes a sailor, cherishes the memory of spring, as he recalls the little girl and the elder blossom she gave him, symbolizing the lasting impact of their experiences during this vibrant season. Thus, spring serves as a backdrop for their innocent play and the blossoming of their friendship, reinforcing the theme of beauty and nostalgia in the tale.\",\n \"J.H. Stickney is the editor of Hans Andersen's Fairy Tales, which is a collection of stories by Hans Christian Andersen.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"synthesizer_name\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 3,\n \"samples\": [\n \"single_hop_specifc_query_synthesizer\",\n \"multi_hop_abstract_query_synthesizer\",\n \"multi_hop_specific_query_synthesizer\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" + } + }, + "metadata": {}, + "execution_count": 101 + } + ], + "source": [ + "import time\n", + "for test_row in finetuned_sdg_dataset:\n", + " response = finetune_rag_chain.invoke({\"question\" : test_row.eval_sample.user_input})\n", + " test_row.eval_sample.response = response[\"response\"]\n", + " test_row.eval_sample.retrieved_contexts = [context.page_content for context in response[\"context\"]]\n", + " time.sleep(5)\n", + "\n", + "time.sleep(5)\n", + "\n", + "finetuned_sdg_dataset.to_pandas()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "cv9JqR-GXkWk" + }, + "source": [ + "**Convert our two datasets into a RAGAS evaluation dataset using a ragas library function**" + ] + }, + { + "cell_type": "code", + "execution_count": 102, + "metadata": { + "id": "qq3E4MSPXkWk" + }, + "outputs": [], + "source": [ + "from ragas import EvaluationDataset\n", + "\n", + "evaluation_dataset = EvaluationDataset.from_pandas(sdg_dataset.to_pandas())\n", + "finetuned_evaluation_dataset = EvaluationDataset.from_pandas(finetuned_sdg_dataset.to_pandas())\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "BgJwNvkdXkWk" + }, + "source": [ + "**Set our evaluation LLM and evaluation config**" + ] + }, + { + "cell_type": "code", + "execution_count": 124, + "metadata": { + "id": "vZzzMg4eXkWk" + }, + "outputs": [], + "source": [ + "from ragas import evaluate\n", + "from ragas.llms import LangchainLLMWrapper\n", + "from ragas.metrics import LLMContextRecall, Faithfulness, FactualCorrectness, ResponseRelevancy, ContextEntityRecall, NoiseSensitivity\n", + "from ragas import evaluate, RunConfig\n", + "\n", + "evaluator_llm = LangchainLLMWrapper(ChatOpenAI(model=\"gpt-4.1.mini\"))\n", + "\n", + "# default max_workers is 16.\n", + "# custom_run_config = RunConfig(timeout=360, max_workers=8)\n", + "custom_run_config = RunConfig(\n", + " timeout=300, # 5 minutes max for operations\n", + " max_retries=15, # More retries for rate limits\n", + " max_wait=90, # Longer wait between retries\n", + " max_workers=8, # Fewer concurrent API calls\n", + " log_tenacity=True # Log retry attempts\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9AzPEprEXkWk" + }, + "source": [ + "**Evaluate the Base model**" + ] + }, + { + "cell_type": "code", + "execution_count": 125, + "metadata": { + "id": "8fZrD-__XkWk", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 408 + }, + "outputId": "f3b3d928-40d0-43a3-f831-e14652930002" + }, + "outputs": [ + { + "output_type": "error", + "ename": "RuntimeError", + "evalue": "Tracing using LangChainTracerV1 is no longer supported. Please set the LANGCHAIN_TRACING_V2 environment variable to enable tracing instead.", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mRuntimeError\u001b[0m Traceback (most recent call last)", + "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m base_result = evaluate(\n\u001b[0m\u001b[1;32m 2\u001b[0m \u001b[0mdataset\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mevaluation_dataset\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0mmetrics\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mLLMContextRecall\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mFaithfulness\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mFactualCorrectness\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mResponseRelevancy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mContextEntityRecall\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mNoiseSensitivity\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0mllm\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mevaluator_llm\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0mrun_config\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcustom_run_config\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/ragas/_analytics.py\u001b[0m in \u001b[0;36mwrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 225\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mwrapper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mP\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mAny\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 226\u001b[0m \u001b[0mtrack\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mIsCompleteEvent\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mevent_type\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mfunc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__name__\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mis_completed\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 227\u001b[0;31m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 228\u001b[0m \u001b[0mtrack\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mIsCompleteEvent\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mevent_type\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mfunc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__name__\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mis_completed\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 229\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/ragas/evaluation.py\u001b[0m in \u001b[0;36mevaluate\u001b[0;34m(dataset, metrics, llm, embeddings, callbacks, run_config, token_usage_parser, raise_exceptions, column_map, show_progress, batch_size, _run_id, _pbar)\u001b[0m\n\u001b[1;32m 246\u001b[0m \u001b[0;31m# new evaluation chain\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 247\u001b[0m \u001b[0mrow_run_managers\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 248\u001b[0;31m evaluation_rm, evaluation_group_cm = new_group(\n\u001b[0m\u001b[1;32m 249\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mRAGAS_EVALUATION_CHAIN_NAME\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 250\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/ragas/callbacks.py\u001b[0m in \u001b[0;36mnew_group\u001b[0;34m(name, inputs, callbacks, tags, metadata)\u001b[0m\n\u001b[1;32m 29\u001b[0m \u001b[0;31m# start evaluation chain\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 30\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcallbacks\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 31\u001b[0;31m \u001b[0mcm\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCallbackManager\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconfigure\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minheritable_callbacks\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcallbacks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 32\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 33\u001b[0m \u001b[0mcm\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcast\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mCallbackManager\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcallbacks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/langchain_core/callbacks/manager.py\u001b[0m in \u001b[0;36mconfigure\u001b[0;34m(cls, inheritable_callbacks, local_callbacks, verbose, inheritable_tags, local_tags, inheritable_metadata, local_metadata)\u001b[0m\n\u001b[1;32m 1574\u001b[0m \u001b[0mCallbackManager\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mThe\u001b[0m \u001b[0mconfigured\u001b[0m \u001b[0mcallback\u001b[0m \u001b[0mmanager\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1575\u001b[0m \"\"\"\n\u001b[0;32m-> 1576\u001b[0;31m return _configure(\n\u001b[0m\u001b[1;32m 1577\u001b[0m \u001b[0mcls\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1578\u001b[0m \u001b[0minheritable_callbacks\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/langchain_core/callbacks/manager.py\u001b[0m in \u001b[0;36m_configure\u001b[0;34m(callback_manager_cls, inheritable_callbacks, local_callbacks, inheritable_tags, local_tags, inheritable_metadata, local_metadata, verbose)\u001b[0m\n\u001b[1;32m 2355\u001b[0m \u001b[0;34m\"tracing instead.\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2356\u001b[0m )\n\u001b[0;32m-> 2357\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mRuntimeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmsg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 2358\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2359\u001b[0m \u001b[0mtracer_project\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_get_tracer_project\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;31mRuntimeError\u001b[0m: Tracing using LangChainTracerV1 is no longer supported. Please set the LANGCHAIN_TRACING_V2 environment variable to enable tracing instead." + ] + } + ], + "source": [ + "base_result = evaluate(\n", + " dataset=evaluation_dataset,\n", + " metrics=[LLMContextRecall(), Faithfulness(), FactualCorrectness(), ResponseRelevancy(), ContextEntityRecall(), NoiseSensitivity()],\n", + " llm=evaluator_llm,\n", + " run_config=custom_run_config\n", + ")\n", + "base_result" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "NwV6gO8AXkWk" + }, + "source": [ + "**Evaluate the Finetuned model**" + ] + }, + { + "cell_type": "code", + "execution_count": 105, + "metadata": { + "id": "Oh4qoQ16XkWk", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 136, + "referenced_widgets": [ + "3e16df886a974a58a20b18537161d257", + "68b38f1c25a54b978cd20b32c3d02b89", + "33726e4a88e7431b84b26f51c19c6384", + "15f04a999af24f4cb638e87f7839ee24", + "4e58212505424fb49592181d3ed49f3c", + "d932d142ac64496e9524be5c5b07810a", + "3f62770fd81247ff9ff69c65324579f1", + "a48d9be896a94f19899b929655c65771", + "e1109a565b1644c0997d35d2234c5f5e", + "b5bbac0cdd7742d69e45d349879dc96e", + "305cd0071eee46bc9bf16b4d07dabcab" + ] + }, + "outputId": "5e6586bb-2943-49c6-d4fc-058e96eb9aa1" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "Evaluating: 0%| | 0/72 [00:00**RAGAS Evaluation Results: PERPLEXING? NOT!! We used a less powerful LLM Model. Since the last time I did this with gpt-4.1, it cost me!!**\n", + "\n", + " The RAGAS evaluation presented mixed results across the two runs as you can see in my tables below. While some metrics like context_recall or noise_sensitivity improved with fine-tuning in one or both runs, others like faithfulness or context_entity_recall sometimes decreased. So, what do we learn from this:\n", + " - Evaluation metrics can be sensitive, and RAGAS itself relies on LLMs for judgments, introducing variability.\n", + " - However, the consistent qualitative improvement and the positive LangSmith results suggest an overall benefit from fine-tuning in this case.\n", + "\n", + "**Run Two**\n", + "\n", + "| Metric | Base Model | Finetuned Model|\n", + "| :------------------------- | :----- | :-------- |\n", + "| context_recall | 0.5913 | 0.6329 |\n", + "| faithfulness | 0.8048 | 0.7716 |\n", + "| factual_correctness | 0.4783 | 0.4508 |\n", + "| answer_relevancy | 0.6075 | 0.6073 |\n", + "| context_entity_recall | 0.5012 | 0.5123 |\n", + "| noise_sensitivity_relevant | 0.3838 | 0.3917 |\n", + "\n", + "**Run One**\n", + "\n", + "| Metric | Base Model | Finetuned model |\n", + "| :------------------------- | :--------- | :-------------- |\n", + "| context_recall | 0.5819 | 0.5864 |\n", + "| faithfulness | 0.8220 | 0.7950 |\n", + "| factual_correctness | 0.3150 | 0.3427 |\n", + "| answer_relevancy | 0.6332 | 0.6332 |\n", + "| context_entity_recall | 0.4674 | 0.4431 |\n", + "| noise_sensitivity_relevant | 0.5305 | 0.6317 |" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "x5787_QrKxsT" + }, + "source": [ + "**OK, Lets try this using LangSmith correctness and helpfulness evaluators we learned in Assignment 7**" + ] + }, + { + "cell_type": "code", + "execution_count": 108, + "metadata": { + "id": "DM8bfIeNiBEn" + }, + "outputs": [], + "source": [ + "from google.colab import userdata\n", + "\n", + "\n", + "os.environ[\"LANGSMITH_API_KEY\"] = userdata.get('LANGSMITH_API_KEY')\n", + "os.environ[\"LANGCHAIN_TRACING\"] = \"true\"\n", + "os.environ[\"LANGCHAIN_PROJECT\"] = \"Assignment 10 LangSmith Trial2\"" + ] + }, + { + "cell_type": "code", + "execution_count": 109, + "metadata": { + "id": "7xzPS6eng3aH" + }, + "outputs": [], + "source": [ + "from langsmith import Client\n", + "from langsmith.evaluation import LangChainStringEvaluator, evaluate\n", + "\n", + "\n", + "client = Client()\n", + "\n", + "dataset_name = \"EmbeddingEval2 10 State of AI Across the Years!\"\n", + "\n", + "langsmith_dataset = client.create_dataset(\n", + " dataset_name=dataset_name,\n", + " description=\"Assignment 10 try 2 State of AI Across the Years!\"\n", + ")\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 110, + "metadata": { + "id": "eVCjmPxtjIWp" + }, + "outputs": [], + "source": [ + "# Convert the dataset from Ragas format to LangSmith format.\n", + "for data_row in sdg_dataset.to_pandas().iterrows():\n", + " client.create_example(\n", + " inputs={\n", + " \"question\": data_row[1][\"user_input\"]\n", + " },\n", + " outputs={\n", + " \"answer\": data_row[1][\"reference\"]\n", + " },\n", + " metadata={\n", + " \"context\": data_row[1][\"reference_contexts\"]\n", + " },\n", + " dataset_id=langsmith_dataset.id\n", + " )" + ] + }, + { + "cell_type": "code", + "execution_count": 111, + "metadata": { + "id": "dZ3PsHNbjZSW" + }, + "outputs": [], + "source": [ + "eval_llm = ChatOpenAI(model=\"gpt-4.1\")\n", + "\n", + "def prep_data(run, example):\n", + " return {\n", + " \"prediction\": run.outputs['response'], # Map 'response' key to 'prediction'\n", + " \"reference\": example.outputs['answer'], # Map 'answer' key to 'reference'\n", + " \"input\": example.inputs['question'], # Map 'question' key to 'input' (or 'query' depending on the evaluator's prompt)\n", + " }\n", + "\n", + "qa_evaluator = LangChainStringEvaluator(\"qa\", config={\"llm\" : eval_llm},prepare_data=prep_data)\n", + "\n", + "labeled_helpfulness_evaluator = LangChainStringEvaluator(\n", + " \"labeled_criteria\",\n", + " config={\n", + " \"criteria\": {\n", + " \"helpfulness\": (\n", + " \"Is this submission helpful to the user,\"\n", + " \" taking into account the correct reference answer?\"\n", + " )\n", + " },\n", + " \"llm\" : eval_llm\n", + " },\n", + " prepare_data=prep_data\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 112, + "metadata": { + "id": "oiJL9ap7h7z4", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000, + "referenced_widgets": [ + "8cffb4e48fff4c31adb0418b59c9414f", + "2d457df813bd4c56a0520b4d3b7f5c62", + "02c5d66fd4634d25aa5a675ae96f0d3d", + "9074775dc05d40d395c7382c46dc0a90", + "0f4de30cc0fa4c7f8c2a81d60c84442e", + "7528a1061fe9417999e7c27090a449fb", + "2c91cac53c8145f7aedb343466462f8d", + "363e420f11114d65bec0d52591c9da4b", + "7d71ebfef41441d6b725b1a673124a40", + "7005ba609e9644d19e64dd04efed2656", + "b36ea90a25bf4667a77b2889b1e2e36b" + ] + }, + "outputId": "06b3332e-c2be-4451-80cd-0d3bc41a8ba3" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "View the evaluation results for experiment: 'slight-title-55' at:\n", + "https://smith.langchain.com/o/39f31aed-fcde-5ca7-a4f1-a8447f10d513/datasets/b6fa05a8-e59d-409d-924d-8ba401da0a61/compare?selectedSessions=6624123c-2f23-4f32-9b06-126e7368181f\n", + "\n", + "\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "0it [00:00, ?it/s]" + ], + "application/vnd.jupyter.widget-view+json": { + "version_major": 2, + "version_minor": 0, + "model_id": "8cffb4e48fff4c31adb0418b59c9414f" + } + }, + "metadata": {} + }, + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
inputs.questionoutputs.responseoutputs.contexterrorreference.answerfeedback.correctnessfeedback.helpfulnessexecution_timeexample_idid
0Why did the duckling feel thankful for being u...The provided context does not indicate that th...[page_content='\"I believe I must go out into t...NoneThe duckling felt thankful for being ugly beca...000.9452692e3c3684-fba9-42f9-b04a-9aaf39b9717378e9e331-3765-426b-b1c9-2753a4e74678
1How does the arrival of autumn affect Thumbeli...The provided context does not include informat...[page_content='\"Yes, I will go with you,\" said...NoneThe arrival of autumn significantly impacts Th...000.612981cefd889f-5da5-4a1d-be14-1b1737cc7a70e2bf7d33-ea41-4f4c-9b22-e2f2e0c1fd82
2How does the theme of spring manifest in the e...I do not know.[page_content='sparrow flew, for she saw many ...NoneIn the provided fairy tale context, the theme ...000.618191a7a6ced9-a9bb-4648-8d54-f0e01cf91bad171d018b-6093-46e5-bd06-34beefc7cd05
3What do the gods require of a man to live a di...According to the teachings in the context, the...[page_content='seest that those things, which ...NoneThe gods require no more of any man than to ke...100.74034124297489-b472-46c2-b91a-b16d2b7b80efe42c87b1-c2fd-4b58-a61d-afa130381f70
4In what ways does Plato's Apology reflect the ...Plato's Apology reflects the themes of rhetori...[page_content='Yet some of the topics may have...NonePlato's Apology reflects the themes of rhetori...111.0941837f273223-0e18-47cd-aba8-1ae2b91e8f5ec9f1e164-aadc-478e-9d4e-7031d791a918
5In what ways does Plato's 'Apology' reflect th...The provided context suggests that Plato's \"Ap...[page_content='Yet some of the topics may have...NonePlato's 'Apology' reflects the themes of rheto...101.850498973d3b87-4ea7-4f16-a53c-572a47cbe3a9ecccb67f-481e-4a7b-b58a-68502806bc51
6In Plato's Apology, how does Socrates' rhetori...The provided context does not explicitly detai...[page_content='Yet some of the topics may have...NoneIn Plato's Apology, Socrates employs a unique ...000.7786258dcd4065-85ae-483b-8b0a-f33f2570479bb2884ecd-b035-4408-bd59-4302b950d852
7In what ways does Socrates' defense in Plato's...The provided context does not contain specific...[page_content='The β€œApology” or Platonic defen...NoneSocrates' defense in Plato's 'Apology' reflect...001.3125589f62caca-c477-4e96-8b80-0ccc9f9cce5f20734421-9989-4bd9-81f4-3f6c1d5eade1
8What does the sun represent for the fir tree i...The sun represents a source of joy and vitalit...[page_content='\"Rejoice in our love,\" said the...NoneThe sun represents warmth and joy for the fir ...100.725027306eaafa-2ead-485d-b9f1-68a8ddb975b22efcd4ae-64b7-46ba-a894-d9b99f345787
9Who is Edna F. Hart in relation to Hans Anders...The provided context does not mention Edna F. ...[page_content='carefully standardized, their p...NoneEdna F. Hart is the illustrator of Hans Anders...000.58920990635fe6-8fea-4756-b2a4-d0de2200133c406872e8-1c60-46c4-b7ce-1d5e42cf6af7
10How does Hans Andersen's storytelling ability ...Hans Andersen's storytelling ability contribut...[page_content='carefully standardized, their p...NoneHans Andersen's storytelling ability is charac...111.1298655c530684-ef34-4056-a712-1883ede23eaf5717e371-b4cf-487c-a06c-d30595827eca
11Who J.H. Stickney and what he do for Hans Ande...J. H. Stickney is the person who reprinted and...[page_content='carefully standardized, their p...NoneJ.H. Stickney is the editor of Hans Andersen's...110.698241594afc92-fb0e-4ac5-b183-bbbc985584efafd7d2c3-607f-418c-9b2b-58dba702b274
\n", + "
" + ] + }, + "metadata": {}, + "execution_count": 112 + } + ], + "source": [ + "evaluate(\n", + " base_rag_chain.invoke,\n", + " data=dataset_name,\n", + " evaluators=[\n", + " qa_evaluator,\n", + " labeled_helpfulness_evaluator\n", + "\n", + " ],\n", + " metadata={\"revision_id\": \"Base_Embedding_Model\"},\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 113, + "metadata": { + "id": "f3J2gZych-xu", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000, + "referenced_widgets": [ + "c612f196069d46b89d8f9abde9f32fd5", + "2f189591fd0446fca62e1ae0b3c68671", + "68c0cd627a9746e090585deb5e10850c", + "b3a4ec7b469844d2b35f613d924d824b", + "e5b2b4e1f7ff464ebd23fbfb2bb39c87", + "e4c3c159bc5945988866d84e1fef5446", + "28d34e0483824662b8e82f58834022c6", + "3b1850b6a67a475cb63a6816592aa840", + "a3286232e80e4fb79d0c3e73a223ca79", + "18a49867fe05498da609dbf6eb23a191", + "4aa3510b80f84c279ffee3db03a81b3e" + ] + }, + "outputId": "251e2e48-77aa-4441-b508-5ffa4e2347ad" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "View the evaluation results for experiment: 'upbeat-pump-30' at:\n", + "https://smith.langchain.com/o/39f31aed-fcde-5ca7-a4f1-a8447f10d513/datasets/b6fa05a8-e59d-409d-924d-8ba401da0a61/compare?selectedSessions=ac150feb-4288-449d-a439-fb80db136eba\n", + "\n", + "\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "0it [00:00, ?it/s]" + ], + "application/vnd.jupyter.widget-view+json": { + "version_major": 2, + "version_minor": 0, + "model_id": "c612f196069d46b89d8f9abde9f32fd5" + } + }, + "metadata": {} + }, + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
inputs.questionoutputs.responseoutputs.contexterrorreference.answerfeedback.correctnessfeedback.helpfulnessexecution_timeexample_idid
0Why did the duckling feel thankful for being u...The provided context does not indicate that th...[page_content='\"I believe I must go out into t...NoneThe duckling felt thankful for being ugly beca...001.5873172e3c3684-fba9-42f9-b04a-9aaf39b971732212e353-ca1d-47d7-a82e-0cec0db4acaf
1How does the arrival of autumn affect Thumbeli...The arrival of autumn makes Thumbelina's situa...[page_content='Every morning when the sun rose...NoneThe arrival of autumn significantly impacts Th...111.336359cefd889f-5da5-4a1d-be14-1b1737cc7a701364b1c8-dced-48ee-9fce-7576f1c41dd6
2How does the theme of spring manifest in the e...The theme of spring manifests in the character...[page_content='[Illustration]\\n\\n\\n\\n\\nTHE CON...NoneIn the provided fairy tale context, the theme ...111.063394a7a6ced9-a9bb-4648-8d54-f0e01cf91badadda52d7-deab-4124-a77e-1bd8457621ea
3What do the gods require of a man to live a di...According to the teachings in the context, the...[page_content='seest that those things, which ...NoneThe gods require no more of any man than to ke...101.00223524297489-b472-46c2-b91a-b16d2b7b80ef700366de-344b-4b46-9fe9-b6827f5d50b2
4In what ways does Plato's Apology reflect the ...Based on the provided context, Plato's Apology...[page_content='Yet some of the topics may have...NonePlato's Apology reflects the themes of rhetori...101.2701197f273223-0e18-47cd-aba8-1ae2b91e8f5efc7b5f58-b4db-4c8a-be98-45b32d4041c2
5In what ways does Plato's 'Apology' reflect th...The provided context suggests that Plato's 'Ap...[page_content='Yet some of the topics may have...NonePlato's 'Apology' reflects the themes of rheto...111.837373973d3b87-4ea7-4f16-a53c-572a47cbe3a960dc1163-e277-44e6-b55c-9b76656af23a
6In Plato's Apology, how does Socrates' rhetori...The provided context indicates that Plato’s β€œA...[page_content='Yet some of the topics may have...NoneIn Plato's Apology, Socrates employs a unique ...111.6547688dcd4065-85ae-483b-8b0a-f33f2570479b1bf8a317-b2fb-4277-9871-b23f0036cba6
7In what ways does Socrates' defense in Plato's...The provided context does not explicitly discu...[page_content='β€œFew persons will be found to w...NoneSocrates' defense in Plato's 'Apology' reflect...000.5867669f62caca-c477-4e96-8b80-0ccc9f9cce5f1b9506ad-87d7-4cea-a2cf-a15dfa59e16f
8What does the sun represent for the fir tree i...The provided context does not explicitly state...[page_content='[Illustration: They danced merr...NoneThe sun represents warmth and joy for the fir ...000.521279306eaafa-2ead-485d-b9f1-68a8ddb975b27ed3f5a9-e34b-4db8-942a-e75747a07a80
9Who is Edna F. Hart in relation to Hans Anders...Edna F. Hart is the illustrator of Hans Anders...[page_content='Language: English\\n\\nCharacter ...NoneEdna F. Hart is the illustrator of Hans Anders...110.54566590635fe6-8fea-4756-b2a4-d0de2200133c140e57fe-d1e2-4483-976c-36ec8ac8d62b
10How does Hans Andersen's storytelling ability ...Hans Andersen's storytelling ability contribut...[page_content='The Hans Andersen Fairy Tales w...NoneHans Andersen's storytelling ability is charac...110.7998085c530684-ef34-4056-a712-1883ede23eaf33b85727-8ca7-40c7-8dec-3352d17ddd89
11Who J.H. Stickney and what he do for Hans Ande...J.H. Stickney is the editor of Hans Andersen's...[page_content='Language: English\\n\\nCharacter ...NoneJ.H. Stickney is the editor of Hans Andersen's...110.528632594afc92-fb0e-4ac5-b183-bbbc985584efc9a98676-53a4-49b0-9b15-097a9ba384d4
\n", + "
" + ] + }, + "metadata": {}, + "execution_count": 113 + } + ], + "source": [ + "evaluate(\n", + " finetune_rag_chain.invoke,\n", + " data=dataset_name,\n", + " evaluators=[\n", + " qa_evaluator,\n", + " labeled_helpfulness_evaluator\n", + " ],\n", + " metadata={\"revision_id\": \"Finetune_Embedding_Model\"},\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Fxj-t2KlKxsT" + }, + "source": [ + "**Langsmith Evaluation Results: Much better!**\n", + "\n", + " Bingo! Langsmith evaluation shows that our fine tuned model got better scores. I tried three runs and the results were the same in each." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "N3upjDhlKxsT" + }, + "source": [ + "![image](ls_eval.png)\n", + "\n" + ] + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "gpuType": "A100", + "machine_shape": "hm", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.2" + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "b6301e443fd84844991a50f6950a34d3": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_108a14d3e60c48979e767e71286b1f9f", + "IPY_MODEL_1769f34120624f7f99a886f774cbcd7e", + "IPY_MODEL_0c7de7a830164795897d1e134fdf3524" + ], + "layout": "IPY_MODEL_1bde6edcfbc64cabbda95a18ee65bfef" + } + }, + "108a14d3e60c48979e767e71286b1f9f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_828d57002ab94e90b6a5b8f404765cb3", + "placeholder": "​", + "style": "IPY_MODEL_e88cad91f2a8499989dad61394725350", + "value": "modules.json: 100%" + } + }, + "1769f34120624f7f99a886f774cbcd7e": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_80e211d62556464dbb71de7dee22f845", + "max": 349, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_c64275e3c65d4599b8c00eaf5400ee6a", + "value": 349 + } + }, + "0c7de7a830164795897d1e134fdf3524": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_0360b811cd314f94aa4e2d4472434a4b", + "placeholder": "​", + "style": "IPY_MODEL_ffd849455079460a81f045505ea04d75", + "value": " 349/349 [00:00<00:00, 41.4kB/s]" + } + }, + "1bde6edcfbc64cabbda95a18ee65bfef": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "828d57002ab94e90b6a5b8f404765cb3": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "e88cad91f2a8499989dad61394725350": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "80e211d62556464dbb71de7dee22f845": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "c64275e3c65d4599b8c00eaf5400ee6a": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "0360b811cd314f94aa4e2d4472434a4b": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "ffd849455079460a81f045505ea04d75": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "cb6b64a5fef5428897527bcf1e231e7f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_dd86e848207f418aaa6de7e06f6502cd", + "IPY_MODEL_0fd9e2d62cf242578da61c79366b9e35", + "IPY_MODEL_695a33a4bab94fd2a70aab044b0356e0" + ], + "layout": "IPY_MODEL_bb57cdd9477641caa1335c7cb464d5e9" + } + }, + "dd86e848207f418aaa6de7e06f6502cd": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_b8d4e32967ef4a5bb4ff8ebff39b244e", + "placeholder": "​", + "style": "IPY_MODEL_0dd4478d22bf4734b97db1e78a5ca548", + "value": "config_sentence_transformers.json: 100%" + } + }, + "0fd9e2d62cf242578da61c79366b9e35": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_ed61b2a09c444747bf61501c045f5acf", + "max": 252, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_4279ec0cb1fe4c899f0f6ccf7ed8a8b4", + "value": 252 + } + }, + "695a33a4bab94fd2a70aab044b0356e0": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_af6b15f7163947ecb713157084c20dfc", + "placeholder": "​", + "style": "IPY_MODEL_c839234d06d44aa8973ebfa1f4e0a100", + "value": " 252/252 [00:00<00:00, 23.3kB/s]" + } + }, + "bb57cdd9477641caa1335c7cb464d5e9": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "b8d4e32967ef4a5bb4ff8ebff39b244e": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "0dd4478d22bf4734b97db1e78a5ca548": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "ed61b2a09c444747bf61501c045f5acf": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "4279ec0cb1fe4c899f0f6ccf7ed8a8b4": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "af6b15f7163947ecb713157084c20dfc": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "c839234d06d44aa8973ebfa1f4e0a100": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "a485f31a2abc49498b5c59e3523fc9be": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_699b42c314cd437b95cda77f7e7384d4", + "IPY_MODEL_7dbb2e58cf874ad5bf11809d7a9d1258", + "IPY_MODEL_651e0533fa8344bc849904ad867c1922" + ], + "layout": "IPY_MODEL_766e15fa5aed44ad86a693eb43095c13" + } + }, + "699b42c314cd437b95cda77f7e7384d4": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_de9f359a37b3450d857fa24f32aacfe4", + "placeholder": "​", + "style": "IPY_MODEL_6f5911d1d54540a380b2c9822c026ba1", + "value": "README.md: 100%" + } + }, + "7dbb2e58cf874ad5bf11809d7a9d1258": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_a993da3455df4454add4623af76d644b", + "max": 85438, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_d8394e5f45894a7387f2544aa7eef6c5", + "value": 85438 + } + }, + "651e0533fa8344bc849904ad867c1922": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_d371178a535c4ff8a777d531caeabe7d", + "placeholder": "​", + "style": "IPY_MODEL_2dbe6450b9284a9fbfa3c80eeb18e36f", + "value": " 85.4k/85.4k [00:00<00:00, 9.89MB/s]" + } + }, + "766e15fa5aed44ad86a693eb43095c13": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "de9f359a37b3450d857fa24f32aacfe4": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "6f5911d1d54540a380b2c9822c026ba1": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "a993da3455df4454add4623af76d644b": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "d8394e5f45894a7387f2544aa7eef6c5": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "d371178a535c4ff8a777d531caeabe7d": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "2dbe6450b9284a9fbfa3c80eeb18e36f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "a57f01a0a0084ea1a0d6d0071c2cab3e": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_efbef2a276c54f288392498f2c9be5dd", + "IPY_MODEL_8661a966d644457f8fb9afacbe5c1974", + "IPY_MODEL_9589b453269e4a4abe514050134ff3a0" + ], + "layout": "IPY_MODEL_ee2e792c16774e33976fbf6b613c904a" + } + }, + "efbef2a276c54f288392498f2c9be5dd": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_16d5892b57b841458c5149de98c3f4ab", + "placeholder": "​", + "style": "IPY_MODEL_ae8158ffe4234d4885e45936e0fa53f8", + "value": "sentence_bert_config.json: 100%" + } + }, + "8661a966d644457f8fb9afacbe5c1974": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_2cc996696fe54861b6587b8c1bebb911", + "max": 107, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_e9c541335cec4e019fc6f03e3581e886", + "value": 107 + } + }, + "9589b453269e4a4abe514050134ff3a0": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_b58d32b86f1c43d5b8c0ac2ad9c4a670", + "placeholder": "​", + "style": "IPY_MODEL_e499f8b0fce64b1ab5e3f7dc0bf4228b", + "value": " 107/107 [00:00<00:00, 13.1kB/s]" + } + }, + "ee2e792c16774e33976fbf6b613c904a": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "16d5892b57b841458c5149de98c3f4ab": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "ae8158ffe4234d4885e45936e0fa53f8": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "2cc996696fe54861b6587b8c1bebb911": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "e9c541335cec4e019fc6f03e3581e886": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "b58d32b86f1c43d5b8c0ac2ad9c4a670": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "e499f8b0fce64b1ab5e3f7dc0bf4228b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "503d81fc173a404d94087c33d9158a41": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_026bed8e4fe8484f9e443d6982c67fe2", + "IPY_MODEL_644d9e8fb743490ea6f0d2c6c8be7a74", + "IPY_MODEL_7d0dffb18e554de6a7130df37f8edcaf" + ], + "layout": "IPY_MODEL_d17b4c123cd54122b69c2a615fb01bbc" + } + }, + "026bed8e4fe8484f9e443d6982c67fe2": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_83f3d5d59e7d4fe48616fcfa7d96818d", + "placeholder": "​", + "style": "IPY_MODEL_c329a098e4074edb814f9b3982f6c7d4", + "value": "config.json: 100%" + } + }, + "644d9e8fb743490ea6f0d2c6c8be7a74": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_1a78ad30d346495984e02b46c37d51f1", + "max": 704, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_d6dc2f432caf4607afe065384421b53c", + "value": 704 + } + }, + "7d0dffb18e554de6a7130df37f8edcaf": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_40cfca37a32d4a668a2755c0e455127f", + "placeholder": "​", + "style": "IPY_MODEL_20b62a0ee2a441a995b9143f1b62b7da", + "value": " 704/704 [00:00<00:00, 85.8kB/s]" + } + }, + "d17b4c123cd54122b69c2a615fb01bbc": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "83f3d5d59e7d4fe48616fcfa7d96818d": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "c329a098e4074edb814f9b3982f6c7d4": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "1a78ad30d346495984e02b46c37d51f1": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "d6dc2f432caf4607afe065384421b53c": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "40cfca37a32d4a668a2755c0e455127f": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "20b62a0ee2a441a995b9143f1b62b7da": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "cfa7400579474370a5351e5b4cc2eebe": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_45940be0514e45e98839f67746ada83b", + "IPY_MODEL_8f6b27ae83754c309c0be766f475aee9", + "IPY_MODEL_18e1ade2b72d4b42ae7ebb4139e4892c" + ], + "layout": "IPY_MODEL_099f1441619245c793fefe89101db7c3" + } + }, + "45940be0514e45e98839f67746ada83b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_528cd3e59bcc46bbb3ed2db02d30c20d", + "placeholder": "​", + "style": "IPY_MODEL_8bd9f734a7a24a77b67ae691ecdb6882", + "value": "model.safetensors: 100%" + } + }, + "8f6b27ae83754c309c0be766f475aee9": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_72272bf41f2548d9a19134adc38e4645", + "max": 1336413848, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_8cfd6ee1ab2240b6b2f4cf56650781e1", + "value": 1336413848 + } + }, + "18e1ade2b72d4b42ae7ebb4139e4892c": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_7564568b8cca457c86f7dd81cf86f3f9", + "placeholder": "​", + "style": "IPY_MODEL_7cf5122a6599450a844bd164fdff5d8c", + "value": " 1.34G/1.34G [00:04<00:00, 362MB/s]" + } + }, + "099f1441619245c793fefe89101db7c3": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "528cd3e59bcc46bbb3ed2db02d30c20d": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "8bd9f734a7a24a77b67ae691ecdb6882": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "72272bf41f2548d9a19134adc38e4645": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "8cfd6ee1ab2240b6b2f4cf56650781e1": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "7564568b8cca457c86f7dd81cf86f3f9": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7cf5122a6599450a844bd164fdff5d8c": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "95daedff61324647bf2e597df23f4b8c": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_fca84d17cad449de943c3374710f9bbc", + "IPY_MODEL_085e6f70bf984f0896e140b636cf34bf", + "IPY_MODEL_e2e6db6f7ac84bbf9084afafbfab0056" + ], + "layout": "IPY_MODEL_bf821f44ed94401a86a822f4b8a46894" + } + }, + "fca84d17cad449de943c3374710f9bbc": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_0575a67c1fb04d5ebd814baa07994817", + "placeholder": "​", + "style": "IPY_MODEL_21ef965a52904d128bc7426088f9083f", + "value": "tokenizer_config.json: 100%" + } + }, + "085e6f70bf984f0896e140b636cf34bf": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_3c8dbcbe555949c9ab9df3c3ceaedede", + "max": 1381, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_7e280a7580ce49328d2e0e2e11d9ae56", + "value": 1381 + } + }, + "e2e6db6f7ac84bbf9084afafbfab0056": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_26ba9f4e69f941e09c2827e6fe0fd2d3", + "placeholder": "​", + "style": "IPY_MODEL_74137c8f95ff4af79826f8921683820d", + "value": " 1.38k/1.38k [00:00<00:00, 118kB/s]" + } + }, + "bf821f44ed94401a86a822f4b8a46894": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "0575a67c1fb04d5ebd814baa07994817": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "21ef965a52904d128bc7426088f9083f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "3c8dbcbe555949c9ab9df3c3ceaedede": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7e280a7580ce49328d2e0e2e11d9ae56": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "26ba9f4e69f941e09c2827e6fe0fd2d3": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "74137c8f95ff4af79826f8921683820d": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "976eba6063754ce1abc0e9c26b4e7dcd": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_1c29d94d2dcb443ab875280651ca4c6c", + "IPY_MODEL_1876b833ddd148c1b11f92763ef4dedc", + "IPY_MODEL_db20e7a134a8431eb6854a7167a27c46" + ], + "layout": "IPY_MODEL_c95ebfbacab14e43b1885c5b02e91285" + } + }, + "1c29d94d2dcb443ab875280651ca4c6c": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_e81544773c7f4a64adca17f543281abc", + "placeholder": "​", + "style": "IPY_MODEL_1eb2314e77b44aa1b3d324cf097169a3", + "value": "vocab.txt: 100%" + } + }, + "1876b833ddd148c1b11f92763ef4dedc": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_ccd97ed6ad20478fa4ee67011a3d37db", + "max": 231508, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_11fb745d929b471ba0c00bd70e22cfc7", + "value": 231508 + } + }, + "db20e7a134a8431eb6854a7167a27c46": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_cb8d61ae7dca484488f3ea43e983a6fb", + "placeholder": "​", + "style": "IPY_MODEL_c9372109bc004e06a2c47908da32e536", + "value": " 232k/232k [00:00<00:00, 1.43MB/s]" + } + }, + "c95ebfbacab14e43b1885c5b02e91285": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "e81544773c7f4a64adca17f543281abc": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "1eb2314e77b44aa1b3d324cf097169a3": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "ccd97ed6ad20478fa4ee67011a3d37db": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "11fb745d929b471ba0c00bd70e22cfc7": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "cb8d61ae7dca484488f3ea43e983a6fb": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "c9372109bc004e06a2c47908da32e536": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "f257141a6f18401dad79a6efd148124f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_3869f8481e2947b888fd2b829e607601", + "IPY_MODEL_6396369ab3d34f08adffebcd457ae47b", + "IPY_MODEL_9774c7359a634c639c180fc2b316839f" + ], + "layout": "IPY_MODEL_53eed3f0eb644e3ab2617d60c91eacc6" + } + }, + "3869f8481e2947b888fd2b829e607601": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_b3314162932f491da150cd7cf07cd2d1", + "placeholder": "​", + "style": "IPY_MODEL_12db42dd32c448d4a5d4f0d1c6406eea", + "value": "tokenizer.json: 100%" + } + }, + "6396369ab3d34f08adffebcd457ae47b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_cede1a8f47294b73872192c997a72fba", + "max": 711649, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_d2441d036c3341be95cc0b1a43264073", + "value": 711649 + } + }, + "9774c7359a634c639c180fc2b316839f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_67d852327a7a48b686b48aee7c262c53", + "placeholder": "​", + "style": "IPY_MODEL_31b65d68e1084b0c86fc1f3d4f6baab1", + "value": " 712k/712k [00:00<00:00, 8.80MB/s]" + } + }, + "53eed3f0eb644e3ab2617d60c91eacc6": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "b3314162932f491da150cd7cf07cd2d1": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "12db42dd32c448d4a5d4f0d1c6406eea": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "cede1a8f47294b73872192c997a72fba": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "d2441d036c3341be95cc0b1a43264073": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "67d852327a7a48b686b48aee7c262c53": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "31b65d68e1084b0c86fc1f3d4f6baab1": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "7cca7d66ca114cc6a28d6f462eef9a8e": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_f59f485740b24966be6f76d31c5aae83", + "IPY_MODEL_1c460d52be2a4d1fb8e6ececf35048f7", + "IPY_MODEL_6a96db6392894eb899a161cd0c480bc8" + ], + "layout": "IPY_MODEL_ddbd04ecee6c4c739933216ac901a28f" + } + }, + "f59f485740b24966be6f76d31c5aae83": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_38391ebc2e2a42ac8a99214e78131464", + "placeholder": "​", + "style": "IPY_MODEL_ea1018a84a5842329bb3a5ec91100a2d", + "value": "special_tokens_map.json: 100%" + } + }, + "1c460d52be2a4d1fb8e6ececf35048f7": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_13f077624ab74511bbec8cfd409f1f7d", + "max": 695, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_c2bb7ebd3328454d94d854b5b1dec4ed", + "value": 695 + } + }, + "6a96db6392894eb899a161cd0c480bc8": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_6f98b0fcfa8d4ea28f0f903d34f5427a", + "placeholder": "​", + "style": "IPY_MODEL_7ca4c152e1cc47fe99bb440d5565e44d", + "value": " 695/695 [00:00<00:00, 91.4kB/s]" + } + }, + "ddbd04ecee6c4c739933216ac901a28f": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "38391ebc2e2a42ac8a99214e78131464": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "ea1018a84a5842329bb3a5ec91100a2d": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "13f077624ab74511bbec8cfd409f1f7d": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "c2bb7ebd3328454d94d854b5b1dec4ed": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "6f98b0fcfa8d4ea28f0f903d34f5427a": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7ca4c152e1cc47fe99bb440d5565e44d": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "4ae9037538584adaa96e80415239a8fa": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_0ce565970c1d40328ac54a157609902f", + "IPY_MODEL_0f2ddc6bafcd4066a9945fa0f1c517e2", + "IPY_MODEL_eb9e77674003464eae6fd0bbea9d1361" + ], + "layout": "IPY_MODEL_3eebface9bcf4bd48a98a3961a474d15" + } + }, + "0ce565970c1d40328ac54a157609902f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_5790b2e22323416a80d56da7a78e3afb", + "placeholder": "​", + "style": "IPY_MODEL_d243906ef5254edf8f1a1f015998d0f2", + "value": "config.json: 100%" + } + }, + "0f2ddc6bafcd4066a9945fa0f1c517e2": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_ca938b533b934559829d7a05b95f54bc", + "max": 297, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_690cda34446c43d7b77437c4a6807a8b", + "value": 297 + } + }, + "eb9e77674003464eae6fd0bbea9d1361": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_0ce8cda0337e4250be7c37f2216eb4c1", + "placeholder": "​", + "style": "IPY_MODEL_56af82497ec64575a65ad47efd13fbe3", + "value": " 297/297 [00:00<00:00, 38.6kB/s]" + } + }, + "3eebface9bcf4bd48a98a3961a474d15": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "5790b2e22323416a80d56da7a78e3afb": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "d243906ef5254edf8f1a1f015998d0f2": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "ca938b533b934559829d7a05b95f54bc": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "690cda34446c43d7b77437c4a6807a8b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "0ce8cda0337e4250be7c37f2216eb4c1": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "56af82497ec64575a65ad47efd13fbe3": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "c1bdc2bee22a4028a1622f2fa207369f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "VBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "VBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "VBoxView", + "box_style": "", + "children": [], + "layout": "IPY_MODEL_b68f3642a0d54b99b10f7a63c793f1ac" + } + }, + "f93934fcb1f74359a786503cb2771a53": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_4d12dd2fd5964d34ab9e83d0ff01570e", + "placeholder": "​", + "style": "IPY_MODEL_480ecbe8908644ff957102c25064025e", + "value": "

Copy a token from your Hugging Face\ntokens page and paste it below.
Immediately click login after copying\nyour token or it might be stored in plain text in this notebook file.
" + } + }, + "e76ee26c1e0540a6855b4c2a09143084": { + "model_module": "@jupyter-widgets/controls", + "model_name": "PasswordModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "PasswordModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "PasswordView", + "continuous_update": true, + "description": "Token:", + "description_tooltip": null, + "disabled": false, + "layout": "IPY_MODEL_3487ade73bd8403186ad7af1b4eb503a", + "placeholder": "​", + "style": "IPY_MODEL_62edd19b390943aab76f8f717dc3de66", + "value": "" + } + }, + "740ec743ea6742549d54eb1d1cbd2e20": { + "model_module": "@jupyter-widgets/controls", + "model_name": "CheckboxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "CheckboxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "CheckboxView", + "description": "Add token as git credential?", + "description_tooltip": null, + "disabled": false, + "indent": true, + "layout": "IPY_MODEL_e3fdb79108064ecdab361420f97a8e4e", + "style": "IPY_MODEL_a05930373152409c956ce7a33cbd489b", + "value": true + } + }, + "482613b5c1bb4b53b4052d050dc78638": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ButtonModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ButtonModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ButtonView", + "button_style": "", + "description": "Login", + "disabled": false, + "icon": "", + "layout": "IPY_MODEL_a99da24492b049dc91c6c87c73047f35", + "style": "IPY_MODEL_416d61f2e90f4d70abc732f15e9b289d", + "tooltip": "" + } + }, + "589bf123f49b45f5b1ad5968e8a6abea": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_f177c091937f406488825e8569de3390", + "placeholder": "​", + "style": "IPY_MODEL_8d72c745b53747ddaafb0fc1bfbf799f", + "value": "\nPro Tip: If you don't already have one, you can create a dedicated\n'notebooks' token with 'write' access, that you can then easily reuse for all\nnotebooks.
" + } + }, + "b68f3642a0d54b99b10f7a63c793f1ac": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": "center", + "align_self": null, + "border": null, + "bottom": null, + "display": "flex", + "flex": null, + "flex_flow": "column", + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": "50%" + } + }, + "4d12dd2fd5964d34ab9e83d0ff01570e": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "480ecbe8908644ff957102c25064025e": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "3487ade73bd8403186ad7af1b4eb503a": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "62edd19b390943aab76f8f717dc3de66": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "e3fdb79108064ecdab361420f97a8e4e": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "a05930373152409c956ce7a33cbd489b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "a99da24492b049dc91c6c87c73047f35": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "416d61f2e90f4d70abc732f15e9b289d": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ButtonStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ButtonStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "button_color": null, + "font_weight": "" + } + }, + "f177c091937f406488825e8569de3390": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "8d72c745b53747ddaafb0fc1bfbf799f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "340204007ee847e78a882b2c204a730f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "LabelModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "LabelModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "LabelView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_2e4ea07138c6451aa8bf003ee195cd0d", + "placeholder": "​", + "style": "IPY_MODEL_30fc4c21dfea4c76b52ca610759ba2bd", + "value": "Connecting..." + } + }, + "2e4ea07138c6451aa8bf003ee195cd0d": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "30fc4c21dfea4c76b52ca610759ba2bd": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "2f21da003dfa4bbebc4ca03f956c834b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_1fee7378391744f4856b89abd10dcbfb", + "IPY_MODEL_649dc610bb384307aa899b0bde22ede6", + "IPY_MODEL_a1d2069710c4497bbf7c1f42ccb964ba" + ], + "layout": "IPY_MODEL_fca4a7f443e146a9a27b9692183c7a7d" + } + }, + "1fee7378391744f4856b89abd10dcbfb": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_9860950f9461453c99266ce7563d1eac", + "placeholder": "​", + "style": "IPY_MODEL_d05751067889425b80bc1211b3cc9ee7", + "value": "model.safetensors: 100%" + } + }, + "649dc610bb384307aa899b0bde22ede6": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_3b913bea15474fc482080f4f6246c3b8", + "max": 1336413848, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_5e8519b28ac24ec9ae466165fd5e627f", + "value": 1336413848 + } + }, + "a1d2069710c4497bbf7c1f42ccb964ba": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_0ab38f3eb0ed4ec6bac9810979610b25", + "placeholder": "​", + "style": "IPY_MODEL_2b6ee10eec8149b0ba4807f33c2e18ec", + "value": " 1.34G/1.34G [00:37<00:00, 11.1MB/s]" + } + }, + "fca4a7f443e146a9a27b9692183c7a7d": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "9860950f9461453c99266ce7563d1eac": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "d05751067889425b80bc1211b3cc9ee7": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "3b913bea15474fc482080f4f6246c3b8": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "5e8519b28ac24ec9ae466165fd5e627f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "0ab38f3eb0ed4ec6bac9810979610b25": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "2b6ee10eec8149b0ba4807f33c2e18ec": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "68d08d2c70a44ae68cfc2a5885e83cb1": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_d426d1252b1c443eaa1d7079cbca2b54", + "IPY_MODEL_b3968843072b4551b7ec6c807b673b72", + "IPY_MODEL_c6c8e0fdade24e83a133b4f98477774a" + ], + "layout": "IPY_MODEL_b7f5d55603814fee86de87cfe4c1d433" + } + }, + "d426d1252b1c443eaa1d7079cbca2b54": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_da141c357a764dcab7441532e0b98c43", + "placeholder": "​", + "style": "IPY_MODEL_0c024e9045904cad81fe7d2f11b97bf9", + "value": "Applying HeadlinesExtractor: 100%" + } + }, + "b3968843072b4551b7ec6c807b673b72": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_ecaf62a0b90a4d39bca075753e495508", + "max": 10, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_43e42e2ff62f44d58423249242effa17", + "value": 10 + } + }, + "c6c8e0fdade24e83a133b4f98477774a": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_8d5370219dfa4f4193210d10cfa3565e", + "placeholder": "​", + "style": "IPY_MODEL_9b6991490078438995b17c476c39e2ad", + "value": " 10/10 [00:10<00:00,  1.70s/it]" + } + }, + "b7f5d55603814fee86de87cfe4c1d433": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": "hidden", + "width": null + } + }, + "da141c357a764dcab7441532e0b98c43": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "0c024e9045904cad81fe7d2f11b97bf9": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "ecaf62a0b90a4d39bca075753e495508": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "43e42e2ff62f44d58423249242effa17": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "8d5370219dfa4f4193210d10cfa3565e": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "9b6991490078438995b17c476c39e2ad": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "cf446e2820fc48178a98809bd9f82bc5": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_67c62bd1c70d43489b841b0981e00511", + "IPY_MODEL_1aeba09816e247b3afaa45c878d41899", + "IPY_MODEL_7d24f2ba4f694bbc8b43756e877712aa" + ], + "layout": "IPY_MODEL_abbf513aa08d44cb94aeba577414c560" + } + }, + "67c62bd1c70d43489b841b0981e00511": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_ce2ab19759e248c2b73760921797d576", + "placeholder": "​", + "style": "IPY_MODEL_6bc39ff9a8604c4b8cae1453bdf55fc7", + "value": "Applying HeadlineSplitter:   0%" + } + }, + "1aeba09816e247b3afaa45c878d41899": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_22cf143a0e234510889df3a3d17318ed", + "max": 18, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_38722dbe9db44ce5a3337bc7d7e7afdf", + "value": 18 + } + }, + "7d24f2ba4f694bbc8b43756e877712aa": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_154fb399e08a467f8fbad618e386ad91", + "placeholder": "​", + "style": "IPY_MODEL_ef2d28f5756947f3ab40b358ef851e62", + "value": " 0/18 [00:00<?, ?it/s]" + } + }, + "abbf513aa08d44cb94aeba577414c560": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": "hidden", + "width": null + } + }, + "ce2ab19759e248c2b73760921797d576": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "6bc39ff9a8604c4b8cae1453bdf55fc7": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "22cf143a0e234510889df3a3d17318ed": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "38722dbe9db44ce5a3337bc7d7e7afdf": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "154fb399e08a467f8fbad618e386ad91": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "ef2d28f5756947f3ab40b358ef851e62": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "451465ccf1ad469b9a0a0e2a8a6f2d61": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_8b479049e375404aac1293a182daeefd", + "IPY_MODEL_bb68fee5c65447ac9403495f855542c2", + "IPY_MODEL_1e9f1f8e3ef94a1eb1434ae328abc94f" + ], + "layout": "IPY_MODEL_42a46644be3b42368b33a39594ab0e2b" + } + }, + "8b479049e375404aac1293a182daeefd": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_13205a4bd8fc41e3b68df4452ceb2a8a", + "placeholder": "​", + "style": "IPY_MODEL_fa82cb2e42ab44cbb2c3666327841330", + "value": "Applying SummaryExtractor: 100%" + } + }, + "bb68fee5c65447ac9403495f855542c2": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_d3f16edb9d164e60a3a2078b815e368f", + "max": 11, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_12f8887f3bfa4d78bdca80c4a5fdce84", + "value": 11 + } + }, + "1e9f1f8e3ef94a1eb1434ae328abc94f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_2a282a4bbc0046848a55d103cf770da2", + "placeholder": "​", + "style": "IPY_MODEL_7737530a1337496388fdf6db04546b29", + "value": " 11/11 [00:05<00:00,  2.72it/s]" + } + }, + "42a46644be3b42368b33a39594ab0e2b": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": "hidden", + "width": null + } + }, + "13205a4bd8fc41e3b68df4452ceb2a8a": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "fa82cb2e42ab44cbb2c3666327841330": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "d3f16edb9d164e60a3a2078b815e368f": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "12f8887f3bfa4d78bdca80c4a5fdce84": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "2a282a4bbc0046848a55d103cf770da2": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7737530a1337496388fdf6db04546b29": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "b130b62aaa0648d589e1db78cefa45dc": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_e60de60334104b6cb0e89f0460cb18af", + "IPY_MODEL_71f21a487fb84576baba26c7d46d39e5", + "IPY_MODEL_f9296de4f2f34224952c6486ea149e58" + ], + "layout": "IPY_MODEL_ae34ca91a0af4ded929efaa27860fe71" + } + }, + "e60de60334104b6cb0e89f0460cb18af": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_7e40823470fc4e439b66da7abb09dc90", + "placeholder": "​", + "style": "IPY_MODEL_1b16421167ab48b8a5917d72d656c7bf", + "value": "Applying CustomNodeFilter:  99%" + } + }, + "71f21a487fb84576baba26c7d46d39e5": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_f7e04af9a15847d5a948a894f610cc49", + "max": 312, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_4327e516e08a4c73b72253077d7e357b", + "value": 312 + } + }, + "f9296de4f2f34224952c6486ea149e58": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_479f97a939d540d7b027414a51d431fb", + "placeholder": "​", + "style": "IPY_MODEL_5df46535a7cf42989003692bf06d4cd8", + "value": " 310/312 [00:11<00:00, 24.44it/s]" + } + }, + "ae34ca91a0af4ded929efaa27860fe71": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": "hidden", + "width": null + } + }, + "7e40823470fc4e439b66da7abb09dc90": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "1b16421167ab48b8a5917d72d656c7bf": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "f7e04af9a15847d5a948a894f610cc49": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "4327e516e08a4c73b72253077d7e357b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "479f97a939d540d7b027414a51d431fb": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "5df46535a7cf42989003692bf06d4cd8": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "bc0b7e8d78ed4f5689444ea0a98ae561": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_77271c647e124d35bbd2b0b08ed40c65", + "IPY_MODEL_f12f0c28c26c4663932c0aeb0ea209e2", + "IPY_MODEL_448f78a96ab54fc4a22575c54617ca0c" + ], + "layout": "IPY_MODEL_2b186131460f4396b14ba5bc7e917533" + } + }, + "77271c647e124d35bbd2b0b08ed40c65": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_41b54f704a7a4111beb0969226aa383d", + "placeholder": "​", + "style": "IPY_MODEL_b196c652848a4c7e9c7569f881c89b1f", + "value": "Applying [EmbeddingExtractor, ThemesExtractor, NERExtractor]: 100%" + } + }, + "f12f0c28c26c4663932c0aeb0ea209e2": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_797447801d634b5a8e08c5318b1353ed", + "max": 573, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_506eb650f3ab4d3fbfc0c0235e53bc7d", + "value": 573 + } + }, + "448f78a96ab54fc4a22575c54617ca0c": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_bffa759cd8674d889e9c2f4ecf12d3e1", + "placeholder": "​", + "style": "IPY_MODEL_315bf7504aa44c958b809900d535d25b", + "value": " 572/573 [00:54<00:00, 10.35it/s]" + } + }, + "2b186131460f4396b14ba5bc7e917533": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": "hidden", + "width": null + } + }, + "41b54f704a7a4111beb0969226aa383d": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "b196c652848a4c7e9c7569f881c89b1f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "797447801d634b5a8e08c5318b1353ed": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "506eb650f3ab4d3fbfc0c0235e53bc7d": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "bffa759cd8674d889e9c2f4ecf12d3e1": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "315bf7504aa44c958b809900d535d25b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "09db23941a9c4904b21c714da581a43a": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_45ca2e30c7a249f891398109562d411c", + "IPY_MODEL_3e507be792e14294a65311adf32f3acd", + "IPY_MODEL_7876a02ea0324dd88219f6dd437f33ad" + ], + "layout": "IPY_MODEL_b304e464495a4694a138305c99b3c1bb" + } + }, + "45ca2e30c7a249f891398109562d411c": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_2d1bdefd99b04c9286a67a32beeb18be", + "placeholder": "​", + "style": "IPY_MODEL_c4ab6cd1de4245dcb646a23ed6e866f7", + "value": "Applying [CosineSimilarityBuilder, OverlapScoreBuilder]:  50%" + } + }, + "3e507be792e14294a65311adf32f3acd": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_2c79cd256dd6412e823486d903c7c29f", + "max": 2, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_3eaed3e9110b4e7791916cacfc3d9e98", + "value": 2 + } + }, + "7876a02ea0324dd88219f6dd437f33ad": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_fffe61a141cd4734ad3672f897377ba0", + "placeholder": "​", + "style": "IPY_MODEL_7183d9c0f4fe46d9a60da91d6251eec7", + "value": " 1/2 [00:00<00:00,  1.02it/s]" + } + }, + "b304e464495a4694a138305c99b3c1bb": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": "hidden", + "width": null + } + }, + "2d1bdefd99b04c9286a67a32beeb18be": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "c4ab6cd1de4245dcb646a23ed6e866f7": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "2c79cd256dd6412e823486d903c7c29f": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "3eaed3e9110b4e7791916cacfc3d9e98": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "fffe61a141cd4734ad3672f897377ba0": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7183d9c0f4fe46d9a60da91d6251eec7": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "79098b28847d454a82b0f059acd9b1fd": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_785d5293fe434795aeb39c1543437713", + "IPY_MODEL_89ae40bc94a44a2da9d087b8cd39fece", + "IPY_MODEL_6240b2910907434aaca164448fec0180" + ], + "layout": "IPY_MODEL_6a3001ea45cc4611b9ada6ae91285d6f" + } + }, + "785d5293fe434795aeb39c1543437713": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_8b0d39314515439a98836ac0c1184c06", + "placeholder": "​", + "style": "IPY_MODEL_52b39c401c5a4921ad7bf5f114903459", + "value": "Generating personas: 100%" + } + }, + "89ae40bc94a44a2da9d087b8cd39fece": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_bdad31eb9e6b4590b3f92a2e200fccb5", + "max": 3, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_72d74876ce7d4a1692ae5ae4d49544f0", + "value": 3 + } + }, + "6240b2910907434aaca164448fec0180": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_0a14d94b59114e40ba6b592cea136702", + "placeholder": "​", + "style": "IPY_MODEL_4c47217a866d4da9a33f0f945af5e3cc", + "value": " 3/3 [00:00<00:00,  3.50it/s]" + } + }, + "6a3001ea45cc4611b9ada6ae91285d6f": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "8b0d39314515439a98836ac0c1184c06": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "52b39c401c5a4921ad7bf5f114903459": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "bdad31eb9e6b4590b3f92a2e200fccb5": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "72d74876ce7d4a1692ae5ae4d49544f0": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "0a14d94b59114e40ba6b592cea136702": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "4c47217a866d4da9a33f0f945af5e3cc": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "9a941919d4ad4670acee80d33543cf3b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_b3dd78bc59194ce5971daa413a11ca7e", + "IPY_MODEL_6ca416d7b6714ed9a026266db2d19e6d", + "IPY_MODEL_c8119aa65b6640dc9c68e04ae66e6357" + ], + "layout": "IPY_MODEL_ee7f9374cd0b402ca1405dcd77d8a436" + } + }, + "b3dd78bc59194ce5971daa413a11ca7e": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_cfb3de7781fa4b69bada2708cbe6b658", + "placeholder": "​", + "style": "IPY_MODEL_33a6c456af8144feb34d20f63499dcca", + "value": "Generating Scenarios: 100%" + } + }, + "6ca416d7b6714ed9a026266db2d19e6d": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_8c982789718348048264095afb8d409c", + "max": 3, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_018816f934b64d2c8ee4329455f6dabd", + "value": 3 + } + }, + "c8119aa65b6640dc9c68e04ae66e6357": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_fd55665525c54123bf52156d132c5137", + "placeholder": "​", + "style": "IPY_MODEL_54db49340c3e4c3e93d04a282c11ac5b", + "value": " 3/3 [00:05<00:00,  2.90s/it]" + } + }, + "ee7f9374cd0b402ca1405dcd77d8a436": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "cfb3de7781fa4b69bada2708cbe6b658": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "33a6c456af8144feb34d20f63499dcca": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "8c982789718348048264095afb8d409c": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "018816f934b64d2c8ee4329455f6dabd": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "fd55665525c54123bf52156d132c5137": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "54db49340c3e4c3e93d04a282c11ac5b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "58d03b48d18c4865996d21d1cd03c1ae": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_fa9f67c70db14b738093c3afce6ac95e", + "IPY_MODEL_19d5df5770114e03bff203815e01eee5", + "IPY_MODEL_d29d0ae1b38c49d69b61cac820ea4083" + ], + "layout": "IPY_MODEL_ad7fd43b8f554a898def1886a2247351" + } + }, + "fa9f67c70db14b738093c3afce6ac95e": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_c8e8c705b0c743cebb662347c85a7834", + "placeholder": "​", + "style": "IPY_MODEL_44e02a685c944750b7947fc040ea5bd5", + "value": "Generating Samples: 100%" + } + }, + "19d5df5770114e03bff203815e01eee5": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_05af8375ea734765b52e3c6cc36bf014", + "max": 12, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_970f40cf6f4541a8a357a30198759241", + "value": 12 + } + }, + "d29d0ae1b38c49d69b61cac820ea4083": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_4b8cf390496a46b78002530bbfb74eac", + "placeholder": "​", + "style": "IPY_MODEL_01234191c7a945c58515a207ebc42dde", + "value": " 12/12 [00:09<00:00,  1.49s/it]" + } + }, + "ad7fd43b8f554a898def1886a2247351": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "c8e8c705b0c743cebb662347c85a7834": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "44e02a685c944750b7947fc040ea5bd5": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "05af8375ea734765b52e3c6cc36bf014": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "970f40cf6f4541a8a357a30198759241": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "4b8cf390496a46b78002530bbfb74eac": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "01234191c7a945c58515a207ebc42dde": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "3e16df886a974a58a20b18537161d257": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_68b38f1c25a54b978cd20b32c3d02b89", + "IPY_MODEL_33726e4a88e7431b84b26f51c19c6384", + "IPY_MODEL_15f04a999af24f4cb638e87f7839ee24" + ], + "layout": "IPY_MODEL_4e58212505424fb49592181d3ed49f3c" + } + }, + "68b38f1c25a54b978cd20b32c3d02b89": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_d932d142ac64496e9524be5c5b07810a", + "placeholder": "​", + "style": "IPY_MODEL_3f62770fd81247ff9ff69c65324579f1", + "value": "Evaluating: 100%" + } + }, + "33726e4a88e7431b84b26f51c19c6384": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_a48d9be896a94f19899b929655c65771", + "max": 72, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_e1109a565b1644c0997d35d2234c5f5e", + "value": 72 + } + }, + "15f04a999af24f4cb638e87f7839ee24": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_b5bbac0cdd7742d69e45d349879dc96e", + "placeholder": "​", + "style": "IPY_MODEL_305cd0071eee46bc9bf16b4d07dabcab", + "value": " 72/72 [07:47<00:00, 31.70s/it]" + } + }, + "4e58212505424fb49592181d3ed49f3c": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "d932d142ac64496e9524be5c5b07810a": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "3f62770fd81247ff9ff69c65324579f1": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "a48d9be896a94f19899b929655c65771": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "e1109a565b1644c0997d35d2234c5f5e": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "b5bbac0cdd7742d69e45d349879dc96e": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "305cd0071eee46bc9bf16b4d07dabcab": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "8cffb4e48fff4c31adb0418b59c9414f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_2d457df813bd4c56a0520b4d3b7f5c62", + "IPY_MODEL_02c5d66fd4634d25aa5a675ae96f0d3d", + "IPY_MODEL_9074775dc05d40d395c7382c46dc0a90" + ], + "layout": "IPY_MODEL_0f4de30cc0fa4c7f8c2a81d60c84442e" + } + }, + "2d457df813bd4c56a0520b4d3b7f5c62": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_7528a1061fe9417999e7c27090a449fb", + "placeholder": "​", + "style": "IPY_MODEL_2c91cac53c8145f7aedb343466462f8d", + "value": "" + } + }, + "02c5d66fd4634d25aa5a675ae96f0d3d": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_363e420f11114d65bec0d52591c9da4b", + "max": 1, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_7d71ebfef41441d6b725b1a673124a40", + "value": 1 + } + }, + "9074775dc05d40d395c7382c46dc0a90": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_7005ba609e9644d19e64dd04efed2656", + "placeholder": "​", + "style": "IPY_MODEL_b36ea90a25bf4667a77b2889b1e2e36b", + "value": " 12/? [01:36<00:00,  8.64s/it]" + } + }, + "0f4de30cc0fa4c7f8c2a81d60c84442e": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7528a1061fe9417999e7c27090a449fb": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "2c91cac53c8145f7aedb343466462f8d": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "363e420f11114d65bec0d52591c9da4b": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": "20px" + } + }, + "7d71ebfef41441d6b725b1a673124a40": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "7005ba609e9644d19e64dd04efed2656": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "b36ea90a25bf4667a77b2889b1e2e36b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "c612f196069d46b89d8f9abde9f32fd5": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_2f189591fd0446fca62e1ae0b3c68671", + "IPY_MODEL_68c0cd627a9746e090585deb5e10850c", + "IPY_MODEL_b3a4ec7b469844d2b35f613d924d824b" + ], + "layout": "IPY_MODEL_e5b2b4e1f7ff464ebd23fbfb2bb39c87" + } + }, + "2f189591fd0446fca62e1ae0b3c68671": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_e4c3c159bc5945988866d84e1fef5446", + "placeholder": "​", + "style": "IPY_MODEL_28d34e0483824662b8e82f58834022c6", + "value": "" + } + }, + "68c0cd627a9746e090585deb5e10850c": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_3b1850b6a67a475cb63a6816592aa840", + "max": 1, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_a3286232e80e4fb79d0c3e73a223ca79", + "value": 1 + } + }, + "b3a4ec7b469844d2b35f613d924d824b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "model_module_version": "1.5.0", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_18a49867fe05498da609dbf6eb23a191", + "placeholder": "​", + "style": "IPY_MODEL_4aa3510b80f84c279ffee3db03a81b3e", + "value": " 12/? [02:04<00:00,  7.47s/it]" + } + }, + "e5b2b4e1f7ff464ebd23fbfb2bb39c87": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "e4c3c159bc5945988866d84e1fef5446": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "28d34e0483824662b8e82f58834022c6": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "3b1850b6a67a475cb63a6816592aa840": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": "20px" + } + }, + "a3286232e80e4fb79d0c3e73a223ca79": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "18a49867fe05498da609dbf6eb23a191": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "model_module_version": "1.2.0", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "4aa3510b80f84c279ffee3db03a81b3e": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "model_module_version": "1.5.0", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + } + } + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file