Operational_Instructions/DM_Sandbox.md

DataMorgana Sandbox

1. Sign in to the AI71 website

2. Click on "DataMorgana" at the left rail, and then click "New test" to start generating Q&A pairs

3. Question/User Categorizations
   3.1 Each Categorization includes a Name
   3.2 You may use the default Categorizations or add new ones
   3.3 You can choose multiple Categorization (e.g., choosing "Factuality" and "Formulation" Question Categorizations)
   3.4 Categorizations includes multiple Categories (e.g., "User-Expertise" includes two categories, "Expert" and "Novice")
   3.4.1 Expand a Categorization by clicking the caret icon on its right-hand side to reveal its Categories

4. Question Categories
   4.1 Each Category includes a Name, a Description, and a "Multi-doc" checkbox
   4.2 You may use the default Categories or add new ones
   4.3 Each Categorization allows only one selected Category (e.g., "Factoid" or "Open-ended" under "Factuality" Categorization)
   4.3.1 In case at least one selected Question Categorization includes a selected Question Category which is checked for "Multi-doc", the generated Q&A will consider exactly two docs (i.e., "Multi-doc" request). Otherwise, it considers only one doc (i.e., "Single-doc" request)

5. User Categories
   5.1 Each Category includes a Name and a Description
   5.2 You may use the default Categories or add new ones
   5.3 Each Categorization allows only one selected Category (e.g., "Expert" or "Novice" under "User-Expertise" Categorization)

6. Once you set the Question/User Categorizations and their Categories:
   6.1 For "Multi-doc" requests you may enter two FineWeb document IDs (e.g., "<urn:uuid:d69cbebc-133a-4ebe-9378-68235ec9f091>") or none, in the respective box
   6.2 For "Single-doc" requests you may enter one FineWeb document ID or none, in the respective box
   6.1 If you leave the document ID field(s) empty, a random document(s) will be assigned automatically

7. Click the "Generate" black button on the top right to generate a DataMorgana synthetic Q&A pair
   7.1 The results (i.e., question, answer, doc(s), doc ID(s), and Question/User Categorizations information) will appear momentarily in a new page
   7.2 You may copy the results to the clipboard by clicking the "Copy to Clipboard" button, or return to the setting page by clicking the "Clear" button
   7.3 You may find Q&A pair examples here

8. Your remaining Q&A pairs budget is displayed by a counter at the top center

9. You may use JSON mode for entering and editing Question/User Categorizations/Categories, by sliding the knob at the top to "JSON Editor"
   9.1 Follow the JSON Schema on the bottom right for correct Categorizations/Categories setting
   9.2 In case your resulting JSON is correct, it will be reflected in the "Visual Editor" once the knob at the top is shifted left

10. The default Categorizations/Categories settings may be restored by refreshing the page