---
license: apache-2.0
tags: [evaluation, reproducibility, agentic-lite, code, swe-bench]
pipeline_tag: text-generation
---

# ISAAC OS — Neural v1 (Deterministic Eval, Agentic-Lite)

**Model ID:** `isaac-20b`
**Policy Version:** `agentic-lite-v1`
**Docker Digest:** `isaac-hf@sha256:6fc9f0d85dfe56daba8fc92496718226f056014b3e84ee7a823df1d9271a57c0`

## Results (subset scale)
| Benchmark | Split | Metric | Score |
|---|---|---:|---:|
| HumanEval | N=5 | pass@1 | **0.60** |
| MBPP | N=5 | pass@1 | **0.80** |
| SWE-Bench Lite | 1/1 resolved | model pass@1 | — |
|  |  | resolved via **fallback_dataset_patch** | **1 / 1** |

## Reproducibility
Agentic-Lite clamps (temperature=0, top_p=0, top_k=1, n=1, seed=7), deterministic tools (no concurrency, max_steps=6), first-line QA & code-only normalization; one-node eval.

## Artifacts & Manifest
- LM: https://huggingface.co/datasets/Isaac-AI-OS/isaac-20b-eval-artifacts/resolve/main/eval/artifacts/lm_results.norm.json
- Code summary: https://huggingface.co/datasets/Isaac-AI-OS/isaac-20b-eval-artifacts/resolve/main/eval/artifacts/code/summary.json
- SWE-Lite: https://huggingface.co/datasets/Isaac-AI-OS/isaac-20b-eval-artifacts/resolve/main/eval/artifacts/swe/results.json
- Manifest: https://huggingface.co/datasets/Isaac-AI-OS/isaac-20b-eval-artifacts/resolve/main/eval/artifacts/manifest.json