upload checkpoint

.gitattributes

@@ -0,0 +1,2 @@
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,230 @@
+---
+language:
+- en
+library_name: transformers
+tags:
+- glm
+- MOE
+- pruning
+- compression
+license: mit
+name: cerebras/GLM-4.5-Air-REAP-82B-A12B
+description: >
+  This model was obtained by uniformly pruning 25% of experts in GLM-4.5-Air using the REAP method.
+readme: >
+  https://huggingface.co/cerebras/GLM-4.5-Air-REAP-82B-A12B/main/README.md
+license_link: https://huggingface.co/zai-org/GLM-4.5-Air/blob/main/LICENSE
+pipeline_tag: text-generation
+base_model:
+- zai-org/GLM-4.5-Air
+---
+
+<p align="center">
+  <em>𓌳 <strong>REAP</strong>𓌳  the Experts: Why Pruning Prevails for One-Shot MoE Compression</em><br>
+  <img src="https://i.imgur.com/rmzG3gg.png" alt="REAP" width="75%">
+</p>
+
+# GLM-4.5-Air-REAP-82B-A12B
+
+## ✨ Highlights
+
+Introducing **GLM-4.5-Air-REAP-82B-A12B**, a **memory-efficient compressed variant** of GLM-4.5-Air that maintains near-identical performance while being **25% lighter**.
+
+This model was created using **REAP (Router-weighted Expert Activation Pruning)**, a novel expert pruning method that selectively removes redundant experts while preserving the router's independent control over remaining experts. Key features include:
+
+- **Near-Lossless Performance**: Maintains almost identical accuracy on code generation, agentic coding, and function calling tasks compared to the full 480B model
+- **25% Memory Reduction**: Compressed from 480B to 363B parameters, significantly lowering deployment costs and memory requirements
+- **Preserved Capabilities**: Retains all core functionalities including code generation, agentic workflows, repository-scale understanding, and function calling
+- **Drop-in Compatibility**: Works with vanilla vLLM - no source modifications or custom patches required
+- **Optimized for Real-World Use**: Particularly effective for resource-constrained environments, local deployments, and academic research
+---
+## 📋 Model Overview
+
+**GLM-4.5-Air-REAP-82B-A12B** has the following specifications:
+
+- **Base Model**: GLM-4.5-Air
+- **Compression Method**: REAP (Router-weighted Expert Activation Pruning)
+- **Compression Ratio**: 25% expert pruning
+- **Type**: Sparse Mixture-of-Experts (SMoE) Causal Language Model
+- **Number of Parameters**: 82B total, 12B activated per token
+- **Number of Layers**: 46
+- **Number of Attention Heads (GQA)**: 96 for Q and 8 for KV
+- **Number of Experts**: 96 (uniformly pruned from 128)
+- **Number of Activated Experts**: 8 per token
+- **Context Length**: 131,072 tokens
+- **License**: MIT
+
+---
+
+## 📊 Evaluations
+
+<table>
+  <thead>
+    <tr>
+      <th align="left">Benchmark</th>
+      <th align="center">GLM-4.5-Air</th>
+      <th align="center"><a href="https://huggingface.co/cerebras/GLM-4.5-Air-REAP-82B-A12B">GLM-4.5-Air-REAP-82B-A12B</a></th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td><strong>Compression</strong></td>
+      <td align="center">—</td>
+      <td align="center">25%</td>
+    </tr>
+    <tr>
+      <td colspan="3" align="center"><strong>Coding</strong></td>
+    </tr>
+    <tr>
+      <td><strong>HumanEval</strong></td>
+      <td align="center">92.7</td>
+      <td align="center">89.6</td>
+    </tr>
+    <tr>
+      <td><strong>HumanEval+</strong></td>
+      <td align="center">86.0</td>
+      <td align="center">84.8</td>
+    </tr>
+    <tr>
+      <td><strong>MBPP</strong></td>
+      <td align="center">86.2</td>
+      <td align="center">84.4</td>
+    </tr>
+    <tr>
+      <td><strong>MBPP+</strong></td>
+      <td align="center">69.8</td>
+      <td align="center">69.6</td>
+    </tr>
+    <tr>
+      <td colspan="3" align="center"><strong>Reasoning</strong></td>
+    </tr>
+    <tr>
+      <td><strong>LiveCodeBench</strong> (25.01 - 25.05, thinking)</td>
+      <td align="center">39.6</td>
+      <td align="center">42.9</td>
+    </tr>
+    <tr>
+      <td><strong>GPQA diamond</strong> (thinking)</td>
+      <td align="center">65.2</td>
+      <td align="center">65.2</td>
+    </tr>
+    <tr>
+      <td><strong>AIME24</strong> (thinking)</td>
+      <td align="center">83.3</td>
+      <td align="center">80.0</td>
+    </tr>
+    <tr>
+      <td><strong>MATH-500</strong> (thinking)</td>
+      <td align="center">94.8</td>
+      <td align="center">94.8</td>
+    </tr>
+    <tr>
+      <td colspan="3" align="center"><strong>Tool Calling</strong></td>
+    </tr>
+    <tr>
+      <td><strong>BFCL-v3</strong></td>
+      <td align="center">73.4</td>
+      <td align="center">71.8</td>
+    </tr>
+    <tr>
+      <td><strong>BFCL-v3</strong> (thinking)</td>
+      <td align="center">76.8</td>
+      <td align="center">76.3</td>
+    </tr>
+    <tr>
+      <td><strong>𝜏²-bench</strong> (airline)</td>
+      <td align="center">63.3</td>
+      <td align="center">64.0</td>
+    </tr>
+    <tr>
+      <td><strong>𝜏²-bench</strong> (retail)</td>
+      <td align="center">72.8</td>
+      <td align="center">75.1</td>
+    </tr>
+    <tr>
+      <td><strong>𝜏²-bench</strong> (telecom)</td>
+      <td align="center">28.4</td>
+      <td align="center">30.7</td>
+    </tr>
+    <tr>
+      <td><strong>𝜏²-bench</strong> (telecom, thinking)</td>
+      <td align="center">27.2</td>
+      <td align="center">26.9</td>
+    </tr>
+  </tbody>
+</table>
+
+🟩 *This checkpoint maintains almost identical performance while being 25% lighter.*
+
+For more details on the evaluation setup, refer to the [REAP arXiv preprint](https://arxiv.org/abs/2510.13999).
+
+---
+
+## 🚀 Deployment
+
+You can deploy the model directly using the **latest vLLM** (v0.11.0), no source modifications or custom patches required.
+
+```bash
+vllm serve cerebras/GLM-4.5-Air-REAP-82B-A12B \
+    --tensor-parallel-size 8 \
+    --tool-call-parser glm45 \
+    --enable-auto-tool-choice \
+    --enable-expert-parallel
+```
+
+If you encounter insufficient memory when running this model, you might need to set a lower value for `--max-num-seqs` flag (e.g. set to 64).
+
+
+## 🧩 Model Creation
+
+This checkpoint was created by applying the **REAP (Router-weighted Expert Activation Pruning)** method uniformly across all Mixture-of-Experts (MoE) blocks of **GLM-4.5-Air**, with a **25% pruning rate**.
+
+### How REAP Works
+
+REAP selects experts to prune based on a novel **saliency criterion** that considers both:
+- **Router gate values**: How frequently and strongly the router activates each expert
+- **Expert activation norms**: The magnitude of each expert's output contributions
+
+This dual consideration ensures that experts contributing minimally to the layer's output are pruned, while preserving those that play critical roles in the model's computations.
+
+### Key Advantages
+
+- **One-Shot Compression**: No fine-tuning required after pruning - the model is immediately ready for deployment
+- **Preserved Router Control**: Unlike expert merging methods, REAP maintains the router's independent, input-dependent control over remaining experts, avoiding "functional subspace collapse"
+- **Generative Task Superiority**: REAP significantly outperforms expert merging approaches on generative benchmarks (code generation, creative writing, mathematical reasoning) while maintaining competitive performance on discriminative tasks
+
+### Calibration
+
+The model was calibrated using a diverse mixture of domain-specific datasets including:
+- Code generation samples ([evol-codealpaca](https://huggingface.co/datasets/theblackcat102/evol-codealpaca-v1))
+- Function calling examples ([xlam-function-calling](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k))
+- Agentic multi-turn trajectories ([SWE-smith-trajectories](https://huggingface.co/datasets/SWE-bench/SWE-smith-trajectories))
+
+📚 For more details, refer to the following resources:
+
+- [🧾 arXiv Preprint](https://arxiv.org/abs/2510.13999)
+- [🧾 REAP Blog](https://www.cerebras.ai/blog/reap)
+- [💻 REAP Codebase (GitHub)](https://github.com/CerebrasResearch/reap)
+
+---
+
+## ⚖️ License
+
+This model is derived from
+**[`zai-org/GLM-4.5-Air`](https://huggingface.co/zai-org/GLM-4.5-Air)**
+and distributed under the **MIT license**.
+
+---
+
+## 🧾 Citation
+
+If you use this checkpoint, please cite the REAP paper:
+
+```bibtex
+@article{lasby-reap,
+  title={REAP the Experts: Why Pruning Prevails for One-Shot MoE compression},
+  author={Lasby, Mike and Lazarevich, Ivan and Sinnadurai, Nish and Lie, Sean and Ioannou, Yani and Thangarasa, Vithursan},
+  journal={arXiv preprint arXiv:2510.13999},
+  year={2025}
+}
+```

chat_template.jinja

@@ -0,0 +1,103 @@
+[gMASK]<sop>
+{%- if tools -%}
+<|system|>
+# Tools
+
+You may call one or more functions to assist with the user query.
+
+You are provided with function signatures within <tools></tools> XML tags:
+<tools>
+{% for tool in tools %}
+{{ tool | tojson(ensure_ascii=False) }}
+{% endfor %}
+</tools>
+
+For each function call, output the function name and arguments within the following XML format:
+<tool_call>{function-name}
+<arg_key>{arg-key-1}</arg_key>
+<arg_value>{arg-value-1}</arg_value>
+<arg_key>{arg-key-2}</arg_key>
+<arg_value>{arg-value-2}</arg_value>
+...
+</tool_call>{%- endif -%}
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text }}
+            {%- elif item is string -%}
+                {{- item }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{- content }}
+    {%- endif -%}
+{%- endmacro -%}
+{%- set ns = namespace(last_user_index=-1) %}
+{%- for m in messages %}
+    {%- if m.role == 'user' %}
+        {% set ns.last_user_index = loop.index0 -%}
+    {%- endif %}
+{%- endfor %}
+{% for m in messages %}
+{%- if m.role == 'user' -%}<|user|>
+{{ visible_text(m.content) }}
+{{- '/nothink' if (enable_thinking is defined and not enable_thinking and not visible_text(m.content).endswith("/nothink")) else '' -}}
+{%- elif m.role == 'assistant' -%}
+<|assistant|>
+{%- set reasoning_content = '' %}
+{%- set content = visible_text(m.content) %}
+{%- if m.reasoning_content is string %}
+    {%- set reasoning_content = m.reasoning_content %}
+{%- else %}
+    {%- if '</think>' in content %}
+        {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+        {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+    {%- endif %}
+{%- endif %}
+{%- if loop.index0 > ns.last_user_index and reasoning_content -%}
+{{ '\n<think>' + reasoning_content.strip() +  '</think>'}}
+{%- else -%}
+{{ '\n<think></think>' }}
+{%- endif -%}
+{%- if content.strip() -%}
+{{ '\n' + content.strip() }}
+{%- endif -%}
+{% if m.tool_calls %}
+{% for tc in m.tool_calls %}
+{%- if tc.function %}
+    {%- set tc = tc.function %}
+{%- endif %}
+{{ '\n<tool_call>' + tc.name }}
+{% set _args = tc.arguments %}
+{% for k, v in _args.items() %}
+<arg_key>{{ k }}</arg_key>
+<arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>
+{% endfor %}
+</tool_call>{% endfor %}
+{% endif %}
+{%- elif m.role == 'tool' -%}
+{%- if m.content is string -%}
+{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+    {{- '<|observation|>' }}
+{%- endif %}
+{{- '\n<tool_response>\n' }}
+{{- m.content }}
+{{- '\n</tool_response>' }}
+{%- else -%}
+<|observation|>{% for tr in m.content %}
+
+<tool_response>
+{{ tr.output if tr.output is defined else tr }}
+</tool_response>{% endfor -%}
+{% endif -%}
+{%- elif m.role == 'system' -%}
+<|system|>
+{{ visible_text(m.content) }}
+{%- endif -%}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    <|assistant|>{{- '\n<think></think>' if (enable_thinking is defined and not enable_thinking) else '' -}}
+{%- endif -%}

config.json

@@ -0,0 +1,43 @@
+{
+  "architectures": [
+    "Glm4MoeForCausalLM"
+  ],
+  "attention_bias": true,
+  "attention_dropout": 0.0,
+  "eos_token_id": [
+    151329,
+    151336,
+    151338
+  ],
+  "first_k_dense_replace": 1,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 10944,
+  "max_position_embeddings": 131072,
+  "model_type": "glm4_moe",
+  "moe_intermediate_size": 1408,
+  "n_group": 1,
+  "n_routed_experts": 96,
+  "n_shared_experts": 1,
+  "norm_topk_prob": true,
+  "num_attention_heads": 96,
+  "num_experts_per_tok": 8,
+  "num_hidden_layers": 46,
+  "num_key_value_heads": 8,
+  "num_nextn_predict_layers": 1,
+  "pad_token_id": 151329,
+  "partial_rotary_factor": 0.5,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 1000000,
+  "routed_scaling_factor": 1.0,
+  "tie_word_embeddings": false,
+  "topk_group": 1,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.55.0",
+  "use_cache": true,
+  "use_qk_norm": false,
+  "vocab_size": 151552
+}

generation_config.json

@@ -0,0 +1,10 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": [
+    151329,
+    151336,
+    151338
+  ],
+  "pad_token_id": 151329,
+  "transformers_version": "4.55.0"
+}

model-00001-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8fe3f1c0598fad3f2714261966c2ab2086d8aabb5f3fbb0d0488716a21a910bd
+size 4991854888

model-00002-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac653d85d27942f62fe9d1fadab084f1c1d89fc8264899a2934906acaa711a42
+size 4993985472

model-00003-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:590fcda1ddd3e837dda23c297d86a818c633a30ea9ef77ceafa66f9bd8fdbd74
+size 4994203320

model-00004-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:930bb34f1e281687d61400897a4d44902f3b9bc6ad158a4773d52455a257b1a2
+size 4993985496

model-00005-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e73c6d3465ee590200a3b7c901673e16781aff06d3a7e77d0721c073da7e8e18
+size 4994203304

model-00006-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a6447f790922c5bd36c0fd8dd966d08f8beac7f07c40cdd7cbf1e378e6470a5
+size 4994203320

model-00007-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6eda359d435fd386d1254f9cf810ee8ac353693f356f8e872c715048b72fef9
+size 4993985584

model-00008-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a56f2ae07824eb79b9601761aa5b5060af80b62de73c4e7be7786b10354fe71a
+size 4994203744

model-00009-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad6e7554f725c5bdaad0ca03d70f2cfb0de0f0e40f1b0d545b90931534c01832
+size 4993985912

model-00010-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6bb2c84e5e55f4e53a5781f4a836b0416d58c93bae4e6114dacad885fa549e9
+size 4994203720

model-00011-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f6856449d0f8d13d6181d560d675e12333acd8a386a052b3dbaacadd3dcfc16
+size 4994203744

model-00012-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e9daf7e13099f92a820c9889915c123374ed5db03ded1bf893b4bda30503b11
+size 4993985888

model-00013-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6083a88bab1df6dbaa63cdfd4b75e57945b9188082c64e69132f4f907c01a7a4
+size 4994203744

model-00014-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:230bb3e0bd5bd7c9590a19143eb33ff013ea04c722c0fe99e67ecb596d67fca8
+size 4993985920

model-00015-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:07b51ec75c635a98a5daed9c1ce6ed6fe941784f31c128a5cf1a5a84087c9502
+size 4994203712

model-00016-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:23cdaeef0f901756ba512282b03f86bee0de81157ef5a6a0011b3874bebdd738
+size 4994203744

model-00017-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61054c50909a061a3b56ec3c5515f9fa4ee8ea975feeae20111e8b481d23c2ec
+size 4993985888

model-00018-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eff4456ba518b00c1aa909d9c09fe0f59246221bda1ec925bf89c1aff9d54536
+size 4994203744

model-00019-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f5c1fc299043627bccd23f98f0e19a6370486ab0b35cc370297bfb2283a516c
+size 4916391432

model-00020-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a680dfa304f67cd9788fdb6bdadc8bb54998621478ca0b13bc7d517a52d8a4f
+size 4991056976

model-00021-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bafc8d07ba21d0efd29380214510504c196583b7b5f1b98488ab9f553b8a5128
+size 4994203744

model-00022-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0ce7a436b3ba1a9383a2aa1ef462a4ba555b7595d4c719174b1fb5e39685324
+size 4993985888

model-00023-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1eb35c2bbda449788c6c90a5f35edbf91db56d28bb704aad9baccfdf20430916
+size 4994203744

model-00024-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95b10327e1d98ed5606494fbbfcd4ad73caf61e7e2595082876e9c5f715a5b33
+size 4937335096

model-00025-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce5882b54dd4855a9f70c8def6593db8fa48943f6c0e44a46614aa044673a29d
+size 4993182232

model-00026-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8661826bbef9500f5db17a5d84b7c3d6d3b6259f805832cd596416f8a78bdf93
+size 4994203744

model-00027-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c014c3e53c86292001cdf0036d34de7790fd9265c859ec3df08333e3e228a5a8
+size 4993985888

model-00028-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45dcd96af4adbdab075804bafacb8edfcb286772ae873962aa4bb4694705f047
+size 4994203744

model-00029-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fca33c12405aac10efc22393902dde819e19307cb73fc6b926bb7cbde220036a
+size 4994203744

model-00030-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cfc2bca4ad731abba1abd28aca904defb3f02a740fe4615be884eb768806a566
+size 4993985880

model-00031-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04c119805ef672615f9f3143a6a641567f23d8fdb5e6b82e3f0f833fb776da6e
+size 4994203744

model-00032-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d576a93bc5dc84b9b56fccf7488eca6da9756b217b81986566961744543e0f6
+size 4993985888

model-00033-of-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7758776512d90f231992766c306fcb206dcd85f1d691004afe330b0bd7554e7f
+size 4195148160

special_tokens_map.json

@@ -0,0 +1,40 @@
+{
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "[MASK]",
+    "[gMASK]",
+    "[sMASK]",
+    "<sop>",
+    "<eop>",
+    "<|system|>",
+    "<|user|>",
+    "<|assistant|>",
+    "<|observation|>",
+    "<|begin_of_image|>",
+    "<|end_of_image|>",
+    "<|begin_of_video|>",
+    "<|end_of_video|>",
+    "<|begin_of_audio|>",
+    "<|end_of_audio|>",
+    "<|begin_of_transcription|>",
+    "<|end_of_transcription|>",
+    "<|code_prefix|>",
+    "<|code_middle|>",
+    "<|code_suffix|>",
+    "/nothink"
+  ],
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bda8e2146c3bb7b7e0fc96dcc4f0aeff041c6c27952e3ace0665663ebff346ba
+size 19970700

tokenizer_config.json

@@ -0,0 +1,325 @@
+{
+  "added_tokens_decoder": {
+    "151329": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151330": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151331": {
+      "content": "[gMASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151332": {
+      "content": "[sMASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151333": {
+      "content": "<sop>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151334": {
+      "content": "<eop>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151335": {
+      "content": "<|system|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151336": {
+      "content": "<|user|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151337": {
+      "content": "<|assistant|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151338": {
+      "content": "<|observation|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151339": {
+      "content": "<|begin_of_image|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151340": {
+      "content": "<|end_of_image|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151341": {
+      "content": "<|begin_of_video|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151342": {
+      "content": "<|end_of_video|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151343": {
+      "content": "<|begin_of_audio|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151344": {
+      "content": "<|end_of_audio|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151345": {
+      "content": "<|begin_of_transcription|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151346": {
+      "content": "<|end_of_transcription|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151347": {
+      "content": "<|code_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151348": {
+      "content": "<|code_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151349": {
+      "content": "<|code_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151350": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151351": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151352": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151353": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151354": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151355": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151356": {
+      "content": "<arg_key>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151357": {
+      "content": "</arg_key>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151358": {
+      "content": "<arg_value>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151359": {
+      "content": "</arg_value>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151360": {
+      "content": "/nothink",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151361": {
+      "content": "<|begin_of_box|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151362": {
+      "content": "<|end_of_box|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151363": {
+      "content": "<|image|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151364": {
+      "content": "<|video|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "[MASK]",
+    "[gMASK]",
+    "[sMASK]",
+    "<sop>",
+    "<eop>",
+    "<|system|>",
+    "<|user|>",
+    "<|assistant|>",
+    "<|observation|>",
+    "<|begin_of_image|>",
+    "<|end_of_image|>",
+    "<|begin_of_video|>",
+    "<|end_of_video|>",
+    "<|begin_of_audio|>",
+    "<|end_of_audio|>",
+    "<|begin_of_transcription|>",
+    "<|end_of_transcription|>",
+    "<|code_prefix|>",
+    "<|code_middle|>",
+    "<|code_suffix|>",
+    "/nothink"
+  ],
+  "clean_up_tokenization_spaces": false,
+  "do_lower_case": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": {},
+  "model_max_length": 128000,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "left",
+  "remove_space": false,
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}