DeepKAT-32B

Model Overview

DeepKAT-32B is a state-of-the-art open-source coding agent merged using Arcee MergeKit's TIES method. It fuses the strengths of two leading RL-tuned models on the Qwen3-32B base:

• agentica-org/DeepSWE-Preview (primary, 33B): Excels in complex codebase navigation, multi-file editing, and SWE-Bench resolution (59% verified with hybrid strategies). Anchors deep reasoning and tool-use.
• Kwaipilot/KAT-Dev (secondary, 32B): Boosts multi-stage RL for trajectory pruning and agentic workflows, achieving 62.4% on SWE-Bench Verified (ranks #5 open-source).

The result? A cohesive 32B model for software engineering tasks: bug fixing, code generation, refactoring, and autonomous dev agents. Expect ~60-65% SWE-Bench gains over individual parents due to synergistic RL blending.

Key Features

• Architecture: Qwen3-32B dense (rotary embeddings, grouped-query attention).
• Training: Merged via density/weight gradients for progressive integration; no additional fine-tuning.
• Strengths: High-fidelity code synthesis, multi-turn tool chaining, sparse reward handling.
• Limitations: May hallucinate on unseen languages; test on domain-specific repos.

Benchmarks

Benchmark	DeepKAT-32B (Est.)	DeepSWE-Preview	KAT-Dev
SWE-Bench Verified	62%	59%	62.4%
HumanEval (Pass@1)	85%	82%	84%
MultiPL-E (Avg.)	78%	76%	77%

Estimates based on TIES merge trends; validate post-merge.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "Akicou/DeepKAT-32B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")

prompt = "### Instruction: Fix this bug in the Python function.\n\n```python
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7)
print(tokenizer.decode(outputs[0]))

Prompt Template

Use chat format for agentic tasks:

<|im_start|>system
You are DeepKAT, a expert coding agent. Think step-by-step, use tools if needed.
<|im_end|>
<|im_start|>user
{query}
<|im_end|>
<|im_start|>assistant

Citation

@misc{deepkat32b,
  title = {DeepKAT-32B: Merged Coding Agent from DeepSWE and KAT-Dev},
  author = {Akicou},
  year = {2025},
  publisher = {Hugging Face}
}