Micro-Merlin-Experimental

This is a fine-tune of LiquidAI/LFM2.5-1.2B-Thinking on GPT-5.2 reasoning traces.

The model was trained with LoRA on the TeichAI/gpt-5.2-high-reasoning-250x dataset, a collection of high-reasoning-depth traces distilled from GPT-5.2, focused on production-grade DevOps, backend, and infrastructure engineering tasks. The goal is to transfer GPT-5.2's structured <think> reasoning style onto a compact 1.2B model that runs comfortably on consumer hardware.

Model & Training Details

Field	Value
Base model	LiquidAI/LFM2.5-1.2B-Thinking
Parameters	1.2B
Method	LoRA (16-bit, rank-stabilized)
Dataset	TeichAI/gpt-5.2-high-reasoning-250x
Training examples	249
Epochs	1
Total steps	~63
Final training loss	2.121
LoRA rank (r)	64
LoRA alpha	64
LoRA dropout	0
rsLoRA	Enabled
Target modules	q_proj, k_proj, v_proj, out_proj, in_proj, w1, w2, w3
Max sequence length	20,480
Batch size (effective)	4 (1 × 4 grad. accum.)
Learning rate	2e-4
LR scheduler	Cosine
Warmup steps	3
Optimizer	adamw_8bit
Weight decay	0.01
Precision	FP16
Loss masking	Responses only (<think> + answer)
Hardware	1× NVIDIA Tesla T4 (16 GB)
Framework	Unsloth + TRL SFTTrainer
Training runtime	608 s (10 min)
Chat template	ChatML (`<

Usage

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="OrionLLM/Micro-Merlin-Experimental",
    max_seq_length=20480,
    load_in_4bit=False,
)
FastLanguageModel.for_inference(model)

messages = [{"role": "user", "content": "Design a rate limiter for a REST API."}]
inputs = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

out = model.generate(**inputs, max_new_tokens=1024, temperature=0.5, repetition_penalty=1.15)
print(tokenizer.decode(out[0], skip_special_tokens=True))

---

Merlin Research • 2026

Developed by DedeProGames