The MHA2MLA model published in the paper "Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-Based LLMs"
AI & ML interests
LLM
Recent Activity
View all activity
Organization Card
Joint OpenMOSS group from Fudan NLP, SII and MoSi Inc.
Collections
3
models
96
fnlp/qwen2.5-0_5B-rope8-d_kv_32-refactor
Text Generation
•
Updated
fnlp/qwen2-0_5B-rope8-d_kv_16-refactor
Text Generation
•
Updated
fnlp/SmolLM-135M-MLA-d_kv_8-refactor
Text Generation
•
Updated
•
2
fnlp/MOSS-TTSD-v0
Text-to-Speech
•
Updated
•
97
•
20
fnlp/XY_Tokenizer_TTSD_V0
Updated
•
2
fnlp/qwen1_5-0_5B-d_kv_32-refactor
Text Generation
•
Updated
•
4
fnlp/qwen1_5-0_5B-d_kv_16-refactor
Text Generation
•
Updated
•
6
fnlp/qwen1_5-0_5B-d_kv_8-refactor
Text Generation
•
Updated
•
4
fnlp/qwen3-0_6B-uniform_r_16-d_kv_64-refactor
Text Generation
•
Updated
•
4
fnlp/qwen3-0_6B-uniform_r_16-d_kv_32-refactor
Text Generation
•
Updated
•
6
datasets
18
fnlp/MHA2MLA-corpus-llama3
Updated
fnlp/MHA2MLA-corpus-qwen1.5
Updated
•
66
fnlp/MHA2MLA-corpus-smollm
Updated
•
766
fnlp/MHA2MLA-corpus-qwen1_5
Updated
•
6
fnlp/MHA2MLA-corpus-qwen2
Updated
•
27
fnlp/MHA2MLA-corpus-mistral-v0_1
Updated
•
18
fnlp/MHA2MLA-corpus-smollm_v1
Updated
•
34
fnlp/MHA2MLA-corpus-llama2
Updated
•
50
fnlp/Ultra-Innerthought
Viewer
•
Updated
•
2.09M
•
46
•
2
fnlp/case2code-data
Viewer
•
Updated
•
887k
•
67
•
2