Hugging Face's logo

NexVeridian
/
gpt-oss-20b-5bit

Text Generation
MLX
Safetensors
gpt_oss
vllm
conversational
4-bit precision
Model card Files
xet
Community
NexVeridian commited on
Commit
593ef66
·
verified ·
1 Parent(s): a770ad6

Add files using upload-large-folder tool

Browse files
Files changed (7) hide show
  1. README.md +1 -1
  2. config.json +1178 -4
  3. generation_config.json +2 -1
  4. model-00001-of-00003.safetensors +2 -2
  5. model-00002-of-00003.safetensors +2 -2
  6. model-00003-of-00003.safetensors +2 -2
  7. model.safetensors.index.json +251 -323
README.md CHANGED
@@ -12,7 +12,7 @@ base_model: openai/gpt-oss-20b
12
 
13
  This model [NexVeridian/gpt-oss-20b-5bit](https://huggingface.co/NexVeridian/gpt-oss-20b-5bit) was
14
  converted to MLX format from [openai/gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b)
15
- using mlx-lm version **0.26.3**.
16
 
17
  ## Use with mlx
18
 
 
12
 
13
  This model [NexVeridian/gpt-oss-20b-5bit](https://huggingface.co/NexVeridian/gpt-oss-20b-5bit) was
14
  converted to MLX format from [openai/gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b)
15
+ using mlx-lm version **0.27.0**.
16
 
17
  ## Use with mlx
18
 
config.json CHANGED
@@ -48,12 +48,1186 @@
48
  "output_router_logits": false,
49
  "pad_token_id": 199999,
50
  "quantization": {
51
- "group_size": 64,
52
- "bits": 5
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
53
  },
54
  "quantization_config": {
55
- "group_size": 64,
56
- "bits": 5
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57
  },
58
  "rms_norm_eps": 1e-05,
59
  "rope_scaling": {
 
48
  "output_router_logits": false,
49
  "pad_token_id": 199999,
50
  "quantization": {
51
+ "group_size": 32,
52
+ "bits": 4,
53
+ "mode": "mxfp4",
54
+ "model.embed_tokens": {
55
+ "group_size": 64,
56
+ "bits": 5,
57
+ "mode": "affine"
58
+ },
59
+ "model.layers.0.self_attn.q_proj": {
60
+ "group_size": 64,
61
+ "bits": 5,
62
+ "mode": "affine"
63
+ },
64
+ "model.layers.0.self_attn.k_proj": {
65
+ "group_size": 64,
66
+ "bits": 5,
67
+ "mode": "affine"
68
+ },
69
+ "model.layers.0.self_attn.v_proj": {
70
+ "group_size": 64,
71
+ "bits": 5,
72
+ "mode": "affine"
73
+ },
74
+ "model.layers.0.self_attn.o_proj": {
75
+ "group_size": 64,
76
+ "bits": 5,
77
+ "mode": "affine"
78
+ },
79
+ "model.layers.0.mlp.router": {
80
+ "group_size": 64,
81
+ "bits": 8
82
+ },
83
+ "model.layers.1.self_attn.q_proj": {
84
+ "group_size": 64,
85
+ "bits": 5,
86
+ "mode": "affine"
87
+ },
88
+ "model.layers.1.self_attn.k_proj": {
89
+ "group_size": 64,
90
+ "bits": 5,
91
+ "mode": "affine"
92
+ },
93
+ "model.layers.1.self_attn.v_proj": {
94
+ "group_size": 64,
95
+ "bits": 5,
96
+ "mode": "affine"
97
+ },
98
+ "model.layers.1.self_attn.o_proj": {
99
+ "group_size": 64,
100
+ "bits": 5,
101
+ "mode": "affine"
102
+ },
103
+ "model.layers.1.mlp.router": {
104
+ "group_size": 64,
105
+ "bits": 8
106
+ },
107
+ "model.layers.2.self_attn.q_proj": {
108
+ "group_size": 64,
109
+ "bits": 5,
110
+ "mode": "affine"
111
+ },
112
+ "model.layers.2.self_attn.k_proj": {
113
+ "group_size": 64,
114
+ "bits": 5,
115
+ "mode": "affine"
116
+ },
117
+ "model.layers.2.self_attn.v_proj": {
118
+ "group_size": 64,
119
+ "bits": 5,
120
+ "mode": "affine"
121
+ },
122
+ "model.layers.2.self_attn.o_proj": {
123
+ "group_size": 64,
124
+ "bits": 5,
125
+ "mode": "affine"
126
+ },
127
+ "model.layers.2.mlp.router": {
128
+ "group_size": 64,
129
+ "bits": 8
130
+ },
131
+ "model.layers.3.self_attn.q_proj": {
132
+ "group_size": 64,
133
+ "bits": 5,
134
+ "mode": "affine"
135
+ },
136
+ "model.layers.3.self_attn.k_proj": {
137
+ "group_size": 64,
138
+ "bits": 5,
139
+ "mode": "affine"
140
+ },
141
+ "model.layers.3.self_attn.v_proj": {
142
+ "group_size": 64,
143
+ "bits": 5,
144
+ "mode": "affine"
145
+ },
146
+ "model.layers.3.self_attn.o_proj": {
147
+ "group_size": 64,
148
+ "bits": 5,
149
+ "mode": "affine"
150
+ },
151
+ "model.layers.3.mlp.router": {
152
+ "group_size": 64,
153
+ "bits": 8
154
+ },
155
+ "model.layers.4.self_attn.q_proj": {
156
+ "group_size": 64,
157
+ "bits": 5,
158
+ "mode": "affine"
159
+ },
160
+ "model.layers.4.self_attn.k_proj": {
161
+ "group_size": 64,
162
+ "bits": 5,
163
+ "mode": "affine"
164
+ },
165
+ "model.layers.4.self_attn.v_proj": {
166
+ "group_size": 64,
167
+ "bits": 5,
168
+ "mode": "affine"
169
+ },
170
+ "model.layers.4.self_attn.o_proj": {
171
+ "group_size": 64,
172
+ "bits": 5,
173
+ "mode": "affine"
174
+ },
175
+ "model.layers.4.mlp.router": {
176
+ "group_size": 64,
177
+ "bits": 8
178
+ },
179
+ "model.layers.5.self_attn.q_proj": {
180
+ "group_size": 64,
181
+ "bits": 5,
182
+ "mode": "affine"
183
+ },
184
+ "model.layers.5.self_attn.k_proj": {
185
+ "group_size": 64,
186
+ "bits": 5,
187
+ "mode": "affine"
188
+ },
189
+ "model.layers.5.self_attn.v_proj": {
190
+ "group_size": 64,
191
+ "bits": 5,
192
+ "mode": "affine"
193
+ },
194
+ "model.layers.5.self_attn.o_proj": {
195
+ "group_size": 64,
196
+ "bits": 5,
197
+ "mode": "affine"
198
+ },
199
+ "model.layers.5.mlp.router": {
200
+ "group_size": 64,
201
+ "bits": 8
202
+ },
203
+ "model.layers.6.self_attn.q_proj": {
204
+ "group_size": 64,
205
+ "bits": 5,
206
+ "mode": "affine"
207
+ },
208
+ "model.layers.6.self_attn.k_proj": {
209
+ "group_size": 64,
210
+ "bits": 5,
211
+ "mode": "affine"
212
+ },
213
+ "model.layers.6.self_attn.v_proj": {
214
+ "group_size": 64,
215
+ "bits": 5,
216
+ "mode": "affine"
217
+ },
218
+ "model.layers.6.self_attn.o_proj": {
219
+ "group_size": 64,
220
+ "bits": 5,
221
+ "mode": "affine"
222
+ },
223
+ "model.layers.6.mlp.router": {
224
+ "group_size": 64,
225
+ "bits": 8
226
+ },
227
+ "model.layers.7.self_attn.q_proj": {
228
+ "group_size": 64,
229
+ "bits": 5,
230
+ "mode": "affine"
231
+ },
232
+ "model.layers.7.self_attn.k_proj": {
233
+ "group_size": 64,
234
+ "bits": 5,
235
+ "mode": "affine"
236
+ },
237
+ "model.layers.7.self_attn.v_proj": {
238
+ "group_size": 64,
239
+ "bits": 5,
240
+ "mode": "affine"
241
+ },
242
+ "model.layers.7.self_attn.o_proj": {
243
+ "group_size": 64,
244
+ "bits": 5,
245
+ "mode": "affine"
246
+ },
247
+ "model.layers.7.mlp.router": {
248
+ "group_size": 64,
249
+ "bits": 8
250
+ },
251
+ "model.layers.8.self_attn.q_proj": {
252
+ "group_size": 64,
253
+ "bits": 5,
254
+ "mode": "affine"
255
+ },
256
+ "model.layers.8.self_attn.k_proj": {
257
+ "group_size": 64,
258
+ "bits": 5,
259
+ "mode": "affine"
260
+ },
261
+ "model.layers.8.self_attn.v_proj": {
262
+ "group_size": 64,
263
+ "bits": 5,
264
+ "mode": "affine"
265
+ },
266
+ "model.layers.8.self_attn.o_proj": {
267
+ "group_size": 64,
268
+ "bits": 5,
269
+ "mode": "affine"
270
+ },
271
+ "model.layers.8.mlp.router": {
272
+ "group_size": 64,
273
+ "bits": 8
274
+ },
275
+ "model.layers.9.self_attn.q_proj": {
276
+ "group_size": 64,
277
+ "bits": 5,
278
+ "mode": "affine"
279
+ },
280
+ "model.layers.9.self_attn.k_proj": {
281
+ "group_size": 64,
282
+ "bits": 5,
283
+ "mode": "affine"
284
+ },
285
+ "model.layers.9.self_attn.v_proj": {
286
+ "group_size": 64,
287
+ "bits": 5,
288
+ "mode": "affine"
289
+ },
290
+ "model.layers.9.self_attn.o_proj": {
291
+ "group_size": 64,
292
+ "bits": 5,
293
+ "mode": "affine"
294
+ },
295
+ "model.layers.9.mlp.router": {
296
+ "group_size": 64,
297
+ "bits": 8
298
+ },
299
+ "model.layers.10.self_attn.q_proj": {
300
+ "group_size": 64,
301
+ "bits": 5,
302
+ "mode": "affine"
303
+ },
304
+ "model.layers.10.self_attn.k_proj": {
305
+ "group_size": 64,
306
+ "bits": 5,
307
+ "mode": "affine"
308
+ },
309
+ "model.layers.10.self_attn.v_proj": {
310
+ "group_size": 64,
311
+ "bits": 5,
312
+ "mode": "affine"
313
+ },
314
+ "model.layers.10.self_attn.o_proj": {
315
+ "group_size": 64,
316
+ "bits": 5,
317
+ "mode": "affine"
318
+ },
319
+ "model.layers.10.mlp.router": {
320
+ "group_size": 64,
321
+ "bits": 8
322
+ },
323
+ "model.layers.11.self_attn.q_proj": {
324
+ "group_size": 64,
325
+ "bits": 5,
326
+ "mode": "affine"
327
+ },
328
+ "model.layers.11.self_attn.k_proj": {
329
+ "group_size": 64,
330
+ "bits": 5,
331
+ "mode": "affine"
332
+ },
333
+ "model.layers.11.self_attn.v_proj": {
334
+ "group_size": 64,
335
+ "bits": 5,
336
+ "mode": "affine"
337
+ },
338
+ "model.layers.11.self_attn.o_proj": {
339
+ "group_size": 64,
340
+ "bits": 5,
341
+ "mode": "affine"
342
+ },
343
+ "model.layers.11.mlp.router": {
344
+ "group_size": 64,
345
+ "bits": 8
346
+ },
347
+ "model.layers.12.self_attn.q_proj": {
348
+ "group_size": 64,
349
+ "bits": 5,
350
+ "mode": "affine"
351
+ },
352
+ "model.layers.12.self_attn.k_proj": {
353
+ "group_size": 64,
354
+ "bits": 5,
355
+ "mode": "affine"
356
+ },
357
+ "model.layers.12.self_attn.v_proj": {
358
+ "group_size": 64,
359
+ "bits": 5,
360
+ "mode": "affine"
361
+ },
362
+ "model.layers.12.self_attn.o_proj": {
363
+ "group_size": 64,
364
+ "bits": 5,
365
+ "mode": "affine"
366
+ },
367
+ "model.layers.12.mlp.router": {
368
+ "group_size": 64,
369
+ "bits": 8
370
+ },
371
+ "model.layers.13.self_attn.q_proj": {
372
+ "group_size": 64,
373
+ "bits": 5,
374
+ "mode": "affine"
375
+ },
376
+ "model.layers.13.self_attn.k_proj": {
377
+ "group_size": 64,
378
+ "bits": 5,
379
+ "mode": "affine"
380
+ },
381
+ "model.layers.13.self_attn.v_proj": {
382
+ "group_size": 64,
383
+ "bits": 5,
384
+ "mode": "affine"
385
+ },
386
+ "model.layers.13.self_attn.o_proj": {
387
+ "group_size": 64,
388
+ "bits": 5,
389
+ "mode": "affine"
390
+ },
391
+ "model.layers.13.mlp.router": {
392
+ "group_size": 64,
393
+ "bits": 8
394
+ },
395
+ "model.layers.14.self_attn.q_proj": {
396
+ "group_size": 64,
397
+ "bits": 5,
398
+ "mode": "affine"
399
+ },
400
+ "model.layers.14.self_attn.k_proj": {
401
+ "group_size": 64,
402
+ "bits": 5,
403
+ "mode": "affine"
404
+ },
405
+ "model.layers.14.self_attn.v_proj": {
406
+ "group_size": 64,
407
+ "bits": 5,
408
+ "mode": "affine"
409
+ },
410
+ "model.layers.14.self_attn.o_proj": {
411
+ "group_size": 64,
412
+ "bits": 5,
413
+ "mode": "affine"
414
+ },
415
+ "model.layers.14.mlp.router": {
416
+ "group_size": 64,
417
+ "bits": 8
418
+ },
419
+ "model.layers.15.self_attn.q_proj": {
420
+ "group_size": 64,
421
+ "bits": 5,
422
+ "mode": "affine"
423
+ },
424
+ "model.layers.15.self_attn.k_proj": {
425
+ "group_size": 64,
426
+ "bits": 5,
427
+ "mode": "affine"
428
+ },
429
+ "model.layers.15.self_attn.v_proj": {
430
+ "group_size": 64,
431
+ "bits": 5,
432
+ "mode": "affine"
433
+ },
434
+ "model.layers.15.self_attn.o_proj": {
435
+ "group_size": 64,
436
+ "bits": 5,
437
+ "mode": "affine"
438
+ },
439
+ "model.layers.15.mlp.router": {
440
+ "group_size": 64,
441
+ "bits": 8
442
+ },
443
+ "model.layers.16.self_attn.q_proj": {
444
+ "group_size": 64,
445
+ "bits": 5,
446
+ "mode": "affine"
447
+ },
448
+ "model.layers.16.self_attn.k_proj": {
449
+ "group_size": 64,
450
+ "bits": 5,
451
+ "mode": "affine"
452
+ },
453
+ "model.layers.16.self_attn.v_proj": {
454
+ "group_size": 64,
455
+ "bits": 5,
456
+ "mode": "affine"
457
+ },
458
+ "model.layers.16.self_attn.o_proj": {
459
+ "group_size": 64,
460
+ "bits": 5,
461
+ "mode": "affine"
462
+ },
463
+ "model.layers.16.mlp.router": {
464
+ "group_size": 64,
465
+ "bits": 8
466
+ },
467
+ "model.layers.17.self_attn.q_proj": {
468
+ "group_size": 64,
469
+ "bits": 5,
470
+ "mode": "affine"
471
+ },
472
+ "model.layers.17.self_attn.k_proj": {
473
+ "group_size": 64,
474
+ "bits": 5,
475
+ "mode": "affine"
476
+ },
477
+ "model.layers.17.self_attn.v_proj": {
478
+ "group_size": 64,
479
+ "bits": 5,
480
+ "mode": "affine"
481
+ },
482
+ "model.layers.17.self_attn.o_proj": {
483
+ "group_size": 64,
484
+ "bits": 5,
485
+ "mode": "affine"
486
+ },
487
+ "model.layers.17.mlp.router": {
488
+ "group_size": 64,
489
+ "bits": 8
490
+ },
491
+ "model.layers.18.self_attn.q_proj": {
492
+ "group_size": 64,
493
+ "bits": 5,
494
+ "mode": "affine"
495
+ },
496
+ "model.layers.18.self_attn.k_proj": {
497
+ "group_size": 64,
498
+ "bits": 5,
499
+ "mode": "affine"
500
+ },
501
+ "model.layers.18.self_attn.v_proj": {
502
+ "group_size": 64,
503
+ "bits": 5,
504
+ "mode": "affine"
505
+ },
506
+ "model.layers.18.self_attn.o_proj": {
507
+ "group_size": 64,
508
+ "bits": 5,
509
+ "mode": "affine"
510
+ },
511
+ "model.layers.18.mlp.router": {
512
+ "group_size": 64,
513
+ "bits": 8
514
+ },
515
+ "model.layers.19.self_attn.q_proj": {
516
+ "group_size": 64,
517
+ "bits": 5,
518
+ "mode": "affine"
519
+ },
520
+ "model.layers.19.self_attn.k_proj": {
521
+ "group_size": 64,
522
+ "bits": 5,
523
+ "mode": "affine"
524
+ },
525
+ "model.layers.19.self_attn.v_proj": {
526
+ "group_size": 64,
527
+ "bits": 5,
528
+ "mode": "affine"
529
+ },
530
+ "model.layers.19.self_attn.o_proj": {
531
+ "group_size": 64,
532
+ "bits": 5,
533
+ "mode": "affine"
534
+ },
535
+ "model.layers.19.mlp.router": {
536
+ "group_size": 64,
537
+ "bits": 8
538
+ },
539
+ "model.layers.20.self_attn.q_proj": {
540
+ "group_size": 64,
541
+ "bits": 5,
542
+ "mode": "affine"
543
+ },
544
+ "model.layers.20.self_attn.k_proj": {
545
+ "group_size": 64,
546
+ "bits": 5,
547
+ "mode": "affine"
548
+ },
549
+ "model.layers.20.self_attn.v_proj": {
550
+ "group_size": 64,
551
+ "bits": 5,
552
+ "mode": "affine"
553
+ },
554
+ "model.layers.20.self_attn.o_proj": {
555
+ "group_size": 64,
556
+ "bits": 5,
557
+ "mode": "affine"
558
+ },
559
+ "model.layers.20.mlp.router": {
560
+ "group_size": 64,
561
+ "bits": 8
562
+ },
563
+ "model.layers.21.self_attn.q_proj": {
564
+ "group_size": 64,
565
+ "bits": 5,
566
+ "mode": "affine"
567
+ },
568
+ "model.layers.21.self_attn.k_proj": {
569
+ "group_size": 64,
570
+ "bits": 5,
571
+ "mode": "affine"
572
+ },
573
+ "model.layers.21.self_attn.v_proj": {
574
+ "group_size": 64,
575
+ "bits": 5,
576
+ "mode": "affine"
577
+ },
578
+ "model.layers.21.self_attn.o_proj": {
579
+ "group_size": 64,
580
+ "bits": 5,
581
+ "mode": "affine"
582
+ },
583
+ "model.layers.21.mlp.router": {
584
+ "group_size": 64,
585
+ "bits": 8
586
+ },
587
+ "model.layers.22.self_attn.q_proj": {
588
+ "group_size": 64,
589
+ "bits": 5,
590
+ "mode": "affine"
591
+ },
592
+ "model.layers.22.self_attn.k_proj": {
593
+ "group_size": 64,
594
+ "bits": 5,
595
+ "mode": "affine"
596
+ },
597
+ "model.layers.22.self_attn.v_proj": {
598
+ "group_size": 64,
599
+ "bits": 5,
600
+ "mode": "affine"
601
+ },
602
+ "model.layers.22.self_attn.o_proj": {
603
+ "group_size": 64,
604
+ "bits": 5,
605
+ "mode": "affine"
606
+ },
607
+ "model.layers.22.mlp.router": {
608
+ "group_size": 64,
609
+ "bits": 8
610
+ },
611
+ "model.layers.23.self_attn.q_proj": {
612
+ "group_size": 64,
613
+ "bits": 5,
614
+ "mode": "affine"
615
+ },
616
+ "model.layers.23.self_attn.k_proj": {
617
+ "group_size": 64,
618
+ "bits": 5,
619
+ "mode": "affine"
620
+ },
621
+ "model.layers.23.self_attn.v_proj": {
622
+ "group_size": 64,
623
+ "bits": 5,
624
+ "mode": "affine"
625
+ },
626
+ "model.layers.23.self_attn.o_proj": {
627
+ "group_size": 64,
628
+ "bits": 5,
629
+ "mode": "affine"
630
+ },
631
+ "model.layers.23.mlp.router": {
632
+ "group_size": 64,
633
+ "bits": 8
634
+ },
635
+ "lm_head": {
636
+ "group_size": 64,
637
+ "bits": 5,
638
+ "mode": "affine"
639
+ }
640
  },
641
  "quantization_config": {
642
+ "group_size": 32,
643
+ "bits": 4,
644
+ "mode": "mxfp4",
645
+ "model.embed_tokens": {
646
+ "group_size": 64,
647
+ "bits": 5,
648
+ "mode": "affine"
649
+ },
650
+ "model.layers.0.self_attn.q_proj": {
651
+ "group_size": 64,
652
+ "bits": 5,
653
+ "mode": "affine"
654
+ },
655
+ "model.layers.0.self_attn.k_proj": {
656
+ "group_size": 64,
657
+ "bits": 5,
658
+ "mode": "affine"
659
+ },
660
+ "model.layers.0.self_attn.v_proj": {
661
+ "group_size": 64,
662
+ "bits": 5,
663
+ "mode": "affine"
664
+ },
665
+ "model.layers.0.self_attn.o_proj": {
666
+ "group_size": 64,
667
+ "bits": 5,
668
+ "mode": "affine"
669
+ },
670
+ "model.layers.0.mlp.router": {
671
+ "group_size": 64,
672
+ "bits": 8
673
+ },
674
+ "model.layers.1.self_attn.q_proj": {
675
+ "group_size": 64,
676
+ "bits": 5,
677
+ "mode": "affine"
678
+ },
679
+ "model.layers.1.self_attn.k_proj": {
680
+ "group_size": 64,
681
+ "bits": 5,
682
+ "mode": "affine"
683
+ },
684
+ "model.layers.1.self_attn.v_proj": {
685
+ "group_size": 64,
686
+ "bits": 5,
687
+ "mode": "affine"
688
+ },
689
+ "model.layers.1.self_attn.o_proj": {
690
+ "group_size": 64,
691
+ "bits": 5,
692
+ "mode": "affine"
693
+ },
694
+ "model.layers.1.mlp.router": {
695
+ "group_size": 64,
696
+ "bits": 8
697
+ },
698
+ "model.layers.2.self_attn.q_proj": {
699
+ "group_size": 64,
700
+ "bits": 5,
701
+ "mode": "affine"
702
+ },
703
+ "model.layers.2.self_attn.k_proj": {
704
+ "group_size": 64,
705
+ "bits": 5,
706
+ "mode": "affine"
707
+ },
708
+ "model.layers.2.self_attn.v_proj": {
709
+ "group_size": 64,
710
+ "bits": 5,
711
+ "mode": "affine"
712
+ },
713
+ "model.layers.2.self_attn.o_proj": {
714
+ "group_size": 64,
715
+ "bits": 5,
716
+ "mode": "affine"
717
+ },
718
+ "model.layers.2.mlp.router": {
719
+ "group_size": 64,
720
+ "bits": 8
721
+ },
722
+ "model.layers.3.self_attn.q_proj": {
723
+ "group_size": 64,
724
+ "bits": 5,
725
+ "mode": "affine"
726
+ },
727
+ "model.layers.3.self_attn.k_proj": {
728
+ "group_size": 64,
729
+ "bits": 5,
730
+ "mode": "affine"
731
+ },
732
+ "model.layers.3.self_attn.v_proj": {
733
+ "group_size": 64,
734
+ "bits": 5,
735
+ "mode": "affine"
736
+ },
737
+ "model.layers.3.self_attn.o_proj": {
738
+ "group_size": 64,
739
+ "bits": 5,
740
+ "mode": "affine"
741
+ },
742
+ "model.layers.3.mlp.router": {
743
+ "group_size": 64,
744
+ "bits": 8
745
+ },
746
+ "model.layers.4.self_attn.q_proj": {
747
+ "group_size": 64,
748
+ "bits": 5,
749
+ "mode": "affine"
750
+ },
751
+ "model.layers.4.self_attn.k_proj": {
752
+ "group_size": 64,
753
+ "bits": 5,
754
+ "mode": "affine"
755
+ },
756
+ "model.layers.4.self_attn.v_proj": {
757
+ "group_size": 64,
758
+ "bits": 5,
759
+ "mode": "affine"
760
+ },
761
+ "model.layers.4.self_attn.o_proj": {
762
+ "group_size": 64,
763
+ "bits": 5,
764
+ "mode": "affine"
765
+ },
766
+ "model.layers.4.mlp.router": {
767
+ "group_size": 64,
768
+ "bits": 8
769
+ },
770
+ "model.layers.5.self_attn.q_proj": {
771
+ "group_size": 64,
772
+ "bits": 5,
773
+ "mode": "affine"
774
+ },
775
+ "model.layers.5.self_attn.k_proj": {
776
+ "group_size": 64,
777
+ "bits": 5,
778
+ "mode": "affine"
779
+ },
780
+ "model.layers.5.self_attn.v_proj": {
781
+ "group_size": 64,
782
+ "bits": 5,
783
+ "mode": "affine"
784
+ },
785
+ "model.layers.5.self_attn.o_proj": {
786
+ "group_size": 64,
787
+ "bits": 5,
788
+ "mode": "affine"
789
+ },
790
+ "model.layers.5.mlp.router": {
791
+ "group_size": 64,
792
+ "bits": 8
793
+ },
794
+ "model.layers.6.self_attn.q_proj": {
795
+ "group_size": 64,
796
+ "bits": 5,
797
+ "mode": "affine"
798
+ },
799
+ "model.layers.6.self_attn.k_proj": {
800
+ "group_size": 64,
801
+ "bits": 5,
802
+ "mode": "affine"
803
+ },
804
+ "model.layers.6.self_attn.v_proj": {
805
+ "group_size": 64,
806
+ "bits": 5,
807
+ "mode": "affine"
808
+ },
809
+ "model.layers.6.self_attn.o_proj": {
810
+ "group_size": 64,
811
+ "bits": 5,
812
+ "mode": "affine"
813
+ },
814
+ "model.layers.6.mlp.router": {
815
+ "group_size": 64,
816
+ "bits": 8
817
+ },
818
+ "model.layers.7.self_attn.q_proj": {
819
+ "group_size": 64,
820
+ "bits": 5,
821
+ "mode": "affine"
822
+ },
823
+ "model.layers.7.self_attn.k_proj": {
824
+ "group_size": 64,
825
+ "bits": 5,
826
+ "mode": "affine"
827
+ },
828
+ "model.layers.7.self_attn.v_proj": {
829
+ "group_size": 64,
830
+ "bits": 5,
831
+ "mode": "affine"
832
+ },
833
+ "model.layers.7.self_attn.o_proj": {
834
+ "group_size": 64,
835
+ "bits": 5,
836
+ "mode": "affine"
837
+ },
838
+ "model.layers.7.mlp.router": {
839
+ "group_size": 64,
840
+ "bits": 8
841
+ },
842
+ "model.layers.8.self_attn.q_proj": {
843
+ "group_size": 64,
844
+ "bits": 5,
845
+ "mode": "affine"
846
+ },
847
+ "model.layers.8.self_attn.k_proj": {
848
+ "group_size": 64,
849
+ "bits": 5,
850
+ "mode": "affine"
851
+ },
852
+ "model.layers.8.self_attn.v_proj": {
853
+ "group_size": 64,
854
+ "bits": 5,
855
+ "mode": "affine"
856
+ },
857
+ "model.layers.8.self_attn.o_proj": {
858
+ "group_size": 64,
859
+ "bits": 5,
860
+ "mode": "affine"
861
+ },
862
+ "model.layers.8.mlp.router": {
863
+ "group_size": 64,
864
+ "bits": 8
865
+ },
866
+ "model.layers.9.self_attn.q_proj": {
867
+ "group_size": 64,
868
+ "bits": 5,
869
+ "mode": "affine"
870
+ },
871
+ "model.layers.9.self_attn.k_proj": {
872
+ "group_size": 64,
873
+ "bits": 5,
874
+ "mode": "affine"
875
+ },
876
+ "model.layers.9.self_attn.v_proj": {
877
+ "group_size": 64,
878
+ "bits": 5,
879
+ "mode": "affine"
880
+ },
881
+ "model.layers.9.self_attn.o_proj": {
882
+ "group_size": 64,
883
+ "bits": 5,
884
+ "mode": "affine"
885
+ },
886
+ "model.layers.9.mlp.router": {
887
+ "group_size": 64,
888
+ "bits": 8
889
+ },
890
+ "model.layers.10.self_attn.q_proj": {
891
+ "group_size": 64,
892
+ "bits": 5,
893
+ "mode": "affine"
894
+ },
895
+ "model.layers.10.self_attn.k_proj": {
896
+ "group_size": 64,
897
+ "bits": 5,
898
+ "mode": "affine"
899
+ },
900
+ "model.layers.10.self_attn.v_proj": {
901
+ "group_size": 64,
902
+ "bits": 5,
903
+ "mode": "affine"
904
+ },
905
+ "model.layers.10.self_attn.o_proj": {
906
+ "group_size": 64,
907
+ "bits": 5,
908
+ "mode": "affine"
909
+ },
910
+ "model.layers.10.mlp.router": {
911
+ "group_size": 64,
912
+ "bits": 8
913
+ },
914
+ "model.layers.11.self_attn.q_proj": {
915
+ "group_size": 64,
916
+ "bits": 5,
917
+ "mode": "affine"
918
+ },
919
+ "model.layers.11.self_attn.k_proj": {
920
+ "group_size": 64,
921
+ "bits": 5,
922
+ "mode": "affine"
923
+ },
924
+ "model.layers.11.self_attn.v_proj": {
925
+ "group_size": 64,
926
+ "bits": 5,
927
+ "mode": "affine"
928
+ },
929
+ "model.layers.11.self_attn.o_proj": {
930
+ "group_size": 64,
931
+ "bits": 5,
932
+ "mode": "affine"
933
+ },
934
+ "model.layers.11.mlp.router": {
935
+ "group_size": 64,
936
+ "bits": 8
937
+ },
938
+ "model.layers.12.self_attn.q_proj": {
939
+ "group_size": 64,
940
+ "bits": 5,
941
+ "mode": "affine"
942
+ },
943
+ "model.layers.12.self_attn.k_proj": {
944
+ "group_size": 64,
945
+ "bits": 5,
946
+ "mode": "affine"
947
+ },
948
+ "model.layers.12.self_attn.v_proj": {
949
+ "group_size": 64,
950
+ "bits": 5,
951
+ "mode": "affine"
952
+ },
953
+ "model.layers.12.self_attn.o_proj": {
954
+ "group_size": 64,
955
+ "bits": 5,
956
+ "mode": "affine"
957
+ },
958
+ "model.layers.12.mlp.router": {
959
+ "group_size": 64,
960
+ "bits": 8
961
+ },
962
+ "model.layers.13.self_attn.q_proj": {
963
+ "group_size": 64,
964
+ "bits": 5,
965
+ "mode": "affine"
966
+ },
967
+ "model.layers.13.self_attn.k_proj": {
968
+ "group_size": 64,
969
+ "bits": 5,
970
+ "mode": "affine"
971
+ },
972
+ "model.layers.13.self_attn.v_proj": {
973
+ "group_size": 64,
974
+ "bits": 5,
975
+ "mode": "affine"
976
+ },
977
+ "model.layers.13.self_attn.o_proj": {
978
+ "group_size": 64,
979
+ "bits": 5,
980
+ "mode": "affine"
981
+ },
982
+ "model.layers.13.mlp.router": {
983
+ "group_size": 64,
984
+ "bits": 8
985
+ },
986
+ "model.layers.14.self_attn.q_proj": {
987
+ "group_size": 64,
988
+ "bits": 5,
989
+ "mode": "affine"
990
+ },
991
+ "model.layers.14.self_attn.k_proj": {
992
+ "group_size": 64,
993
+ "bits": 5,
994
+ "mode": "affine"
995
+ },
996
+ "model.layers.14.self_attn.v_proj": {
997
+ "group_size": 64,
998
+ "bits": 5,
999
+ "mode": "affine"
1000
+ },
1001
+ "model.layers.14.self_attn.o_proj": {
1002
+ "group_size": 64,
1003
+ "bits": 5,
1004
+ "mode": "affine"
1005
+ },
1006
+ "model.layers.14.mlp.router": {
1007
+ "group_size": 64,
1008
+ "bits": 8
1009
+ },
1010
+ "model.layers.15.self_attn.q_proj": {
1011
+ "group_size": 64,
1012
+ "bits": 5,
1013
+ "mode": "affine"
1014
+ },
1015
+ "model.layers.15.self_attn.k_proj": {
1016
+ "group_size": 64,
1017
+ "bits": 5,
1018
+ "mode": "affine"
1019
+ },
1020
+ "model.layers.15.self_attn.v_proj": {
1021
+ "group_size": 64,
1022
+ "bits": 5,
1023
+ "mode": "affine"
1024
+ },
1025
+ "model.layers.15.self_attn.o_proj": {
1026
+ "group_size": 64,
1027
+ "bits": 5,
1028
+ "mode": "affine"
1029
+ },
1030
+ "model.layers.15.mlp.router": {
1031
+ "group_size": 64,
1032
+ "bits": 8
1033
+ },
1034
+ "model.layers.16.self_attn.q_proj": {
1035
+ "group_size": 64,
1036
+ "bits": 5,
1037
+ "mode": "affine"
1038
+ },
1039
+ "model.layers.16.self_attn.k_proj": {
1040
+ "group_size": 64,
1041
+ "bits": 5,
1042
+ "mode": "affine"
1043
+ },
1044
+ "model.layers.16.self_attn.v_proj": {
1045
+ "group_size": 64,
1046
+ "bits": 5,
1047
+ "mode": "affine"
1048
+ },
1049
+ "model.layers.16.self_attn.o_proj": {
1050
+ "group_size": 64,
1051
+ "bits": 5,
1052
+ "mode": "affine"
1053
+ },
1054
+ "model.layers.16.mlp.router": {
1055
+ "group_size": 64,
1056
+ "bits": 8
1057
+ },
1058
+ "model.layers.17.self_attn.q_proj": {
1059
+ "group_size": 64,
1060
+ "bits": 5,
1061
+ "mode": "affine"
1062
+ },
1063
+ "model.layers.17.self_attn.k_proj": {
1064
+ "group_size": 64,
1065
+ "bits": 5,
1066
+ "mode": "affine"
1067
+ },
1068
+ "model.layers.17.self_attn.v_proj": {
1069
+ "group_size": 64,
1070
+ "bits": 5,
1071
+ "mode": "affine"
1072
+ },
1073
+ "model.layers.17.self_attn.o_proj": {
1074
+ "group_size": 64,
1075
+ "bits": 5,
1076
+ "mode": "affine"
1077
+ },
1078
+ "model.layers.17.mlp.router": {
1079
+ "group_size": 64,
1080
+ "bits": 8
1081
+ },
1082
+ "model.layers.18.self_attn.q_proj": {
1083
+ "group_size": 64,
1084
+ "bits": 5,
1085
+ "mode": "affine"
1086
+ },
1087
+ "model.layers.18.self_attn.k_proj": {
1088
+ "group_size": 64,
1089
+ "bits": 5,
1090
+ "mode": "affine"
1091
+ },
1092
+ "model.layers.18.self_attn.v_proj": {
1093
+ "group_size": 64,
1094
+ "bits": 5,
1095
+ "mode": "affine"
1096
+ },
1097
+ "model.layers.18.self_attn.o_proj": {
1098
+ "group_size": 64,
1099
+ "bits": 5,
1100
+ "mode": "affine"
1101
+ },
1102
+ "model.layers.18.mlp.router": {
1103
+ "group_size": 64,
1104
+ "bits": 8
1105
+ },
1106
+ "model.layers.19.self_attn.q_proj": {
1107
+ "group_size": 64,
1108
+ "bits": 5,
1109
+ "mode": "affine"
1110
+ },
1111
+ "model.layers.19.self_attn.k_proj": {
1112
+ "group_size": 64,
1113
+ "bits": 5,
1114
+ "mode": "affine"
1115
+ },
1116
+ "model.layers.19.self_attn.v_proj": {
1117
+ "group_size": 64,
1118
+ "bits": 5,
1119
+ "mode": "affine"
1120
+ },
1121
+ "model.layers.19.self_attn.o_proj": {
1122
+ "group_size": 64,
1123
+ "bits": 5,
1124
+ "mode": "affine"
1125
+ },
1126
+ "model.layers.19.mlp.router": {
1127
+ "group_size": 64,
1128
+ "bits": 8
1129
+ },
1130
+ "model.layers.20.self_attn.q_proj": {
1131
+ "group_size": 64,
1132
+ "bits": 5,
1133
+ "mode": "affine"
1134
+ },
1135
+ "model.layers.20.self_attn.k_proj": {
1136
+ "group_size": 64,
1137
+ "bits": 5,
1138
+ "mode": "affine"
1139
+ },
1140
+ "model.layers.20.self_attn.v_proj": {
1141
+ "group_size": 64,
1142
+ "bits": 5,
1143
+ "mode": "affine"
1144
+ },
1145
+ "model.layers.20.self_attn.o_proj": {
1146
+ "group_size": 64,
1147
+ "bits": 5,
1148
+ "mode": "affine"
1149
+ },
1150
+ "model.layers.20.mlp.router": {
1151
+ "group_size": 64,
1152
+ "bits": 8
1153
+ },
1154
+ "model.layers.21.self_attn.q_proj": {
1155
+ "group_size": 64,
1156
+ "bits": 5,
1157
+ "mode": "affine"
1158
+ },
1159
+ "model.layers.21.self_attn.k_proj": {
1160
+ "group_size": 64,
1161
+ "bits": 5,
1162
+ "mode": "affine"
1163
+ },
1164
+ "model.layers.21.self_attn.v_proj": {
1165
+ "group_size": 64,
1166
+ "bits": 5,
1167
+ "mode": "affine"
1168
+ },
1169
+ "model.layers.21.self_attn.o_proj": {
1170
+ "group_size": 64,
1171
+ "bits": 5,
1172
+ "mode": "affine"
1173
+ },
1174
+ "model.layers.21.mlp.router": {
1175
+ "group_size": 64,
1176
+ "bits": 8
1177
+ },
1178
+ "model.layers.22.self_attn.q_proj": {
1179
+ "group_size": 64,
1180
+ "bits": 5,
1181
+ "mode": "affine"
1182
+ },
1183
+ "model.layers.22.self_attn.k_proj": {
1184
+ "group_size": 64,
1185
+ "bits": 5,
1186
+ "mode": "affine"
1187
+ },
1188
+ "model.layers.22.self_attn.v_proj": {
1189
+ "group_size": 64,
1190
+ "bits": 5,
1191
+ "mode": "affine"
1192
+ },
1193
+ "model.layers.22.self_attn.o_proj": {
1194
+ "group_size": 64,
1195
+ "bits": 5,
1196
+ "mode": "affine"
1197
+ },
1198
+ "model.layers.22.mlp.router": {
1199
+ "group_size": 64,
1200
+ "bits": 8
1201
+ },
1202
+ "model.layers.23.self_attn.q_proj": {
1203
+ "group_size": 64,
1204
+ "bits": 5,
1205
+ "mode": "affine"
1206
+ },
1207
+ "model.layers.23.self_attn.k_proj": {
1208
+ "group_size": 64,
1209
+ "bits": 5,
1210
+ "mode": "affine"
1211
+ },
1212
+ "model.layers.23.self_attn.v_proj": {
1213
+ "group_size": 64,
1214
+ "bits": 5,
1215
+ "mode": "affine"
1216
+ },
1217
+ "model.layers.23.self_attn.o_proj": {
1218
+ "group_size": 64,
1219
+ "bits": 5,
1220
+ "mode": "affine"
1221
+ },
1222
+ "model.layers.23.mlp.router": {
1223
+ "group_size": 64,
1224
+ "bits": 8
1225
+ },
1226
+ "lm_head": {
1227
+ "group_size": 64,
1228
+ "bits": 5,
1229
+ "mode": "affine"
1230
+ }
1231
  },
1232
  "rms_norm_eps": 1e-05,
1233
  "rope_scaling": {
generation_config.json CHANGED
@@ -3,7 +3,8 @@
3
  "do_sample": true,
4
  "eos_token_id": [
5
  200002,
6
- 199999
 
7
  ],
8
  "pad_token_id": 199999,
9
  "transformers_version": "4.55.0.dev0"
 
3
  "do_sample": true,
4
  "eos_token_id": [
5
  200002,
6
+ 199999,
7
+ 200012
8
  ],
9
  "pad_token_id": 199999,
10
  "transformers_version": "4.55.0.dev0"
model-00001-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3e773699e6882256aa76b98894ad59e6b52914f7ae1078b05a37551249ddc8c1
3
- size 5312358329
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:54af46e4aa9ad54e1128f9c9b2ee505616f3bfda8d07df25fb697f310b44a3e2
3
+ size 5277813520
model-00002-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:27374654d1dcba36afaa3a778cb02b2d91744b9abdeaf17ed3f52712b525e7f5
3
- size 5297938277
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e5e742f768b7526887cd1d3035b69841fedad7251a7f9d4a8f68bd91e5d12459
3
+ size 5303332040
model-00003-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:30431d893ce28bb82995bd91ee6933e3a79857c51ef69eaa45877869466e0101
3
- size 3777845658
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:57468f9683ca7a3d98de1fcda6c74f3f3a55fe58e5bfe16a715abda0f28e88d4
3
+ size 821833127
model.safetensors.index.json CHANGED
@@ -1,6 +1,6 @@
1
  {
2
  "metadata": {
3
- "total_size": 14388044928,
4
  "total_parameters": 20914755648
5
  },
6
  "weight_map": {
@@ -12,15 +12,12 @@
12
  "model.embed_tokens.weight": "model-00001-of-00003.safetensors",
13
  "model.layers.0.input_layernorm.weight": "model-00001-of-00003.safetensors",
14
  "model.layers.0.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
15
- "model.layers.0.mlp.experts.down_proj.biases": "model-00001-of-00003.safetensors",
16
  "model.layers.0.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
17
  "model.layers.0.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
18
  "model.layers.0.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
19
- "model.layers.0.mlp.experts.gate_proj.biases": "model-00001-of-00003.safetensors",
20
  "model.layers.0.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
21
  "model.layers.0.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
22
  "model.layers.0.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
23
- "model.layers.0.mlp.experts.up_proj.biases": "model-00001-of-00003.safetensors",
24
  "model.layers.0.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
25
  "model.layers.0.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
26
  "model.layers.0.mlp.router.bias": "model-00001-of-00003.safetensors",
@@ -47,15 +44,12 @@
47
  "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
48
  "model.layers.1.input_layernorm.weight": "model-00001-of-00003.safetensors",
49
  "model.layers.1.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
50
- "model.layers.1.mlp.experts.down_proj.biases": "model-00001-of-00003.safetensors",
51
  "model.layers.1.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
52
  "model.layers.1.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
53
  "model.layers.1.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
54
- "model.layers.1.mlp.experts.gate_proj.biases": "model-00001-of-00003.safetensors",
55
  "model.layers.1.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
56
  "model.layers.1.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
57
  "model.layers.1.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
58
- "model.layers.1.mlp.experts.up_proj.biases": "model-00001-of-00003.safetensors",
59
  "model.layers.1.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
60
  "model.layers.1.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
61
  "model.layers.1.mlp.router.bias": "model-00001-of-00003.safetensors",
@@ -80,52 +74,46 @@
80
  "model.layers.1.self_attn.v_proj.biases": "model-00001-of-00003.safetensors",
81
  "model.layers.1.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
82
  "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
83
- "model.layers.10.input_layernorm.weight": "model-00002-of-00003.safetensors",
84
- "model.layers.10.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
85
- "model.layers.10.mlp.experts.down_proj.biases": "model-00002-of-00003.safetensors",
86
- "model.layers.10.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
87
- "model.layers.10.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
88
- "model.layers.10.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
89
- "model.layers.10.mlp.experts.gate_proj.biases": "model-00002-of-00003.safetensors",
90
- "model.layers.10.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
91
- "model.layers.10.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
92
- "model.layers.10.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
93
- "model.layers.10.mlp.experts.up_proj.biases": "model-00002-of-00003.safetensors",
94
- "model.layers.10.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
95
- "model.layers.10.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
96
- "model.layers.10.mlp.router.bias": "model-00002-of-00003.safetensors",
97
- "model.layers.10.mlp.router.biases": "model-00002-of-00003.safetensors",
98
- "model.layers.10.mlp.router.scales": "model-00002-of-00003.safetensors",
99
- "model.layers.10.mlp.router.weight": "model-00002-of-00003.safetensors",
100
- "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
101
- "model.layers.10.self_attn.k_proj.bias": "model-00002-of-00003.safetensors",
102
- "model.layers.10.self_attn.k_proj.biases": "model-00002-of-00003.safetensors",
103
- "model.layers.10.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
104
- "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
105
- "model.layers.10.self_attn.o_proj.bias": "model-00002-of-00003.safetensors",
106
- "model.layers.10.self_attn.o_proj.biases": "model-00002-of-00003.safetensors",
107
- "model.layers.10.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
108
- "model.layers.10.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
109
- "model.layers.10.self_attn.q_proj.bias": "model-00002-of-00003.safetensors",
110
- "model.layers.10.self_attn.q_proj.biases": "model-00002-of-00003.safetensors",
111
- "model.layers.10.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
112
- "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
113
- "model.layers.10.self_attn.sinks": "model-00002-of-00003.safetensors",
114
- "model.layers.10.self_attn.v_proj.bias": "model-00002-of-00003.safetensors",
115
- "model.layers.10.self_attn.v_proj.biases": "model-00002-of-00003.safetensors",
116
- "model.layers.10.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
117
- "model.layers.10.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
118
  "model.layers.11.input_layernorm.weight": "model-00002-of-00003.safetensors",
119
  "model.layers.11.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
120
- "model.layers.11.mlp.experts.down_proj.biases": "model-00002-of-00003.safetensors",
121
  "model.layers.11.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
122
  "model.layers.11.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
123
  "model.layers.11.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
124
- "model.layers.11.mlp.experts.gate_proj.biases": "model-00002-of-00003.safetensors",
125
  "model.layers.11.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
126
  "model.layers.11.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
127
  "model.layers.11.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
128
- "model.layers.11.mlp.experts.up_proj.biases": "model-00002-of-00003.safetensors",
129
  "model.layers.11.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
130
  "model.layers.11.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
131
  "model.layers.11.mlp.router.bias": "model-00002-of-00003.safetensors",
@@ -133,34 +121,31 @@
133
  "model.layers.11.mlp.router.scales": "model-00002-of-00003.safetensors",
134
  "model.layers.11.mlp.router.weight": "model-00002-of-00003.safetensors",
135
  "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
136
- "model.layers.11.self_attn.k_proj.bias": "model-00002-of-00003.safetensors",
137
- "model.layers.11.self_attn.k_proj.biases": "model-00002-of-00003.safetensors",
138
- "model.layers.11.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
139
- "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
140
- "model.layers.11.self_attn.o_proj.bias": "model-00002-of-00003.safetensors",
141
- "model.layers.11.self_attn.o_proj.biases": "model-00002-of-00003.safetensors",
142
- "model.layers.11.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
143
- "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
144
- "model.layers.11.self_attn.q_proj.bias": "model-00002-of-00003.safetensors",
145
- "model.layers.11.self_attn.q_proj.biases": "model-00002-of-00003.safetensors",
146
- "model.layers.11.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
147
- "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
148
- "model.layers.11.self_attn.sinks": "model-00002-of-00003.safetensors",
149
- "model.layers.11.self_attn.v_proj.bias": "model-00002-of-00003.safetensors",
150
- "model.layers.11.self_attn.v_proj.biases": "model-00002-of-00003.safetensors",
151
- "model.layers.11.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
152
- "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
153
  "model.layers.12.input_layernorm.weight": "model-00002-of-00003.safetensors",
154
  "model.layers.12.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
155
- "model.layers.12.mlp.experts.down_proj.biases": "model-00002-of-00003.safetensors",
156
  "model.layers.12.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
157
  "model.layers.12.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
158
  "model.layers.12.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
159
- "model.layers.12.mlp.experts.gate_proj.biases": "model-00002-of-00003.safetensors",
160
  "model.layers.12.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
161
  "model.layers.12.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
162
  "model.layers.12.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
163
- "model.layers.12.mlp.experts.up_proj.biases": "model-00002-of-00003.safetensors",
164
  "model.layers.12.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
165
  "model.layers.12.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
166
  "model.layers.12.mlp.router.bias": "model-00002-of-00003.safetensors",
@@ -187,15 +172,12 @@
187
  "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
188
  "model.layers.13.input_layernorm.weight": "model-00002-of-00003.safetensors",
189
  "model.layers.13.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
190
- "model.layers.13.mlp.experts.down_proj.biases": "model-00002-of-00003.safetensors",
191
  "model.layers.13.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
192
  "model.layers.13.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
193
  "model.layers.13.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
194
- "model.layers.13.mlp.experts.gate_proj.biases": "model-00002-of-00003.safetensors",
195
  "model.layers.13.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
196
  "model.layers.13.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
197
  "model.layers.13.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
198
- "model.layers.13.mlp.experts.up_proj.biases": "model-00002-of-00003.safetensors",
199
  "model.layers.13.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
200
  "model.layers.13.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
201
  "model.layers.13.mlp.router.bias": "model-00002-of-00003.safetensors",
@@ -222,15 +204,12 @@
222
  "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
223
  "model.layers.14.input_layernorm.weight": "model-00002-of-00003.safetensors",
224
  "model.layers.14.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
225
- "model.layers.14.mlp.experts.down_proj.biases": "model-00002-of-00003.safetensors",
226
  "model.layers.14.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
227
  "model.layers.14.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
228
  "model.layers.14.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
229
- "model.layers.14.mlp.experts.gate_proj.biases": "model-00002-of-00003.safetensors",
230
  "model.layers.14.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
231
  "model.layers.14.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
232
  "model.layers.14.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
233
- "model.layers.14.mlp.experts.up_proj.biases": "model-00002-of-00003.safetensors",
234
  "model.layers.14.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
235
  "model.layers.14.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
236
  "model.layers.14.mlp.router.bias": "model-00002-of-00003.safetensors",
@@ -257,15 +236,12 @@
257
  "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
258
  "model.layers.15.input_layernorm.weight": "model-00002-of-00003.safetensors",
259
  "model.layers.15.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
260
- "model.layers.15.mlp.experts.down_proj.biases": "model-00002-of-00003.safetensors",
261
  "model.layers.15.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
262
  "model.layers.15.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
263
  "model.layers.15.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
264
- "model.layers.15.mlp.experts.gate_proj.biases": "model-00002-of-00003.safetensors",
265
  "model.layers.15.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
266
  "model.layers.15.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
267
  "model.layers.15.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
268
- "model.layers.15.mlp.experts.up_proj.biases": "model-00002-of-00003.safetensors",
269
  "model.layers.15.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
270
  "model.layers.15.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
271
  "model.layers.15.mlp.router.bias": "model-00002-of-00003.safetensors",
@@ -292,15 +268,12 @@
292
  "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
293
  "model.layers.16.input_layernorm.weight": "model-00002-of-00003.safetensors",
294
  "model.layers.16.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
295
- "model.layers.16.mlp.experts.down_proj.biases": "model-00002-of-00003.safetensors",
296
  "model.layers.16.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
297
  "model.layers.16.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
298
  "model.layers.16.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
299
- "model.layers.16.mlp.experts.gate_proj.biases": "model-00002-of-00003.safetensors",
300
  "model.layers.16.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
301
  "model.layers.16.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
302
  "model.layers.16.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
303
- "model.layers.16.mlp.experts.up_proj.biases": "model-00002-of-00003.safetensors",
304
  "model.layers.16.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
305
  "model.layers.16.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
306
  "model.layers.16.mlp.router.bias": "model-00002-of-00003.safetensors",
@@ -327,15 +300,12 @@
327
  "model.layers.16.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
328
  "model.layers.17.input_layernorm.weight": "model-00002-of-00003.safetensors",
329
  "model.layers.17.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
330
- "model.layers.17.mlp.experts.down_proj.biases": "model-00002-of-00003.safetensors",
331
  "model.layers.17.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
332
  "model.layers.17.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
333
  "model.layers.17.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
334
- "model.layers.17.mlp.experts.gate_proj.biases": "model-00002-of-00003.safetensors",
335
  "model.layers.17.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
336
  "model.layers.17.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
337
  "model.layers.17.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
338
- "model.layers.17.mlp.experts.up_proj.biases": "model-00002-of-00003.safetensors",
339
  "model.layers.17.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
340
  "model.layers.17.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
341
  "model.layers.17.mlp.router.bias": "model-00002-of-00003.safetensors",
@@ -360,24 +330,21 @@
360
  "model.layers.17.self_attn.v_proj.biases": "model-00002-of-00003.safetensors",
361
  "model.layers.17.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
362
  "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
363
- "model.layers.18.input_layernorm.weight": "model-00003-of-00003.safetensors",
364
- "model.layers.18.mlp.experts.down_proj.bias": "model-00003-of-00003.safetensors",
365
- "model.layers.18.mlp.experts.down_proj.biases": "model-00003-of-00003.safetensors",
366
- "model.layers.18.mlp.experts.down_proj.scales": "model-00003-of-00003.safetensors",
367
- "model.layers.18.mlp.experts.down_proj.weight": "model-00003-of-00003.safetensors",
368
- "model.layers.18.mlp.experts.gate_proj.bias": "model-00003-of-00003.safetensors",
369
- "model.layers.18.mlp.experts.gate_proj.biases": "model-00003-of-00003.safetensors",
370
- "model.layers.18.mlp.experts.gate_proj.scales": "model-00003-of-00003.safetensors",
371
- "model.layers.18.mlp.experts.gate_proj.weight": "model-00003-of-00003.safetensors",
372
- "model.layers.18.mlp.experts.up_proj.bias": "model-00003-of-00003.safetensors",
373
- "model.layers.18.mlp.experts.up_proj.biases": "model-00003-of-00003.safetensors",
374
- "model.layers.18.mlp.experts.up_proj.scales": "model-00003-of-00003.safetensors",
375
- "model.layers.18.mlp.experts.up_proj.weight": "model-00003-of-00003.safetensors",
376
- "model.layers.18.mlp.router.bias": "model-00003-of-00003.safetensors",
377
- "model.layers.18.mlp.router.biases": "model-00003-of-00003.safetensors",
378
- "model.layers.18.mlp.router.scales": "model-00003-of-00003.safetensors",
379
- "model.layers.18.mlp.router.weight": "model-00003-of-00003.safetensors",
380
- "model.layers.18.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
381
  "model.layers.18.self_attn.k_proj.bias": "model-00002-of-00003.safetensors",
382
  "model.layers.18.self_attn.k_proj.biases": "model-00002-of-00003.safetensors",
383
  "model.layers.18.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
@@ -395,52 +362,46 @@
395
  "model.layers.18.self_attn.v_proj.biases": "model-00002-of-00003.safetensors",
396
  "model.layers.18.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
397
  "model.layers.18.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
398
- "model.layers.19.input_layernorm.weight": "model-00003-of-00003.safetensors",
399
- "model.layers.19.mlp.experts.down_proj.bias": "model-00003-of-00003.safetensors",
400
- "model.layers.19.mlp.experts.down_proj.biases": "model-00003-of-00003.safetensors",
401
- "model.layers.19.mlp.experts.down_proj.scales": "model-00003-of-00003.safetensors",
402
- "model.layers.19.mlp.experts.down_proj.weight": "model-00003-of-00003.safetensors",
403
- "model.layers.19.mlp.experts.gate_proj.bias": "model-00003-of-00003.safetensors",
404
- "model.layers.19.mlp.experts.gate_proj.biases": "model-00003-of-00003.safetensors",
405
- "model.layers.19.mlp.experts.gate_proj.scales": "model-00003-of-00003.safetensors",
406
- "model.layers.19.mlp.experts.gate_proj.weight": "model-00003-of-00003.safetensors",
407
- "model.layers.19.mlp.experts.up_proj.bias": "model-00003-of-00003.safetensors",
408
- "model.layers.19.mlp.experts.up_proj.biases": "model-00003-of-00003.safetensors",
409
- "model.layers.19.mlp.experts.up_proj.scales": "model-00003-of-00003.safetensors",
410
- "model.layers.19.mlp.experts.up_proj.weight": "model-00003-of-00003.safetensors",
411
- "model.layers.19.mlp.router.bias": "model-00003-of-00003.safetensors",
412
- "model.layers.19.mlp.router.biases": "model-00003-of-00003.safetensors",
413
- "model.layers.19.mlp.router.scales": "model-00003-of-00003.safetensors",
414
- "model.layers.19.mlp.router.weight": "model-00003-of-00003.safetensors",
415
- "model.layers.19.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
416
- "model.layers.19.self_attn.k_proj.bias": "model-00003-of-00003.safetensors",
417
- "model.layers.19.self_attn.k_proj.biases": "model-00003-of-00003.safetensors",
418
- "model.layers.19.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
419
- "model.layers.19.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
420
- "model.layers.19.self_attn.o_proj.bias": "model-00003-of-00003.safetensors",
421
- "model.layers.19.self_attn.o_proj.biases": "model-00003-of-00003.safetensors",
422
- "model.layers.19.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
423
- "model.layers.19.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
424
- "model.layers.19.self_attn.q_proj.bias": "model-00003-of-00003.safetensors",
425
- "model.layers.19.self_attn.q_proj.biases": "model-00003-of-00003.safetensors",
426
- "model.layers.19.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
427
- "model.layers.19.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
428
- "model.layers.19.self_attn.sinks": "model-00003-of-00003.safetensors",
429
- "model.layers.19.self_attn.v_proj.bias": "model-00003-of-00003.safetensors",
430
- "model.layers.19.self_attn.v_proj.biases": "model-00003-of-00003.safetensors",
431
- "model.layers.19.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
432
- "model.layers.19.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
433
  "model.layers.2.input_layernorm.weight": "model-00001-of-00003.safetensors",
434
  "model.layers.2.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
435
- "model.layers.2.mlp.experts.down_proj.biases": "model-00001-of-00003.safetensors",
436
  "model.layers.2.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
437
  "model.layers.2.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
438
  "model.layers.2.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
439
- "model.layers.2.mlp.experts.gate_proj.biases": "model-00001-of-00003.safetensors",
440
  "model.layers.2.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
441
  "model.layers.2.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
442
  "model.layers.2.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
443
- "model.layers.2.mlp.experts.up_proj.biases": "model-00001-of-00003.safetensors",
444
  "model.layers.2.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
445
  "model.layers.2.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
446
  "model.layers.2.mlp.router.bias": "model-00001-of-00003.safetensors",
@@ -465,122 +426,110 @@
465
  "model.layers.2.self_attn.v_proj.biases": "model-00001-of-00003.safetensors",
466
  "model.layers.2.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
467
  "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
468
- "model.layers.20.input_layernorm.weight": "model-00003-of-00003.safetensors",
469
- "model.layers.20.mlp.experts.down_proj.bias": "model-00003-of-00003.safetensors",
470
- "model.layers.20.mlp.experts.down_proj.biases": "model-00003-of-00003.safetensors",
471
- "model.layers.20.mlp.experts.down_proj.scales": "model-00003-of-00003.safetensors",
472
- "model.layers.20.mlp.experts.down_proj.weight": "model-00003-of-00003.safetensors",
473
- "model.layers.20.mlp.experts.gate_proj.bias": "model-00003-of-00003.safetensors",
474
- "model.layers.20.mlp.experts.gate_proj.biases": "model-00003-of-00003.safetensors",
475
- "model.layers.20.mlp.experts.gate_proj.scales": "model-00003-of-00003.safetensors",
476
- "model.layers.20.mlp.experts.gate_proj.weight": "model-00003-of-00003.safetensors",
477
- "model.layers.20.mlp.experts.up_proj.bias": "model-00003-of-00003.safetensors",
478
- "model.layers.20.mlp.experts.up_proj.biases": "model-00003-of-00003.safetensors",
479
- "model.layers.20.mlp.experts.up_proj.scales": "model-00003-of-00003.safetensors",
480
- "model.layers.20.mlp.experts.up_proj.weight": "model-00003-of-00003.safetensors",
481
- "model.layers.20.mlp.router.bias": "model-00003-of-00003.safetensors",
482
- "model.layers.20.mlp.router.biases": "model-00003-of-00003.safetensors",
483
- "model.layers.20.mlp.router.scales": "model-00003-of-00003.safetensors",
484
- "model.layers.20.mlp.router.weight": "model-00003-of-00003.safetensors",
485
- "model.layers.20.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
486
- "model.layers.20.self_attn.k_proj.bias": "model-00003-of-00003.safetensors",
487
- "model.layers.20.self_attn.k_proj.biases": "model-00003-of-00003.safetensors",
488
- "model.layers.20.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
489
- "model.layers.20.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
490
- "model.layers.20.self_attn.o_proj.bias": "model-00003-of-00003.safetensors",
491
- "model.layers.20.self_attn.o_proj.biases": "model-00003-of-00003.safetensors",
492
- "model.layers.20.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
493
- "model.layers.20.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
494
- "model.layers.20.self_attn.q_proj.bias": "model-00003-of-00003.safetensors",
495
- "model.layers.20.self_attn.q_proj.biases": "model-00003-of-00003.safetensors",
496
- "model.layers.20.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
497
- "model.layers.20.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
498
- "model.layers.20.self_attn.sinks": "model-00003-of-00003.safetensors",
499
- "model.layers.20.self_attn.v_proj.bias": "model-00003-of-00003.safetensors",
500
- "model.layers.20.self_attn.v_proj.biases": "model-00003-of-00003.safetensors",
501
- "model.layers.20.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
502
- "model.layers.20.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
503
- "model.layers.21.input_layernorm.weight": "model-00003-of-00003.safetensors",
504
- "model.layers.21.mlp.experts.down_proj.bias": "model-00003-of-00003.safetensors",
505
- "model.layers.21.mlp.experts.down_proj.biases": "model-00003-of-00003.safetensors",
506
- "model.layers.21.mlp.experts.down_proj.scales": "model-00003-of-00003.safetensors",
507
- "model.layers.21.mlp.experts.down_proj.weight": "model-00003-of-00003.safetensors",
508
- "model.layers.21.mlp.experts.gate_proj.bias": "model-00003-of-00003.safetensors",
509
- "model.layers.21.mlp.experts.gate_proj.biases": "model-00003-of-00003.safetensors",
510
- "model.layers.21.mlp.experts.gate_proj.scales": "model-00003-of-00003.safetensors",
511
- "model.layers.21.mlp.experts.gate_proj.weight": "model-00003-of-00003.safetensors",
512
- "model.layers.21.mlp.experts.up_proj.bias": "model-00003-of-00003.safetensors",
513
- "model.layers.21.mlp.experts.up_proj.biases": "model-00003-of-00003.safetensors",
514
- "model.layers.21.mlp.experts.up_proj.scales": "model-00003-of-00003.safetensors",
515
- "model.layers.21.mlp.experts.up_proj.weight": "model-00003-of-00003.safetensors",
516
- "model.layers.21.mlp.router.bias": "model-00003-of-00003.safetensors",
517
- "model.layers.21.mlp.router.biases": "model-00003-of-00003.safetensors",
518
- "model.layers.21.mlp.router.scales": "model-00003-of-00003.safetensors",
519
- "model.layers.21.mlp.router.weight": "model-00003-of-00003.safetensors",
520
- "model.layers.21.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
521
- "model.layers.21.self_attn.k_proj.bias": "model-00003-of-00003.safetensors",
522
- "model.layers.21.self_attn.k_proj.biases": "model-00003-of-00003.safetensors",
523
- "model.layers.21.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
524
- "model.layers.21.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
525
- "model.layers.21.self_attn.o_proj.bias": "model-00003-of-00003.safetensors",
526
- "model.layers.21.self_attn.o_proj.biases": "model-00003-of-00003.safetensors",
527
- "model.layers.21.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
528
- "model.layers.21.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
529
- "model.layers.21.self_attn.q_proj.bias": "model-00003-of-00003.safetensors",
530
- "model.layers.21.self_attn.q_proj.biases": "model-00003-of-00003.safetensors",
531
- "model.layers.21.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
532
- "model.layers.21.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
533
- "model.layers.21.self_attn.sinks": "model-00003-of-00003.safetensors",
534
- "model.layers.21.self_attn.v_proj.bias": "model-00003-of-00003.safetensors",
535
- "model.layers.21.self_attn.v_proj.biases": "model-00003-of-00003.safetensors",
536
- "model.layers.21.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
537
- "model.layers.21.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
538
- "model.layers.22.input_layernorm.weight": "model-00003-of-00003.safetensors",
539
- "model.layers.22.mlp.experts.down_proj.bias": "model-00003-of-00003.safetensors",
540
- "model.layers.22.mlp.experts.down_proj.biases": "model-00003-of-00003.safetensors",
541
- "model.layers.22.mlp.experts.down_proj.scales": "model-00003-of-00003.safetensors",
542
- "model.layers.22.mlp.experts.down_proj.weight": "model-00003-of-00003.safetensors",
543
- "model.layers.22.mlp.experts.gate_proj.bias": "model-00003-of-00003.safetensors",
544
- "model.layers.22.mlp.experts.gate_proj.biases": "model-00003-of-00003.safetensors",
545
- "model.layers.22.mlp.experts.gate_proj.scales": "model-00003-of-00003.safetensors",
546
- "model.layers.22.mlp.experts.gate_proj.weight": "model-00003-of-00003.safetensors",
547
- "model.layers.22.mlp.experts.up_proj.bias": "model-00003-of-00003.safetensors",
548
- "model.layers.22.mlp.experts.up_proj.biases": "model-00003-of-00003.safetensors",
549
- "model.layers.22.mlp.experts.up_proj.scales": "model-00003-of-00003.safetensors",
550
- "model.layers.22.mlp.experts.up_proj.weight": "model-00003-of-00003.safetensors",
551
- "model.layers.22.mlp.router.bias": "model-00003-of-00003.safetensors",
552
- "model.layers.22.mlp.router.biases": "model-00003-of-00003.safetensors",
553
- "model.layers.22.mlp.router.scales": "model-00003-of-00003.safetensors",
554
- "model.layers.22.mlp.router.weight": "model-00003-of-00003.safetensors",
555
- "model.layers.22.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
556
- "model.layers.22.self_attn.k_proj.bias": "model-00003-of-00003.safetensors",
557
- "model.layers.22.self_attn.k_proj.biases": "model-00003-of-00003.safetensors",
558
- "model.layers.22.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
559
- "model.layers.22.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
560
- "model.layers.22.self_attn.o_proj.bias": "model-00003-of-00003.safetensors",
561
- "model.layers.22.self_attn.o_proj.biases": "model-00003-of-00003.safetensors",
562
- "model.layers.22.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
563
- "model.layers.22.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
564
- "model.layers.22.self_attn.q_proj.bias": "model-00003-of-00003.safetensors",
565
- "model.layers.22.self_attn.q_proj.biases": "model-00003-of-00003.safetensors",
566
- "model.layers.22.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
567
- "model.layers.22.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
568
- "model.layers.22.self_attn.sinks": "model-00003-of-00003.safetensors",
569
- "model.layers.22.self_attn.v_proj.bias": "model-00003-of-00003.safetensors",
570
- "model.layers.22.self_attn.v_proj.biases": "model-00003-of-00003.safetensors",
571
- "model.layers.22.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
572
- "model.layers.22.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
573
  "model.layers.23.input_layernorm.weight": "model-00003-of-00003.safetensors",
574
  "model.layers.23.mlp.experts.down_proj.bias": "model-00003-of-00003.safetensors",
575
- "model.layers.23.mlp.experts.down_proj.biases": "model-00003-of-00003.safetensors",
576
  "model.layers.23.mlp.experts.down_proj.scales": "model-00003-of-00003.safetensors",
577
  "model.layers.23.mlp.experts.down_proj.weight": "model-00003-of-00003.safetensors",
578
  "model.layers.23.mlp.experts.gate_proj.bias": "model-00003-of-00003.safetensors",
579
- "model.layers.23.mlp.experts.gate_proj.biases": "model-00003-of-00003.safetensors",
580
  "model.layers.23.mlp.experts.gate_proj.scales": "model-00003-of-00003.safetensors",
581
  "model.layers.23.mlp.experts.gate_proj.weight": "model-00003-of-00003.safetensors",
582
  "model.layers.23.mlp.experts.up_proj.bias": "model-00003-of-00003.safetensors",
583
- "model.layers.23.mlp.experts.up_proj.biases": "model-00003-of-00003.safetensors",
584
  "model.layers.23.mlp.experts.up_proj.scales": "model-00003-of-00003.safetensors",
585
  "model.layers.23.mlp.experts.up_proj.weight": "model-00003-of-00003.safetensors",
586
  "model.layers.23.mlp.router.bias": "model-00003-of-00003.safetensors",
@@ -588,34 +537,31 @@
588
  "model.layers.23.mlp.router.scales": "model-00003-of-00003.safetensors",
589
  "model.layers.23.mlp.router.weight": "model-00003-of-00003.safetensors",
590
  "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
591
- "model.layers.23.self_attn.k_proj.bias": "model-00003-of-00003.safetensors",
592
- "model.layers.23.self_attn.k_proj.biases": "model-00003-of-00003.safetensors",
593
- "model.layers.23.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
594
- "model.layers.23.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
595
- "model.layers.23.self_attn.o_proj.bias": "model-00003-of-00003.safetensors",
596
- "model.layers.23.self_attn.o_proj.biases": "model-00003-of-00003.safetensors",
597
- "model.layers.23.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
598
- "model.layers.23.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
599
- "model.layers.23.self_attn.q_proj.bias": "model-00003-of-00003.safetensors",
600
- "model.layers.23.self_attn.q_proj.biases": "model-00003-of-00003.safetensors",
601
- "model.layers.23.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
602
- "model.layers.23.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
603
- "model.layers.23.self_attn.sinks": "model-00003-of-00003.safetensors",
604
- "model.layers.23.self_attn.v_proj.bias": "model-00003-of-00003.safetensors",
605
- "model.layers.23.self_attn.v_proj.biases": "model-00003-of-00003.safetensors",
606
- "model.layers.23.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
607
- "model.layers.23.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
608
  "model.layers.3.input_layernorm.weight": "model-00001-of-00003.safetensors",
609
  "model.layers.3.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
610
- "model.layers.3.mlp.experts.down_proj.biases": "model-00001-of-00003.safetensors",
611
  "model.layers.3.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
612
  "model.layers.3.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
613
  "model.layers.3.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
614
- "model.layers.3.mlp.experts.gate_proj.biases": "model-00001-of-00003.safetensors",
615
  "model.layers.3.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
616
  "model.layers.3.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
617
  "model.layers.3.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
618
- "model.layers.3.mlp.experts.up_proj.biases": "model-00001-of-00003.safetensors",
619
  "model.layers.3.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
620
  "model.layers.3.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
621
  "model.layers.3.mlp.router.bias": "model-00001-of-00003.safetensors",
@@ -642,15 +588,12 @@
642
  "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
643
  "model.layers.4.input_layernorm.weight": "model-00001-of-00003.safetensors",
644
  "model.layers.4.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
645
- "model.layers.4.mlp.experts.down_proj.biases": "model-00001-of-00003.safetensors",
646
  "model.layers.4.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
647
  "model.layers.4.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
648
  "model.layers.4.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
649
- "model.layers.4.mlp.experts.gate_proj.biases": "model-00001-of-00003.safetensors",
650
  "model.layers.4.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
651
  "model.layers.4.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
652
  "model.layers.4.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
653
- "model.layers.4.mlp.experts.up_proj.biases": "model-00001-of-00003.safetensors",
654
  "model.layers.4.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
655
  "model.layers.4.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
656
  "model.layers.4.mlp.router.bias": "model-00001-of-00003.safetensors",
@@ -677,15 +620,12 @@
677
  "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
678
  "model.layers.5.input_layernorm.weight": "model-00001-of-00003.safetensors",
679
  "model.layers.5.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
680
- "model.layers.5.mlp.experts.down_proj.biases": "model-00001-of-00003.safetensors",
681
  "model.layers.5.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
682
  "model.layers.5.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
683
  "model.layers.5.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
684
- "model.layers.5.mlp.experts.gate_proj.biases": "model-00001-of-00003.safetensors",
685
  "model.layers.5.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
686
  "model.layers.5.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
687
  "model.layers.5.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
688
- "model.layers.5.mlp.experts.up_proj.biases": "model-00001-of-00003.safetensors",
689
  "model.layers.5.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
690
  "model.layers.5.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
691
  "model.layers.5.mlp.router.bias": "model-00001-of-00003.safetensors",
@@ -712,15 +652,12 @@
712
  "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
713
  "model.layers.6.input_layernorm.weight": "model-00001-of-00003.safetensors",
714
  "model.layers.6.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
715
- "model.layers.6.mlp.experts.down_proj.biases": "model-00001-of-00003.safetensors",
716
  "model.layers.6.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
717
  "model.layers.6.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
718
  "model.layers.6.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
719
- "model.layers.6.mlp.experts.gate_proj.biases": "model-00001-of-00003.safetensors",
720
  "model.layers.6.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
721
  "model.layers.6.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
722
  "model.layers.6.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
723
- "model.layers.6.mlp.experts.up_proj.biases": "model-00001-of-00003.safetensors",
724
  "model.layers.6.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
725
  "model.layers.6.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
726
  "model.layers.6.mlp.router.bias": "model-00001-of-00003.safetensors",
@@ -747,15 +684,12 @@
747
  "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
748
  "model.layers.7.input_layernorm.weight": "model-00001-of-00003.safetensors",
749
  "model.layers.7.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
750
- "model.layers.7.mlp.experts.down_proj.biases": "model-00001-of-00003.safetensors",
751
  "model.layers.7.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
752
  "model.layers.7.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
753
  "model.layers.7.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
754
- "model.layers.7.mlp.experts.gate_proj.biases": "model-00001-of-00003.safetensors",
755
  "model.layers.7.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
756
  "model.layers.7.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
757
  "model.layers.7.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
758
- "model.layers.7.mlp.experts.up_proj.biases": "model-00001-of-00003.safetensors",
759
  "model.layers.7.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
760
  "model.layers.7.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
761
  "model.layers.7.mlp.router.bias": "model-00001-of-00003.safetensors",
@@ -780,24 +714,21 @@
780
  "model.layers.7.self_attn.v_proj.biases": "model-00001-of-00003.safetensors",
781
  "model.layers.7.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
782
  "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
783
- "model.layers.8.input_layernorm.weight": "model-00002-of-00003.safetensors",
784
- "model.layers.8.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
785
- "model.layers.8.mlp.experts.down_proj.biases": "model-00002-of-00003.safetensors",
786
- "model.layers.8.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
787
- "model.layers.8.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
788
  "model.layers.8.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
789
- "model.layers.8.mlp.experts.gate_proj.biases": "model-00001-of-00003.safetensors",
790
  "model.layers.8.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
791
  "model.layers.8.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
792
  "model.layers.8.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
793
- "model.layers.8.mlp.experts.up_proj.biases": "model-00001-of-00003.safetensors",
794
  "model.layers.8.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
795
  "model.layers.8.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
796
- "model.layers.8.mlp.router.bias": "model-00002-of-00003.safetensors",
797
- "model.layers.8.mlp.router.biases": "model-00002-of-00003.safetensors",
798
- "model.layers.8.mlp.router.scales": "model-00002-of-00003.safetensors",
799
- "model.layers.8.mlp.router.weight": "model-00002-of-00003.safetensors",
800
- "model.layers.8.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
801
  "model.layers.8.self_attn.k_proj.bias": "model-00001-of-00003.safetensors",
802
  "model.layers.8.self_attn.k_proj.biases": "model-00001-of-00003.safetensors",
803
  "model.layers.8.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
@@ -815,41 +746,38 @@
815
  "model.layers.8.self_attn.v_proj.biases": "model-00001-of-00003.safetensors",
816
  "model.layers.8.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
817
  "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
818
- "model.layers.9.input_layernorm.weight": "model-00002-of-00003.safetensors",
819
- "model.layers.9.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
820
- "model.layers.9.mlp.experts.down_proj.biases": "model-00002-of-00003.safetensors",
821
- "model.layers.9.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
822
- "model.layers.9.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
823
- "model.layers.9.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
824
- "model.layers.9.mlp.experts.gate_proj.biases": "model-00002-of-00003.safetensors",
825
- "model.layers.9.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
826
- "model.layers.9.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
827
- "model.layers.9.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
828
- "model.layers.9.mlp.experts.up_proj.biases": "model-00002-of-00003.safetensors",
829
- "model.layers.9.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
830
- "model.layers.9.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
831
- "model.layers.9.mlp.router.bias": "model-00002-of-00003.safetensors",
832
- "model.layers.9.mlp.router.biases": "model-00002-of-00003.safetensors",
833
- "model.layers.9.mlp.router.scales": "model-00002-of-00003.safetensors",
834
- "model.layers.9.mlp.router.weight": "model-00002-of-00003.safetensors",
835
- "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
836
- "model.layers.9.self_attn.k_proj.bias": "model-00002-of-00003.safetensors",
837
- "model.layers.9.self_attn.k_proj.biases": "model-00002-of-00003.safetensors",
838
- "model.layers.9.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
839
- "model.layers.9.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
840
- "model.layers.9.self_attn.o_proj.bias": "model-00002-of-00003.safetensors",
841
- "model.layers.9.self_attn.o_proj.biases": "model-00002-of-00003.safetensors",
842
- "model.layers.9.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
843
- "model.layers.9.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
844
- "model.layers.9.self_attn.q_proj.bias": "model-00002-of-00003.safetensors",
845
- "model.layers.9.self_attn.q_proj.biases": "model-00002-of-00003.safetensors",
846
- "model.layers.9.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
847
- "model.layers.9.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
848
- "model.layers.9.self_attn.sinks": "model-00002-of-00003.safetensors",
849
- "model.layers.9.self_attn.v_proj.bias": "model-00002-of-00003.safetensors",
850
- "model.layers.9.self_attn.v_proj.biases": "model-00002-of-00003.safetensors",
851
- "model.layers.9.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
852
- "model.layers.9.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
853
  "model.norm.weight": "model-00001-of-00003.safetensors"
854
  }
855
  }
 
1
  {
2
  "metadata": {
3
+ "total_size": 11402890368,
4
  "total_parameters": 20914755648
5
  },
6
  "weight_map": {
 
12
  "model.embed_tokens.weight": "model-00001-of-00003.safetensors",
13
  "model.layers.0.input_layernorm.weight": "model-00001-of-00003.safetensors",
14
  "model.layers.0.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
 
15
  "model.layers.0.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
16
  "model.layers.0.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
17
  "model.layers.0.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
 
18
  "model.layers.0.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
19
  "model.layers.0.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
20
  "model.layers.0.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
 
21
  "model.layers.0.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
22
  "model.layers.0.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
23
  "model.layers.0.mlp.router.bias": "model-00001-of-00003.safetensors",
 
44
  "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
45
  "model.layers.1.input_layernorm.weight": "model-00001-of-00003.safetensors",
46
  "model.layers.1.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
 
47
  "model.layers.1.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
48
  "model.layers.1.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
49
  "model.layers.1.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
 
50
  "model.layers.1.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
51
  "model.layers.1.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
52
  "model.layers.1.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
 
53
  "model.layers.1.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
54
  "model.layers.1.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
55
  "model.layers.1.mlp.router.bias": "model-00001-of-00003.safetensors",
 
74
  "model.layers.1.self_attn.v_proj.biases": "model-00001-of-00003.safetensors",
75
  "model.layers.1.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
76
  "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
77
+ "model.layers.10.input_layernorm.weight": "model-00001-of-00003.safetensors",
78
+ "model.layers.10.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
79
+ "model.layers.10.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
80
+ "model.layers.10.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
81
+ "model.layers.10.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
82
+ "model.layers.10.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
83
+ "model.layers.10.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
84
+ "model.layers.10.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
85
+ "model.layers.10.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
86
+ "model.layers.10.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
87
+ "model.layers.10.mlp.router.bias": "model-00001-of-00003.safetensors",
88
+ "model.layers.10.mlp.router.biases": "model-00001-of-00003.safetensors",
89
+ "model.layers.10.mlp.router.scales": "model-00001-of-00003.safetensors",
90
+ "model.layers.10.mlp.router.weight": "model-00001-of-00003.safetensors",
91
+ "model.layers.10.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
92
+ "model.layers.10.self_attn.k_proj.bias": "model-00001-of-00003.safetensors",
93
+ "model.layers.10.self_attn.k_proj.biases": "model-00001-of-00003.safetensors",
94
+ "model.layers.10.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
95
+ "model.layers.10.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
96
+ "model.layers.10.self_attn.o_proj.bias": "model-00001-of-00003.safetensors",
97
+ "model.layers.10.self_attn.o_proj.biases": "model-00001-of-00003.safetensors",
98
+ "model.layers.10.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
99
+ "model.layers.10.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
100
+ "model.layers.10.self_attn.q_proj.bias": "model-00001-of-00003.safetensors",
101
+ "model.layers.10.self_attn.q_proj.biases": "model-00001-of-00003.safetensors",
102
+ "model.layers.10.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
103
+ "model.layers.10.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
104
+ "model.layers.10.self_attn.sinks": "model-00001-of-00003.safetensors",
105
+ "model.layers.10.self_attn.v_proj.bias": "model-00001-of-00003.safetensors",
106
+ "model.layers.10.self_attn.v_proj.biases": "model-00001-of-00003.safetensors",
107
+ "model.layers.10.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
108
+ "model.layers.10.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
 
 
 
109
  "model.layers.11.input_layernorm.weight": "model-00002-of-00003.safetensors",
110
  "model.layers.11.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
 
111
  "model.layers.11.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
112
  "model.layers.11.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
113
  "model.layers.11.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
 
114
  "model.layers.11.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
115
  "model.layers.11.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
116
  "model.layers.11.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
 
117
  "model.layers.11.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
118
  "model.layers.11.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
119
  "model.layers.11.mlp.router.bias": "model-00002-of-00003.safetensors",
 
121
  "model.layers.11.mlp.router.scales": "model-00002-of-00003.safetensors",
122
  "model.layers.11.mlp.router.weight": "model-00002-of-00003.safetensors",
123
  "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
124
+ "model.layers.11.self_attn.k_proj.bias": "model-00001-of-00003.safetensors",
125
+ "model.layers.11.self_attn.k_proj.biases": "model-00001-of-00003.safetensors",
126
+ "model.layers.11.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
127
+ "model.layers.11.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
128
+ "model.layers.11.self_attn.o_proj.bias": "model-00001-of-00003.safetensors",
129
+ "model.layers.11.self_attn.o_proj.biases": "model-00001-of-00003.safetensors",
130
+ "model.layers.11.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
131
+ "model.layers.11.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
132
+ "model.layers.11.self_attn.q_proj.bias": "model-00001-of-00003.safetensors",
133
+ "model.layers.11.self_attn.q_proj.biases": "model-00001-of-00003.safetensors",
134
+ "model.layers.11.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
135
+ "model.layers.11.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
136
+ "model.layers.11.self_attn.sinks": "model-00001-of-00003.safetensors",
137
+ "model.layers.11.self_attn.v_proj.bias": "model-00001-of-00003.safetensors",
138
+ "model.layers.11.self_attn.v_proj.biases": "model-00001-of-00003.safetensors",
139
+ "model.layers.11.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
140
+ "model.layers.11.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
141
  "model.layers.12.input_layernorm.weight": "model-00002-of-00003.safetensors",
142
  "model.layers.12.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
 
143
  "model.layers.12.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
144
  "model.layers.12.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
145
  "model.layers.12.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
 
146
  "model.layers.12.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
147
  "model.layers.12.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
148
  "model.layers.12.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
 
149
  "model.layers.12.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
150
  "model.layers.12.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
151
  "model.layers.12.mlp.router.bias": "model-00002-of-00003.safetensors",
 
172
  "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
173
  "model.layers.13.input_layernorm.weight": "model-00002-of-00003.safetensors",
174
  "model.layers.13.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
 
175
  "model.layers.13.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
176
  "model.layers.13.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
177
  "model.layers.13.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
 
178
  "model.layers.13.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
179
  "model.layers.13.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
180
  "model.layers.13.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
 
181
  "model.layers.13.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
182
  "model.layers.13.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
183
  "model.layers.13.mlp.router.bias": "model-00002-of-00003.safetensors",
 
204
  "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
205
  "model.layers.14.input_layernorm.weight": "model-00002-of-00003.safetensors",
206
  "model.layers.14.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
 
207
  "model.layers.14.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
208
  "model.layers.14.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
209
  "model.layers.14.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
 
210
  "model.layers.14.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
211
  "model.layers.14.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
212
  "model.layers.14.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
 
213
  "model.layers.14.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
214
  "model.layers.14.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
215
  "model.layers.14.mlp.router.bias": "model-00002-of-00003.safetensors",
 
236
  "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
237
  "model.layers.15.input_layernorm.weight": "model-00002-of-00003.safetensors",
238
  "model.layers.15.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
 
239
  "model.layers.15.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
240
  "model.layers.15.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
241
  "model.layers.15.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
 
242
  "model.layers.15.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
243
  "model.layers.15.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
244
  "model.layers.15.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
 
245
  "model.layers.15.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
246
  "model.layers.15.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
247
  "model.layers.15.mlp.router.bias": "model-00002-of-00003.safetensors",
 
268
  "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
269
  "model.layers.16.input_layernorm.weight": "model-00002-of-00003.safetensors",
270
  "model.layers.16.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
 
271
  "model.layers.16.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
272
  "model.layers.16.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
273
  "model.layers.16.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
 
274
  "model.layers.16.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
275
  "model.layers.16.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
276
  "model.layers.16.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
 
277
  "model.layers.16.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
278
  "model.layers.16.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
279
  "model.layers.16.mlp.router.bias": "model-00002-of-00003.safetensors",
 
300
  "model.layers.16.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
301
  "model.layers.17.input_layernorm.weight": "model-00002-of-00003.safetensors",
302
  "model.layers.17.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
 
303
  "model.layers.17.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
304
  "model.layers.17.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
305
  "model.layers.17.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
 
306
  "model.layers.17.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
307
  "model.layers.17.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
308
  "model.layers.17.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
 
309
  "model.layers.17.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
310
  "model.layers.17.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
311
  "model.layers.17.mlp.router.bias": "model-00002-of-00003.safetensors",
 
330
  "model.layers.17.self_attn.v_proj.biases": "model-00002-of-00003.safetensors",
331
  "model.layers.17.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
332
  "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
333
+ "model.layers.18.input_layernorm.weight": "model-00002-of-00003.safetensors",
334
+ "model.layers.18.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
335
+ "model.layers.18.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
336
+ "model.layers.18.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
337
+ "model.layers.18.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
338
+ "model.layers.18.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
339
+ "model.layers.18.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
340
+ "model.layers.18.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
341
+ "model.layers.18.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
342
+ "model.layers.18.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
343
+ "model.layers.18.mlp.router.bias": "model-00002-of-00003.safetensors",
344
+ "model.layers.18.mlp.router.biases": "model-00002-of-00003.safetensors",
345
+ "model.layers.18.mlp.router.scales": "model-00002-of-00003.safetensors",
346
+ "model.layers.18.mlp.router.weight": "model-00002-of-00003.safetensors",
347
+ "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
 
 
 
348
  "model.layers.18.self_attn.k_proj.bias": "model-00002-of-00003.safetensors",
349
  "model.layers.18.self_attn.k_proj.biases": "model-00002-of-00003.safetensors",
350
  "model.layers.18.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
 
362
  "model.layers.18.self_attn.v_proj.biases": "model-00002-of-00003.safetensors",
363
  "model.layers.18.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
364
  "model.layers.18.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
365
+ "model.layers.19.input_layernorm.weight": "model-00002-of-00003.safetensors",
366
+ "model.layers.19.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
367
+ "model.layers.19.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
368
+ "model.layers.19.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
369
+ "model.layers.19.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
370
+ "model.layers.19.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
371
+ "model.layers.19.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
372
+ "model.layers.19.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
373
+ "model.layers.19.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
374
+ "model.layers.19.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
375
+ "model.layers.19.mlp.router.bias": "model-00002-of-00003.safetensors",
376
+ "model.layers.19.mlp.router.biases": "model-00002-of-00003.safetensors",
377
+ "model.layers.19.mlp.router.scales": "model-00002-of-00003.safetensors",
378
+ "model.layers.19.mlp.router.weight": "model-00002-of-00003.safetensors",
379
+ "model.layers.19.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
380
+ "model.layers.19.self_attn.k_proj.bias": "model-00002-of-00003.safetensors",
381
+ "model.layers.19.self_attn.k_proj.biases": "model-00002-of-00003.safetensors",
382
+ "model.layers.19.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
383
+ "model.layers.19.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
384
+ "model.layers.19.self_attn.o_proj.bias": "model-00002-of-00003.safetensors",
385
+ "model.layers.19.self_attn.o_proj.biases": "model-00002-of-00003.safetensors",
386
+ "model.layers.19.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
387
+ "model.layers.19.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
388
+ "model.layers.19.self_attn.q_proj.bias": "model-00002-of-00003.safetensors",
389
+ "model.layers.19.self_attn.q_proj.biases": "model-00002-of-00003.safetensors",
390
+ "model.layers.19.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
391
+ "model.layers.19.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
392
+ "model.layers.19.self_attn.sinks": "model-00002-of-00003.safetensors",
393
+ "model.layers.19.self_attn.v_proj.bias": "model-00002-of-00003.safetensors",
394
+ "model.layers.19.self_attn.v_proj.biases": "model-00002-of-00003.safetensors",
395
+ "model.layers.19.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
396
+ "model.layers.19.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
 
 
 
397
  "model.layers.2.input_layernorm.weight": "model-00001-of-00003.safetensors",
398
  "model.layers.2.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
 
399
  "model.layers.2.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
400
  "model.layers.2.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
401
  "model.layers.2.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
 
402
  "model.layers.2.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
403
  "model.layers.2.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
404
  "model.layers.2.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
 
405
  "model.layers.2.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
406
  "model.layers.2.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
407
  "model.layers.2.mlp.router.bias": "model-00001-of-00003.safetensors",
 
426
  "model.layers.2.self_attn.v_proj.biases": "model-00001-of-00003.safetensors",
427
  "model.layers.2.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
428
  "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
429
+ "model.layers.20.input_layernorm.weight": "model-00002-of-00003.safetensors",
430
+ "model.layers.20.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
431
+ "model.layers.20.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
432
+ "model.layers.20.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
433
+ "model.layers.20.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
434
+ "model.layers.20.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
435
+ "model.layers.20.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
436
+ "model.layers.20.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
437
+ "model.layers.20.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
438
+ "model.layers.20.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
439
+ "model.layers.20.mlp.router.bias": "model-00002-of-00003.safetensors",
440
+ "model.layers.20.mlp.router.biases": "model-00002-of-00003.safetensors",
441
+ "model.layers.20.mlp.router.scales": "model-00002-of-00003.safetensors",
442
+ "model.layers.20.mlp.router.weight": "model-00002-of-00003.safetensors",
443
+ "model.layers.20.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
444
+ "model.layers.20.self_attn.k_proj.bias": "model-00002-of-00003.safetensors",
445
+ "model.layers.20.self_attn.k_proj.biases": "model-00002-of-00003.safetensors",
446
+ "model.layers.20.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
447
+ "model.layers.20.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
448
+ "model.layers.20.self_attn.o_proj.bias": "model-00002-of-00003.safetensors",
449
+ "model.layers.20.self_attn.o_proj.biases": "model-00002-of-00003.safetensors",
450
+ "model.layers.20.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
451
+ "model.layers.20.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
452
+ "model.layers.20.self_attn.q_proj.bias": "model-00002-of-00003.safetensors",
453
+ "model.layers.20.self_attn.q_proj.biases": "model-00002-of-00003.safetensors",
454
+ "model.layers.20.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
455
+ "model.layers.20.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
456
+ "model.layers.20.self_attn.sinks": "model-00002-of-00003.safetensors",
457
+ "model.layers.20.self_attn.v_proj.bias": "model-00002-of-00003.safetensors",
458
+ "model.layers.20.self_attn.v_proj.biases": "model-00002-of-00003.safetensors",
459
+ "model.layers.20.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
460
+ "model.layers.20.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
461
+ "model.layers.21.input_layernorm.weight": "model-00002-of-00003.safetensors",
462
+ "model.layers.21.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
463
+ "model.layers.21.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
464
+ "model.layers.21.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
465
+ "model.layers.21.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
466
+ "model.layers.21.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
467
+ "model.layers.21.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
468
+ "model.layers.21.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
469
+ "model.layers.21.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
470
+ "model.layers.21.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
471
+ "model.layers.21.mlp.router.bias": "model-00002-of-00003.safetensors",
472
+ "model.layers.21.mlp.router.biases": "model-00002-of-00003.safetensors",
473
+ "model.layers.21.mlp.router.scales": "model-00002-of-00003.safetensors",
474
+ "model.layers.21.mlp.router.weight": "model-00002-of-00003.safetensors",
475
+ "model.layers.21.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
476
+ "model.layers.21.self_attn.k_proj.bias": "model-00002-of-00003.safetensors",
477
+ "model.layers.21.self_attn.k_proj.biases": "model-00002-of-00003.safetensors",
478
+ "model.layers.21.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
479
+ "model.layers.21.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
480
+ "model.layers.21.self_attn.o_proj.bias": "model-00002-of-00003.safetensors",
481
+ "model.layers.21.self_attn.o_proj.biases": "model-00002-of-00003.safetensors",
482
+ "model.layers.21.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
483
+ "model.layers.21.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
484
+ "model.layers.21.self_attn.q_proj.bias": "model-00002-of-00003.safetensors",
485
+ "model.layers.21.self_attn.q_proj.biases": "model-00002-of-00003.safetensors",
486
+ "model.layers.21.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
487
+ "model.layers.21.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
488
+ "model.layers.21.self_attn.sinks": "model-00002-of-00003.safetensors",
489
+ "model.layers.21.self_attn.v_proj.bias": "model-00002-of-00003.safetensors",
490
+ "model.layers.21.self_attn.v_proj.biases": "model-00002-of-00003.safetensors",
491
+ "model.layers.21.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
492
+ "model.layers.21.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
493
+ "model.layers.22.input_layernorm.weight": "model-00002-of-00003.safetensors",
494
+ "model.layers.22.mlp.experts.down_proj.bias": "model-00002-of-00003.safetensors",
495
+ "model.layers.22.mlp.experts.down_proj.scales": "model-00002-of-00003.safetensors",
496
+ "model.layers.22.mlp.experts.down_proj.weight": "model-00002-of-00003.safetensors",
497
+ "model.layers.22.mlp.experts.gate_proj.bias": "model-00002-of-00003.safetensors",
498
+ "model.layers.22.mlp.experts.gate_proj.scales": "model-00002-of-00003.safetensors",
499
+ "model.layers.22.mlp.experts.gate_proj.weight": "model-00002-of-00003.safetensors",
500
+ "model.layers.22.mlp.experts.up_proj.bias": "model-00002-of-00003.safetensors",
501
+ "model.layers.22.mlp.experts.up_proj.scales": "model-00002-of-00003.safetensors",
502
+ "model.layers.22.mlp.experts.up_proj.weight": "model-00002-of-00003.safetensors",
503
+ "model.layers.22.mlp.router.bias": "model-00002-of-00003.safetensors",
504
+ "model.layers.22.mlp.router.biases": "model-00002-of-00003.safetensors",
505
+ "model.layers.22.mlp.router.scales": "model-00002-of-00003.safetensors",
506
+ "model.layers.22.mlp.router.weight": "model-00002-of-00003.safetensors",
507
+ "model.layers.22.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
508
+ "model.layers.22.self_attn.k_proj.bias": "model-00002-of-00003.safetensors",
509
+ "model.layers.22.self_attn.k_proj.biases": "model-00002-of-00003.safetensors",
510
+ "model.layers.22.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
511
+ "model.layers.22.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
512
+ "model.layers.22.self_attn.o_proj.bias": "model-00002-of-00003.safetensors",
513
+ "model.layers.22.self_attn.o_proj.biases": "model-00002-of-00003.safetensors",
514
+ "model.layers.22.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
515
+ "model.layers.22.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
516
+ "model.layers.22.self_attn.q_proj.bias": "model-00002-of-00003.safetensors",
517
+ "model.layers.22.self_attn.q_proj.biases": "model-00002-of-00003.safetensors",
518
+ "model.layers.22.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
519
+ "model.layers.22.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
520
+ "model.layers.22.self_attn.sinks": "model-00002-of-00003.safetensors",
521
+ "model.layers.22.self_attn.v_proj.bias": "model-00002-of-00003.safetensors",
522
+ "model.layers.22.self_attn.v_proj.biases": "model-00002-of-00003.safetensors",
523
+ "model.layers.22.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
524
+ "model.layers.22.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
 
 
 
 
 
 
 
 
 
525
  "model.layers.23.input_layernorm.weight": "model-00003-of-00003.safetensors",
526
  "model.layers.23.mlp.experts.down_proj.bias": "model-00003-of-00003.safetensors",
 
527
  "model.layers.23.mlp.experts.down_proj.scales": "model-00003-of-00003.safetensors",
528
  "model.layers.23.mlp.experts.down_proj.weight": "model-00003-of-00003.safetensors",
529
  "model.layers.23.mlp.experts.gate_proj.bias": "model-00003-of-00003.safetensors",
 
530
  "model.layers.23.mlp.experts.gate_proj.scales": "model-00003-of-00003.safetensors",
531
  "model.layers.23.mlp.experts.gate_proj.weight": "model-00003-of-00003.safetensors",
532
  "model.layers.23.mlp.experts.up_proj.bias": "model-00003-of-00003.safetensors",
 
533
  "model.layers.23.mlp.experts.up_proj.scales": "model-00003-of-00003.safetensors",
534
  "model.layers.23.mlp.experts.up_proj.weight": "model-00003-of-00003.safetensors",
535
  "model.layers.23.mlp.router.bias": "model-00003-of-00003.safetensors",
 
537
  "model.layers.23.mlp.router.scales": "model-00003-of-00003.safetensors",
538
  "model.layers.23.mlp.router.weight": "model-00003-of-00003.safetensors",
539
  "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
540
+ "model.layers.23.self_attn.k_proj.bias": "model-00002-of-00003.safetensors",
541
+ "model.layers.23.self_attn.k_proj.biases": "model-00002-of-00003.safetensors",
542
+ "model.layers.23.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
543
+ "model.layers.23.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
544
+ "model.layers.23.self_attn.o_proj.bias": "model-00002-of-00003.safetensors",
545
+ "model.layers.23.self_attn.o_proj.biases": "model-00002-of-00003.safetensors",
546
+ "model.layers.23.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
547
+ "model.layers.23.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
548
+ "model.layers.23.self_attn.q_proj.bias": "model-00002-of-00003.safetensors",
549
+ "model.layers.23.self_attn.q_proj.biases": "model-00002-of-00003.safetensors",
550
+ "model.layers.23.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
551
+ "model.layers.23.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
552
+ "model.layers.23.self_attn.sinks": "model-00002-of-00003.safetensors",
553
+ "model.layers.23.self_attn.v_proj.bias": "model-00002-of-00003.safetensors",
554
+ "model.layers.23.self_attn.v_proj.biases": "model-00002-of-00003.safetensors",
555
+ "model.layers.23.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
556
+ "model.layers.23.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
557
  "model.layers.3.input_layernorm.weight": "model-00001-of-00003.safetensors",
558
  "model.layers.3.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
 
559
  "model.layers.3.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
560
  "model.layers.3.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
561
  "model.layers.3.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
 
562
  "model.layers.3.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
563
  "model.layers.3.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
564
  "model.layers.3.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
 
565
  "model.layers.3.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
566
  "model.layers.3.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
567
  "model.layers.3.mlp.router.bias": "model-00001-of-00003.safetensors",
 
588
  "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
589
  "model.layers.4.input_layernorm.weight": "model-00001-of-00003.safetensors",
590
  "model.layers.4.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
 
591
  "model.layers.4.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
592
  "model.layers.4.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
593
  "model.layers.4.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
 
594
  "model.layers.4.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
595
  "model.layers.4.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
596
  "model.layers.4.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
 
597
  "model.layers.4.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
598
  "model.layers.4.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
599
  "model.layers.4.mlp.router.bias": "model-00001-of-00003.safetensors",
 
620
  "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
621
  "model.layers.5.input_layernorm.weight": "model-00001-of-00003.safetensors",
622
  "model.layers.5.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
 
623
  "model.layers.5.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
624
  "model.layers.5.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
625
  "model.layers.5.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
 
626
  "model.layers.5.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
627
  "model.layers.5.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
628
  "model.layers.5.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
 
629
  "model.layers.5.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
630
  "model.layers.5.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
631
  "model.layers.5.mlp.router.bias": "model-00001-of-00003.safetensors",
 
652
  "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
653
  "model.layers.6.input_layernorm.weight": "model-00001-of-00003.safetensors",
654
  "model.layers.6.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
 
655
  "model.layers.6.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
656
  "model.layers.6.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
657
  "model.layers.6.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
 
658
  "model.layers.6.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
659
  "model.layers.6.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
660
  "model.layers.6.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
 
661
  "model.layers.6.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
662
  "model.layers.6.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
663
  "model.layers.6.mlp.router.bias": "model-00001-of-00003.safetensors",
 
684
  "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
685
  "model.layers.7.input_layernorm.weight": "model-00001-of-00003.safetensors",
686
  "model.layers.7.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
 
687
  "model.layers.7.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
688
  "model.layers.7.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
689
  "model.layers.7.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
 
690
  "model.layers.7.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
691
  "model.layers.7.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
692
  "model.layers.7.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
 
693
  "model.layers.7.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
694
  "model.layers.7.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
695
  "model.layers.7.mlp.router.bias": "model-00001-of-00003.safetensors",
 
714
  "model.layers.7.self_attn.v_proj.biases": "model-00001-of-00003.safetensors",
715
  "model.layers.7.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
716
  "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
717
+ "model.layers.8.input_layernorm.weight": "model-00001-of-00003.safetensors",
718
+ "model.layers.8.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
719
+ "model.layers.8.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
720
+ "model.layers.8.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
 
721
  "model.layers.8.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
 
722
  "model.layers.8.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
723
  "model.layers.8.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
724
  "model.layers.8.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
 
725
  "model.layers.8.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
726
  "model.layers.8.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
727
+ "model.layers.8.mlp.router.bias": "model-00001-of-00003.safetensors",
728
+ "model.layers.8.mlp.router.biases": "model-00001-of-00003.safetensors",
729
+ "model.layers.8.mlp.router.scales": "model-00001-of-00003.safetensors",
730
+ "model.layers.8.mlp.router.weight": "model-00001-of-00003.safetensors",
731
+ "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
732
  "model.layers.8.self_attn.k_proj.bias": "model-00001-of-00003.safetensors",
733
  "model.layers.8.self_attn.k_proj.biases": "model-00001-of-00003.safetensors",
734
  "model.layers.8.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
 
746
  "model.layers.8.self_attn.v_proj.biases": "model-00001-of-00003.safetensors",
747
  "model.layers.8.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
748
  "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
749
+ "model.layers.9.input_layernorm.weight": "model-00001-of-00003.safetensors",
750
+ "model.layers.9.mlp.experts.down_proj.bias": "model-00001-of-00003.safetensors",
751
+ "model.layers.9.mlp.experts.down_proj.scales": "model-00001-of-00003.safetensors",
752
+ "model.layers.9.mlp.experts.down_proj.weight": "model-00001-of-00003.safetensors",
753
+ "model.layers.9.mlp.experts.gate_proj.bias": "model-00001-of-00003.safetensors",
754
+ "model.layers.9.mlp.experts.gate_proj.scales": "model-00001-of-00003.safetensors",
755
+ "model.layers.9.mlp.experts.gate_proj.weight": "model-00001-of-00003.safetensors",
756
+ "model.layers.9.mlp.experts.up_proj.bias": "model-00001-of-00003.safetensors",
757
+ "model.layers.9.mlp.experts.up_proj.scales": "model-00001-of-00003.safetensors",
758
+ "model.layers.9.mlp.experts.up_proj.weight": "model-00001-of-00003.safetensors",
759
+ "model.layers.9.mlp.router.bias": "model-00001-of-00003.safetensors",
760
+ "model.layers.9.mlp.router.biases": "model-00001-of-00003.safetensors",
761
+ "model.layers.9.mlp.router.scales": "model-00001-of-00003.safetensors",
762
+ "model.layers.9.mlp.router.weight": "model-00001-of-00003.safetensors",
763
+ "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
764
+ "model.layers.9.self_attn.k_proj.bias": "model-00001-of-00003.safetensors",
765
+ "model.layers.9.self_attn.k_proj.biases": "model-00001-of-00003.safetensors",
766
+ "model.layers.9.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
767
+ "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
768
+ "model.layers.9.self_attn.o_proj.bias": "model-00001-of-00003.safetensors",
769
+ "model.layers.9.self_attn.o_proj.biases": "model-00001-of-00003.safetensors",
770
+ "model.layers.9.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
771
+ "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
772
+ "model.layers.9.self_attn.q_proj.bias": "model-00001-of-00003.safetensors",
773
+ "model.layers.9.self_attn.q_proj.biases": "model-00001-of-00003.safetensors",
774
+ "model.layers.9.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
775
+ "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
776
+ "model.layers.9.self_attn.sinks": "model-00001-of-00003.safetensors",
777
+ "model.layers.9.self_attn.v_proj.bias": "model-00001-of-00003.safetensors",
778
+ "model.layers.9.self_attn.v_proj.biases": "model-00001-of-00003.safetensors",
779
+ "model.layers.9.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
780
+ "model.layers.9.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
 
 
 
781
  "model.norm.weight": "model-00001-of-00003.safetensors"
782
  }
783
  }