Hugging Face's logo

Alibaba-NLP
/
gte-modernbert-base

Sentence Similarity
Transformers
PyTorch
ONNX
Safetensors
sentence-transformers
Transformers.js
English
modernbert
feature-extraction
mteb
embedding
text-embeddings-inference
Model card Files
xet
 Community
17

COIR Repro

#3
by bclavie - opened
Discussion
bclavie

Hi there!

First of all, thank you for doing this -- we're big fans of the GTE family and it was very exciting seeing this model pop up!

I have one question about the CoIR results: would you be able to share the code you used to get them? I'm struggling to reproduce them 1:1, but they're very excited, so I'm curious what I'm doing wrong at the moment!

Thanks a lot
zyznull
 Alibaba-NLP org

You can use the office mteb code to reproduce the our result on CoIR.

bclavie

Thanks for the super quick response!

I'm indeed getting the same results as yours when using the mteb code. What's really strange is that the results are very different from the ones using the coir package, even though the CoIR team presents both as interchangeable :think:

I'll flag it in their repo and try to figure out what's going on. Thank you!

Sign up or log in to comment