stephantulkens

/

NIFE-gte-modernbert-base

like 4

Hey,

I don't mind doing that but I also think those libraries might be better if they don't require a config. This is the standard output of a sentence-transformers trainer, so if the libraries require a config any model trained with sentence-transformers would not be loadable with that library.

Let me check.

EDIT: after checking I realized this model can't be loaded at all using model2vec-rs. It only supports the model2vec format. So adding the config also would not have helped. One thing you can do is load this model using the model2vec python library (using from_sentence_transformers) and then save it as a model2vec model.

Thanks for investigating! Will give it a spin next week as a workaround.

Maybe @Pringled : do you think it would be technically possible to import said sentence transformers output directly in model2vec-rs?

Hey, I think as is NIFE models won't work in either model2vec or model2vec-rs. The StaticModel.from_sentence_transformers() loading method in Model2Vec requires a different format (the one used in the static models released by Tom Aarsen, e.g. https://huggingface.co/sentence-transformers/static-similarity-mrl-multilingual-v1/tree/main). Model2vec-rs expects the Model2Vec format, in which the model.safetensors file has an embeddings attribute, but for NIFE models it's embedding.weight I believe.

Tbh I think the easiest solution would be to make the NIFE models Model2Vec compatible (I believe that means adding a config.json and changing the model.safetensors to the format described above), since Model2Vec models are fully compatible with SentenceTransformers. AFAIK a config.json is also pretty standard, for example you need one to track downloads.

I will think a bit about how we load in Model2Vec (and Model2Vec-rs) as well; I can have a look at whether it can be made fully compatible with Sentence Transformers (and thus NIFE) instead since a unified method for loading might be nicer anyway, but if you want to try it out sooner your best bet might be to just load a NIFE model in Model2Vec and save it. To do that I think you can:

• Change https://github.com/MinishLab/model2vec/blob/main/model2vec/hf_utils.py#L140 to:

    if from_sentence_transformers:
        model_file = "model.safetensors"
        tokenizer_file = "tokenizer.json"
        config_name = "config_sentence_transformers.json"

• And then run something like:

from model2vec import StaticModel

model = StaticModel.from_sentence_transformers("stephantulkens/NIFE-mxbai-embed-large-v1")
model.save_pretrained("local/models/NIFE-mxbai-embed-large-v1-m2v")

Which will give you a Model2Vec (and Model2Vec-rs) compatible model.

Super interesting, thanks to both of you and please keep me in the loop! I'll ping you when I start experimenting.