How did you train SparseEmbed?

[richardklafter]

First, awesome project!

How did you train your model at https://huggingface.co/raphaelsty/neural-cherche-sparse-embed? Did you train it from scratch? I found an old copy of your sparsembed library. Was that library used or was this repository? What data did your train on exactly?

I am surveying various sparse embedding models and SparseEmbed while interesting has very little code or docs beyond the original google paper. Any assistance would be appreciated. Thanks!

[raphaelsty]

Hi @richardklafter, I used the msmarco dataset, trained using neural-search without negative samples. The checkpoint is either a distilbert or a co-condenser, don't remember. I trained the model with a single GPU on google colab, I wasn't aiming for extraordinary accuracy. I'm sure it's easy to do better.

[richardklafter]

Thanks for letting me know! Feel free to close this.

But, if you want to give me more specifics, a notebook or something, I would happily run it on an H100 and give you results. I am curious where this lands given I could find very few public implementations.