Investing in Performance: Fine-tune small models with LLM insights - a CFM case study
โข
25
Florent Gbelidji
florentgbelidji
AI & ML interests
None yet
Recent Activity
updated
a Space
4 days ago
florentgbelidji/test_folium
reacted
to
MoritzLaurer's
post
with ๐ฅ
5 days ago
๐ Releasing a new zeroshot-classifier based on ModernBERT! Some key takeaways:
- โก Speed & efficiency: It's multiple times faster and uses significantly less memory than DeBERTav3. You can use larger batch sizes and enabling bf16 (instead of fp16) gave me a ~2x speed boost as well
- ๐ Performance tradeoff: It performs slightly worse than DeBERTav3 on average across my zeroshot classification task collection
- ๐ง Use cases: I recommend using it for scenarios requiring speed and a larger context window (8k).
- ๐ก Whatโs next? Iโm preparing a newer version trained on better + longer synthetic data to fully leverage the 8k context window and improve upon the training mix of my older zeroshot-v2.0 models. I also hope that there will be a multilingual variant in the future.
Great work by https://huggingface.co/answerdotai !
If youโre looking for a high-speed zeroshot classifier, give it a try!
๐ Resources below: ๐
Base model: https://huggingface.co/MoritzLaurer/ModernBERT-base-zeroshot-v2.0
Large model: https://huggingface.co/MoritzLaurer/ModernBERT-large-zeroshot-v2.0
Updated zeroshot collection: https://huggingface.co/collections/MoritzLaurer/zeroshot-classifiers-6548b4ff407bb19ff5c3ad6f
ModernBERT collection with paper: https://huggingface.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb
Articles
Organizations
florentgbelidji's activity
liked
a
Space
25 days ago
liked
a
Space
about 1 month ago
liked
a
model
about 1 month ago
liked
a
model
about 2 months ago
liked
a
model
2 months ago
liked
a
Space
2 months ago
liked
a
Space
3 months ago
liked
a
model
3 months ago
liked
a
model
4 months ago
liked
a
model
6 months ago
liked
a
Space
7 months ago
liked
a
model
7 months ago
liked
a
Space
7 months ago
liked
a
Space
8 months ago
liked
2
models
8 months ago
liked
a
Space
8 months ago
liked
a
model
8 months ago
liked
a
Space
8 months ago
liked
a
dataset
9 months ago