ISCA - International Speech
Communication Association

ISCApad Archive  »  2023  »  ISCApad #301  »  Resources  »  Database  »  Biomedical language model DrBERT

ISCApad #301

Thursday, July 06, 2023 by Chris Wellekens

5-2-29 Biomedical language model DrBERT
We are proud to announce our first biomedical language model for French called DrBERT. It's now available on HuggingFace and Arxiv (
You can now use the model on your own documents and get state-of-the-art performances in only 3 lines of code.
Check out the:
- Hugging Face models:
Our model was trained on 128 GPU from Jean-Zay and assessed on 11 distinct practical biomedical tasks for French language, which came from public and private data. These tasks include : Named Entity Recognition (NER), Part-Of-Speech tagging (POS), binary/multi-class/multi-label classification, and multiple-choice question answering. The outcomes revealed that DrBERT enhanced the performance of most tasks compared to prior techniques, indicating that from-scratch pre-trained strategy is still the most effective for BERT language models on French Biomedical.
Tutorials about biomedical natural language processing are coming soon, stay tuned !!
With Yanis Labrak (LIA / Zenidoc), Adrien Bazoge (LS2N), Richard Dufour (LS2N), Mickael Rouvier (LIA), Emmanuel Morin (LS2N), Béatrice Daille (LS2N) and Pierre-Antoine Gourraud (Nantes University / CHU Nantes).

Back  Top

 Organisation  Events   Membership   Help 
 > Board  > Interspeech  > Join - renew  > Sitemap
 > Legal documents  > Workshops  > Membership directory  > Contact
 > Logos      > FAQ
       > Privacy policy

© Copyright 2024 - ISCA International Speech Communication Association - All right reserved.

Powered by ISCA