FQuAD

The French Question Answering Dataset

What is FQuAD?

French Question Answering Dataset (FQuAD) is a new reading comprehension dataset, consisting of questions on a set of Wikipedia french articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage. With 25,000+ question-answer pairs on 145 articles, FQuAD is significantly larger than previous french reading comprehension datasets.

Explore FQuAD and model predictionsFQuAD paper (d’Hoffschmidt et al. '20)

Getting Started

We've built a few resources to help you get started with the dataset.

Download a copy of the dataset (distributed under the CC BY-SA 4.0 license):


To evaluate your models, we have also made available the evaluation script we will use for official evaluation, along with a sample prediction file that the script will take as input. To run the evaluation, use python evaluate-fquad.py <path_to_dev-fquad> <path_to_predictions>.


Once you have a built a model that works to your expectations on the dev set, send us an email with your model and we will evaluate it on FQuAD test set and update the leaderboard accordingly.


Have Questions?

You can contact us here.

Star

Leaderboard

Here are the ExactMatch (EM) and F1 scores evaluated on the test set of FQuAD.

RankModelEMF1
Human Performance

Illuin Technology

(d’Hoffschmidt et al. '20)
75.991.2

1

Apr 24, 2020
Camembert-large (single model)

Illuin Technology

https://arxiv.org/abs/2002.06071
82.091.5

2

Apr 24, 2020
XLM-Roberta-large (single model)

Illuin Technology

https://arxiv.org/abs/2002.06071
78.588.7

3

Feb 14, 2020
Camembert-base (single model)

Illuin Technology

https://arxiv.org/abs/2002.06071
75.886.0

4

Apr 24, 2020
Bert-multilingual-base (single model)

Illuin Technology

https://arxiv.org/abs/2002.06071
72.383.9

5

Apr 24, 2020
XLM-Roberta-base (single model)

Illuin Technology

https://arxiv.org/abs/2002.06071
71.482.2

5

Apr 24, 2020
XLM-Roberta-base finetuned on SQuAD1.1 zero-shot (single model)

Illuin Technology

https://arxiv.org/abs/2002.06071
70.382.7