micm_nlp.evals.metrics.string_f1.string_f1

String-level F1 for QA-style span overlap, packaged as an evaluate.Metric.

Token-overlap F1 computed after SQuAD-style normalisation: lowercase, strip punctuation, drop articles, collapse whitespace.

Note the normalisation is English-specific — the article regex removes a, an and the and nothing else, so scores on other languages are comparable to each other but not to published English numbers.

Classes

StringF1

String-level F1 packaged as an evaluate.Metric.

Functions

f1_score(prediction, ground_truth)	Token-overlap F1 between two strings, after SQuAD-style normalisation.
normalize_answer(s)	Lowercase, remove punctuation, articles, and extra whitespace.

Module Contents

class micm_nlp.evals.metrics.string_f1.string_f1.StringF1

Bases: evaluate.Metric

String-level F1 packaged as an evaluate.Metric.

Loaded by path rather than imported – see the docstring in micm_nlp.evals.metrics for why this module lives in a directory of its own.

The normalisation is English-specific: the article regex strips a, an and the and nothing else, so scores on other languages are comparable to each other but not to published English numbers.

micm_nlp.evals.metrics.string_f1.string_f1.f1_score(prediction, ground_truth)

Token-overlap F1 between two strings, after SQuAD-style normalisation.

Overlap is measured on token sets, so repeated tokens count once – a prediction that repeats a correct word does not score higher for it.

Returns:

F1 in [0, 1]; 0.0 when nothing overlaps.

micm_nlp.evals.metrics.string_f1.string_f1.normalize_answer(s)

Lowercase, remove punctuation, articles, and extra whitespace.