ROUGE-1

1. 1

   En evalueringsmetrik, der måler overlap af enkeltord (unigrammer) mellem en automatisk genereret tekst og en eller flere referencetekster, typisk brugt til at vurdere kvaliteten af tekstopsummering.

   • Systemets ROUGE-1 F1-score var 0.45, hvilket indikerer en moderat overlap med referencesammenfatningerne. — forskningsartikel, 2023
   • For at beregne ROUGE-1 opdeles både kandidat- og referenceteksten i unigrammer, og antallet af overlappende unigrammer tælles. — forskningsartikel, 2004

Bruges typisk til at vurdere kvaliteten af maskinelle opsummeringer. Beregnes som recall, precision eller F1-score for unigramoverlap. Det er en standardmetrik i summarisering, men fanger ikke synonymi eller syntaktisk variation.

ROUGE-1 Recall = (count of overlapping unigrams) / (total unigrams in reference); ROUGE-1 Precision = (count of overlapping unigrams) / (total unigrams in candidate); ROUGE-1 F1 = 2·(P·R)/(P+R)

from collections import Counter

def rouge_1(candidate, reference):
    cand_tokens = candidate.split()
    ref_tokens = reference.split()
    cand_counter = Counter(cand_tokens)
    ref_counter = Counter(ref_tokens)
    overlap = sum((cand_counter & ref_counter).values())
    recall = overlap / len(ref_tokens) if ref_tokens else 0
    precision = overlap / len(cand_tokens) if cand_tokens else 0
    f1 = 2 * (precision * recall) / (precision + recall) if (precision + recall) > 0 else 0
    return recall, precision, f1

Simpel Python-implementering af ROUGE-1, der beregner recall, precision og F1 for unigramoverlap.

ROUGE står for Recall-Oriented Understudy for Gisting Evaluation, introduceret af Chin-Yew Lin i 2004. ROUGE-1 er varianten for unigrammer.

• ROUGE: A Package for Automatic Evaluation of Summaries (Lin, 2004)