Aus SDQ-Institutsseminar
Termin (Alle Termine)
Datum Freitag, 9. Juli 2021
Uhrzeit 14:00 – 14:30 Uhr (Dauer: 30 min)
Webkonferenz Teams
Vorheriger Termin Fr 2. Juli 2021
Nächster Termin Fr 16. Juli 2021

Termin in Kalender importieren: iCal (Download)


Vortragende(r) Dennis Grötzinger
Titel Exploring The Robustness Of The Natural Language Inference Capabilties Of T5
Vortragstyp Bachelorarbeit
Betreuer(in) Jan Keim
Kurzfassung Large language models like T5 perform excellently on various NLI benchmarks. However, it has been shown that even small changes in the structure of these tasks can significantly reduce accuracy. I build upon this insight and explore how robust the NLI skills of T5 are in three scenarios. First, I show that T5 is robust to some variations in the MNLI pattern, while others degenerate performance significantly. Second, I observe that some other patterns that T5 was trained on can be substituted for the MNLI pattern and still achieve good results. Third, I demonstrate that the MNLI pattern translate well to other NLI datasets, even improving accuracy by 13% in the case of RTE. All things considered, I conclude that the robustness of the NLI skills of T5 really depend on which alterations are applied.
Neuen Vortrag erstellen