Exploring The Robustness Of The Natural Language Inference Capabilties Of T5: Unterschied zwischen den Versionen
(Die Seite wurde neu angelegt: „{{Vortrag |vortragender=Dennis Grötzinger |email=dennis.groetzinger98@gmail.com |vortragstyp=Bachelorarbeit |betreuer=Jan Keim |termin=Institutsseminar/2021-0…“) |
Keine Bearbeitungszusammenfassung |
||
Zeile 5: | Zeile 5: | ||
|betreuer=Jan Keim | |betreuer=Jan Keim | ||
|termin=Institutsseminar/2021-07-09 | |termin=Institutsseminar/2021-07-09 | ||
|kurzfassung= | |kurzfassung=Large language models like T5 perform excellently on various NLI benchmarks. However, it has been shown that even small changes in the structure of these tasks can significantly reduce accuracy. I build upon this insight and explore how robust the NLI skills of T5 are in three scenarios. First, I show that T5 is robust to some variations in the MNLI pattern, while others degenerate performance significantly. Second, I observe that some other patterns that T5 was trained on can be substituted for the MNLI pattern and still achieve good results. Third, I demonstrate that the MNLI pattern translate well to other NLI datasets, even improving accuracy by 13% in the case of RTE. All things considered, I conclude that the robustness of the NLI skills of T5 really depend on which alterations are applied. | ||
}} | }} |
Aktuelle Version vom 30. Juni 2021, 17:26 Uhr
Vortragende(r) | Dennis Grötzinger | |
---|---|---|
Vortragstyp | Bachelorarbeit | |
Betreuer(in) | Jan Keim | |
Termin | Fr 9. Juli 2021 | |
Vortragsmodus | ||
Kurzfassung | Large language models like T5 perform excellently on various NLI benchmarks. However, it has been shown that even small changes in the structure of these tasks can significantly reduce accuracy. I build upon this insight and explore how robust the NLI skills of T5 are in three scenarios. First, I show that T5 is robust to some variations in the MNLI pattern, while others degenerate performance significantly. Second, I observe that some other patterns that T5 was trained on can be substituted for the MNLI pattern and still achieve good results. Third, I demonstrate that the MNLI pattern translate well to other NLI datasets, even improving accuracy by 13% in the case of RTE. All things considered, I conclude that the robustness of the NLI skills of T5 really depend on which alterations are applied. |