This is a blog version of my talk at the ACL 2021 workshop Representation Learning for NLP (and updated version of that at NAACL 2021 workshop Deep Learning Inside Out (DeeLIO) ).
In the last decade, machine translation shifted from the traditional statistical approaches with distinct components and hand-crafted features to the end-to-end neural ones. We try to understand how NMT works and show that:
- NMT model components can learn to extract features which in SMT were modelled explicitly;
- for NMT, we can also look at how it balances the two different types of context: the source and the prefix;
- NMT training consists of the stages where it focuses on competences mirroring three core SMT components.
