Neural machine translation improves translation quality

This case study reminded me how significant and wide-ranging impacts from research can be.

The researchers, from Edinburgh University, created a toolkit for neural machine-translation: a translation method which uses deep learning and artificial neural networks to translate text between languages, learning patterns from large datasets.

The toolkit was adopted by Microsoft and, because the researchers made the technology open-source, the United Nations was able to create a bespoke translation tool that is now used by 192 countries.

Not all research will be quite so hard-hitting, but the case study reminded me how dissemination strategy can have a profound effect on the Reach and Significance  of the impacts achieved by a set of research findings.

A non-standard approach

One of the first things I noticed about the narrative was the unusual start of section 2: it does not start with a standalone problem statement. Initially, this worried me, but I soon discovered that the problems addressed by the research were clearly articulated, using non-technical language, further down the page.

I was also concerned that there did not seem to be descriptions of the research methods and that the research findings had been rechristened as ‘innovations’.

However, instead I discovered a narrative that told the story of three issues that were solved, together with enough information for someone not steeped in machine translation to understand the details of the research and what it had delivered.

My takeaway? There are different ways of telling an effective story and sometimes, when the research area is especially technical, a clear higher-level description can work best.

Focus on the good stuff

I was pleased to see that the narrative in the impact section uses the experiences of a selection of beneficiaries to allow the impacts to speak for themselves.

As you would expect, the most high profile adopter, Microsoft, is the first company mentioned. To ensure the impact figures hit home, and to prevent the assessor from being overwhelmed by too much information, the authors focus on a subset of the impacts. By giving detail about those over a single year, they allow the reader to reflect on how much additional impact there must have been since. This could be a risky strategy, but in a case like this, highlighting your most impressive data and not burying it in a lot of secondary information may avoid the reader thinking the impact was all relatively low level. It lets you focus on getting a specific story across in the most effective way, and removes the temptation to ramble and fill the text with multiple, less convincing, testimonies.

Good testimonies are often very hard to get from commercial organisations. In this case the authors side-stepped this issue by citing Microsoft documentation that includes descriptions of why it used the software. This allows the authors to demonstrate that there was significant impact on Microsoft, despite not having an official testimonial.

End on a high

Next we learn how a much smaller company benefitted from the research – further extending the Reach  of the impact. A letter from a senior manager provides financial information that demonstrates the impact of the software on specific areas of its business.

Following the commercial impact, the authors describe  how the United Nations patent agency used the tool to cross-translate between 18 language pairs. The initial evidence is in the form of public domain documents that mentioned the technology, published by the UN. Then they sign off with a direct quote from the WTO’s Director of Languages – a final flourish that would have been appreciated by the assessors.

I enjoyed reading this submission because, although it broke some of the ‘rules’ we use to define a textbook case study, it proves that there is more than one ‘right way’ to write them. What matters is to tell the story in a logical, clear and convincing manner. From the point of view of the UoA11 panel, it certainly worked – the case study was rewarded with a 4* rating.