The TAUS Quality Evaluation (QE) Summit 2015 took place in Dublin this year. Welocalize’s VP of Technology Solutions, Olga Beregovaya and Program Manager of Language Tools, Lena Marg, participated in the event and delivered a joint presentation at the event, “Dynamic Quality Revisited: Stakeholders, Challenges, Best Practices”. In this blog, Lena Marg sums up experience from the TAUS QE summit.
After attending a TAUS QE summit for the first time in 2014, I was looking forward to taking part and presenting at this year’s event again with Olga. The TAUS QE summit 2015 in Dublin was hosted by Microsoft at their European Product Development Center and the venue created a cheerful, open and pragmatic atmosphere for discussions.
The strength of these meetings lies in bringing together competing translation providers, technology providers, translation buyers, as well as academia, well-known contributors as well as newcomers. The format invites participants to present solutions and new approaches, also to share their challenges and questions. With most attendees facing localization challenges on a daily basis, everybody is looking for solutions, answers and best practices.
This year’s core topics at the QE Summit were business intelligence, productivity metrics, ephemeral translations and profiling and assessing non-conventional content, reflecting the breadth and different angles and trends around today’s most topical localization and automation challenges.
TOPIC FOCUS: BUSINESS INTELLIGENCE
The presentations by Tom Shaw (Capita) and John Tiensley (Iconic) on business intelligence were very informative. They covered key points like how to calculate ROI when investing in machine translation (MT)? When to invest in MT? What type of MT (generic or customized)? When and how to evaluate MT? These sessions were very relevant to translation buyers unsure of how to approach the spectre of MT and implement it into production. At Welocalize, we run MT evaluations and benchmark on an ongoing basis so I was grateful for John’s reminder of “seeing the wood for the trees”. For linguists, too, it is important to look beyond the segment-level and see the bigger picture of what MT is trying to achieve and how useful it actually is in its different implementations.
TOPIC FOCUS: PRODUCTIVITY METRICS
The presentations by Paul Filkin (SDL), Mirko Plitt (Modulo) and Attila Görög (TAUS) on productivity metrics, lead to very lively breakout sessions. Many attendees regretted not being able to attend all of them! The question of what data to collect from translators, when and why, remained controversial. The notion of “big brother is watching” still looms large. Plugins like the Studio Time Tracker Professional, developed by Patrick Hartnett on SDL OpenExchange (presented by Paul Filkin), are therefore welcome developments. These allow translators to track their productivity independently of productivity evaluations set up by LSPs or translation buyers. Teamed together with the newly launched TAUS DQF API, we can hope to see an increase in translators and translation providers gathering granular productivity data in production, allowing for a more balanced dialogue and providing further impetus to MT and language technology researchers and developers.
Mirko Plitt’s presentation on Automatic Detection of Poor Translation Work, resonated particularly well with the stakeholders of Quality Assurance (QA) in the room. They are looking for faster and lighter models of quality validation and also bigger ranges of quality requirements and real-time translation requests.
TOPIC FOCUS: EPHEMERAL CONTENT
The presentation I delivered with Olga Beregovaya, “Dynamic Quality Revisited: Stakeholders, Challenges, Best Practices” provided examples of the breadth and noticeable increase in translation requests for ephemeral content that we have seen at Welocalize in the last nine months. What these requests have in common is that both source content and translations have an extremely short shelf life. They tend to be huge in volume, too big for conventional translation, and they can be user-generated at source but the translation still has significant impact for the translation buyer.
Ephemeral content may be used to enable one-off communications, increase market and online presence, or be used to gauge website traffic and future translation needs. Quality requirements vary depending on the exact purpose, although often “gisting quality” is good enough. Gisting quality enables the end-user to grasp the overall meaning effectively. This content requires simple, integrated workflows and flat supply chains.
In view of this trend, it was great to share experiences with João Graça of Unbabel, who offer a new translation platform to combine machine and crowd translation for very fast turnaround translation tickets. MT, crowd approaches as well as simpler translation platforms or translation apps are all necessary to cater for the increase in ephemeral content and its translation.
TOPIC FOCUS: PROFILING AND ASSESSING CONTENT
Content classification and profiling in localization is surprisingly challenging. How to further classify content to further customize MT output? What new content types are out there and what do they mean for translation providers?
Different categories overlap and content at the author stage can be defined, grouped and merged by workflow choices, product lines or authoring groups rather than typical and convenient “techdoc”, “UA” and “marketing” categories. As Sharon O’Brien (DCU) showed in her findings during her content profiling session, there also doesn’t seem to be a common approach to content profiling and labelling in the industry, despite the clear benefits of content classification in defining processes, translation quality requirements as well as customizing MT systems. Something to keep watching.
If you would like to see the slides from our TAUS QE Summit presentation, simply click on the following link TAUS QE Summit_Dublin_05_2015_Welocalize.
Lena Marg is Program Manager of Language Tools at Welocalize.