Welocalize StyleScorer Helps MT and Linguistic Review Workflow
Innovation is one of Welocalize’s four pillars which form the foundation of everything we do as a business. Clients and partners rely on our leadership to drive technological innovation in the localization industry. One of our latest innovative efforts is the soon-to-be-deployed language tool, Welocalize StyleScorer which will form part of the Welocalize weMT suite of linguistic and automation language tools. One of the driving forces behind StyleScorer is Dave Landan, computational linguist at Welocalize and a key player in many Welocalize MT programs.
In this blog, Dave shares the key components of StyleScorer and how style analysis tools can help the MT and linguistic review workflow.
At Welocalize, we are constantly looking for ways to improve the quality and efficiency of the translation process. Part of my job as a computational linguist is to create tools that help people spend less time on looking for potential problems and more time on fixing them. One of my team’s latest efforts in this area is StyleScorer.
Welocalize StyleScorer is currently in the early deployment testing phase. This tool will be deployed as part of the Welocalize weMT suite of language tools around linguistic analysis and process automation. I’d like to share some of the key components of StyleScorer and the role it will play in the MT and linguistic review workflow.
What is StyleScorer?
Welocalize StyleScorer is a tool that compares a single document to a set of two or more other documents and evaluates how closely they match in terms of writing style. The documents being compared must all be in the same language; however, there is no restriction on what that language is in the source content.
The main difference between StyleScorer and existing style analysis tools is that rather than summarize types of style differences (for example: “17 sentences with passive voice”), it takes a gestalt approach and gives each document a score anywhere between 0 and 4, with 0 being a very poor match to the style and 4 being a very good match.
To do this, StyleScorer uses statistical language modeling as well as innovations from NLP (natural language processing), forensic linguistics and neural networks (machine learning) in order to rate documents on how closely they match the style of an existing body of work. Because it learns from the documents it’s given, even if you don’t have a formal style guide, StyleScorer will still work as long as the training documents can be identified by a human as belonging to a cohesive group.
How does StyleScorer help the MT workflow?
While we think StyleScorer will be very useful as part of the linguistic review workflow for human translation, we are even more excited about how it can benefit the MT (machine translation) workflow at several points of the process both on source and target language documents.
One of the key components to training a successful MT system is starting with a sufficient amount of quality bilingual data. We are seeing more and more clients who are very interested in MT; however, they don’t have a lot of bilingual training data to get started. In the past, the only option available to those clients was a generic MT engine (similar to what you’d get off-the-shelf). This gets someone started in MT, though the quality of generic engines is generally lower than engines trained with documents that match the client’s domain and style.
We can use StyleScorer to filter open-source training data to find additional documents to train from that are closest to the client’s documents. High-scoring open-source data can then be used to augment the client’s training data, which allows us to build better quality MT engines for those clients early in the project life cycle.
If some documents are getting lower quality translations from MT than others, we can use StyleScorer as a sanity check as to whether the source document being translated matches the style of the client’s other documents in the same language and domain. An engine trained exclusively on user manuals probably won’t do well on translating marketing materials. StyleScorer gives us a way to look for those anomalies automatically.
We are particularly excited about using StyleScorer on target language documents to help streamline workflows. If we run StyleScorer on raw MT output, we can use the scores to rank which documents are likely to need more PE (post-editing) effort to bring them in line with the style of known target documents. This is particularly useful for clients with limited budgets for PE and clients with projects that require extremely fast turnaround because it allows us to focus PE work where it is needed the most.
Finally, we envision StyleScorer becoming part of the QA & linguistic review process by spot-checking post-edited and/or human translated documents against existing target language documents. Translations that receive lower scores may need to be double-checked by a linguist to make sure the translations adhere to established style guides. If it turns out that low-scoring translations pass linguistic review, we use them to update the StyleScorer training set for the client’s next batch of documents.
Based in Portland, Oregon, Dave Landan is a Senior Computational Linguist for Welocalize’s MT and language tools team.