Category: Uncategorized

Translation Crowdsourcing Best Practices

• Translation Crowdsourcing: Motivating the Crowd

• Translation Crowdsourcing 2: Where Quality Meets Reality

A Step Further: Translation Crowdsourcing Initiative Simulation

You might already be familiar with the project management triangle, also known as the project triangle, which demonstrates an inevitable tension between its three dimensions; cost, time, and quality. It is a common belief that you can only get two out of the three, and the last one has to suffer.

As with many initiatives that rely on crowdsourcing, achieving consistency in crowdsourced translation is a real challenge. Although cost and speed are the true gainers, quality will always suffer. This should not discourage you from considering crowdsourcing as a valid solution for your translation project. With the right kind of processes and measures in place, you can ensure that the quality is just as good, or in some cases even better, than the quality you might receive from traditional translation workflows.

Ensuring Quality in Translation Crowdsourcing

Volunteer Vetting

Always vet your volunteers. This might add a little bit of extra time, but the gains are worthwhile. There are multiple ways to vet your crowd. A simple test might be just enough to ensure that they are bi-lingual and are capable of doing the job. Also, asking for a proof of qualification like previous experience or education, might be of help along with the test.

Try to identify some sort of hierarchy between your contributors. You would need senior contributors who would review and vet your volunteer translators and do the final quality check before publication.

Content Type

Crowdsourced content should be of medium-to-low-level visibility. It is highly discouraged to crowdsource highly visible or highly technical content. This is your most valuable asset, and you should ensure that it gets translated by professional translators and later undergoes a strict quality assurance process.   

Help Documents and Training

Volunteers should be provided with references, contextual information, glossaries, screen shots, and any aiding material that would facilitate their job.

It would be ideal if you could provide some training at the beginning on the tools that they will be using, best practices of translation, and to familiarize them with your business. Always remember that every investment you put in your crowdsourced project will pay off at the end.

Voting System

The voting system is a good way to filter and ensure quality in crowdsourced translation. Using this system, which resembles peer-to-peer feedback, volunteers can vote up or down a version of translation submitted by another volunteer. This system could potentially compensate for the traditional quality assurance process, and it could yield better translations or as good as those done by professionals. You could also structure the voting system to be weighted, which means that contributors with more experience could have a higher casting power (worth two points for instance) than a novel translator.

It is not an easy task to have people give up their time or effort for free. Even for the best of causes, it is still hard enough to find people who won’t give up on the project along the way.

Crowdsourcing as a concept has been around since ancient times, and it has evolved over the years into a business model based on the willingness of companies to engage and involve a potentially large and unknown population (i.e. the crowd) to accomplish a certain goal. In this model, all elements need to be aligned with the diverse and cost-effective crowd-based value creation.

In the translation and localization industry, as much as other industries, crowdsourcing has been utilized and has proven its value to the point that it has been utilized by non-profit and for-profit organizations alike. However, crowdsourcing is most appropriate when the business has already a dedicated community who is passionate about helping out without any monetary rewards. Conversely, it is least appropriate when the business is just using this model to avoid the cost of hiring professional translators. As magical as it might sound as a business solution, there are lots of considerations that need to go into its success. Keeping the crowd motivated and ensuring that the quality standards are maintained are the top key factors for a crowdsourced translation project’s success. In this blog post, I will share some best practices that are fundamentally critical to achieving sustainable success with either translation crowdsourcing, community translation, or social localization.

Motivating the Crowd

Typically, translation crowdsourcing is performed via the internet. As convenient as it seems, it is still quite a challenge to market for your project and ensure that you have enough volunteers or participants who would like to get involved either for free or paid. I suggest that you start planning out based on Aristotle’s mode of persuasion. In simple words, you should appeal to people’s ethos, pathos, and logos as primary motivators. A very important component of crowdsourcing is appealing to people’s emotions or pathos. There should be a strong emotional reason to get them give up their time and energy for free. There are different strategies to win your crowd over. First, the project itself could be an emotional motivator. You might already have a community who is passionate about your service/product/message and would love to be part of it, and this is where the ethos comes in. If you still do not have a strong community yet, you could easily build and maintain one by looking into one of the following strategies.

Marketing

A good marketing strategy is the key to the success of your first or even tenth translation crowdsourcing initiative. Showcasing your initiative is the first step to attract considerable attention to your project. Always consider making a strong case for your project and appeal to your audience’s ethos, pathos, and logos. If the business is for-profit, the case has to be even stronger because it is more likely that people would be skeptical of your intentions.

Acknowledgements

Acknowledgement is generally a powerful motivator in any business and in some cases it might be more powerful than monetary rewards. People like to be appreciated and feel that their work is seen. It is a subtle tool that could drive performance and keep people motivated. Some of the ways with which you could show your appreciation in the virtual world are rewarding badges, leader boards, points, or even more deliberate ways like acknowledgment posts or gift cards and swags.

Feedback

Giving consistent and meaningful feedback is also key to motivation. For many of your devoted volunteers, this might be a learning opportunity. This could be done by providing peer-to-peer reviews by super users, those who are devoting the most time and know the product best, or by professional reviewers who are helping with the project. It is also wise to listen to any ideas or suggestions that the crowd might have, since they share your passion about the project, and they might be more knowledgeable about the target market. Providing feedback also appeals to people’s logos. You are motivating them by the value of their contribution and are giving them more reasons to participate.

Technology

Most of those who are involved in crowdsourced translation are usually non-professionals. Having good and easy technology in place will provide a good experience for your volunteers and can also boost the quantity of translation. Both the technology that they will use and the process they will follow should be simple and easy to learn and use, or else it might discourage them from participating.

Whether you are recruiting, on-boarding, or retaining your crowd, the above best practices would always help in keeping them motivated and engaged.

Just recently, I learned about localizing games made with Unity, a flexible real-time development platform that streamlines and simplifies the game development process. Two teammates and I decided to test our newly acquired skills and embarked on a mission to localize Jumping Giraffe, a 2017 2D open-source game we found on UnityList, into Arabic, French, and Russian. We were aware from the beginning that Unity does not have its own built-in solution for localization. For us to progress, we started out by evaluating the game’s-readiness for localization. The game consists of 7 main scenes and there are few strings in the code, but most of the strings were in the UI.  When we started looking at the game, we knew that there was zero support for Arabic in Unity, and by that, I mean text display and text direction. So, we knew that we needed to fix this problem before starting any internationalization or localization work.

Preparation

Luckily, we had access to the I2 Localization plugin, an easy and powerful localization system that includes all the necessary tools for localization. We were extremely happy with the plugin when we learned about it, however, it did not take us long to realize that the plugin alone won’t be able to meet some of our localization needs.

The plugin is powerful. It was mostly straight forward to add French and Russian support. However, adding Arabic was another story. Although the plugin has support for RTL languages and the code has methods to deal with Arabic, the text direction and the orientation were LTR instead of RTL.

We researched how others solved Arabic localization problems with Unity. We found this free feature-complete Arabic Plugin named “Arabic Support for Unity” on the Unity Asset Store. The plugin has a method ArabicFixer.Fix(“”) that you can annotate strings in the code with, to correct the orientation and connect the Arabic letters.

Workflow

The ArabicFixer plugin and the I2 plugin did not play well together for two reasons:

a) Since the only way to use ArabicFixer is to annotate the strings in the code, the Arabic fixer script gets invoked even for non-Arabic script-based languages, causing them to not be displayed properly.

b) Since the I2 plugin claims to include Arabic support, it actually reversed the impact of the ArabicFixer. Additionally, ArabicFixer only works in code, this means that any UI strings won’t be fixed.

To overcome these issues, we consulted with an engineer who strongly advised against embedding text into the code and advised instead to do the following:

a) We changed the ArabicFixer plugin to check if the string to fix only contains Arabic characters. If the string didn’t contain any Arabic characters, we return the string as is.

b) We changed the Text rendering class in Unity to have it call the ArabicFixer before it returns the rendered text. This simple change prevented us from wrapping each and every code string with the ArabicFixer method. This brilliant fix made the process more dynamic, reducing recoding and adaptation time.  

c) We changed the I2 plugin to disable RTL fixing by default.

Attaching text objects to the UI

Unity does a good job recognizing text objects. For us to get around the localization of UI the elements, we created a variable for each UI element in the corresponding script for the scene and each variable had a string value, thus changing every UI string to be a code string. After getting this step done, we went back again to Unity and we attached each text object to UI element on the specific canvas. The below picture shows our workaround.

After fixing issues with RTL and UI strings, we were finally able to work on internationalization and localization.  

All strings that appear in code were externalized using the I2.Loc.LocalizationManager.GetTranslation(“”) method, and then adding the keys and their translations in the three languages in the language source of the I2 package.

After thoughts

I am really glad that I have learned so much about Unity working on this game. Even though my workaround seems to be long, it gave me the opportunity to work with C# and to really think outside of the box but not relying entirely on the two plugins I used. However, now that I am looking back, we needed to spend some time exploring the I2 plugin. It seems to be a powerful tool and we might have only needed to invest some time to fix issues with RTL to make the most out of the plugin. We also could not figure out how to make the languages appear in-language, something that we learned later from some of our colleagues and could easily be done in Unity.

Even though Unity does not have its built-in localization environment, it still supports the process by allowing you to create components, which cuts out a good chunk of the manual coding process.

Anyone who knows me knows that if I did not have a career in language services, I would have chosen a career in graphic design. When I signed up for the Desktop Publishing course this semester at MIIS, I did not expect that I would enjoy it as much as I did. It was by far my favorite course during my first year at the School.

Over the course of the semester, we learned many tools that would eventually allow us to successfully localize DTP projects into any language. The most interesting part about this course was that it was not language pair restrictive, by that I mean that I had the chance to try localizing and preparing DTP projects in languages other than Arabic.   

For my final project for this course, known to the MIIS TILM circle as Project Showcase, I chose to subtitle a 3-minute video suggested by my Professor Max Troyer. The video is part of Sesame Street’s 50th anniversary celebration, in which they launched their joint campaign with HBO, “Respect Brings Us Together,” to highlight the importance of respecting one another, even in disagreement.

Disclaimer:

This project is intended as a “proof of concept” and in no way represents either HBO or Sesame Street.

Project Analysis and Workflow

This project was mainly about subtitling and video editing of on-screen text. I started off by analyzing the video and identifying what needed to be localized. Below is what I have identified and ended up working on:

1. Running “Sesame Street” logo throughout the video.
2. On-screen text (OST) that needed to be masked and replaced.
3. Another “Sesame Street” logo celebrating its 50th anniversary which appears along with OST.
4. Replacing the fonts for the OST with matching fonts that would work in Arabic.

I created a picture list for the OST using MS Word. The next step was spotting, subtitling and QA-ing which were fairly easy in VisualSubSync Enhanced. I put everything together in a memoQ project and translated it and kept the files aside for later. Next, was the video editing stage which I will discuss in full in the section on challenges.

Tools

For this project, I had to use a total of 5 tools, not including basic tools like MS Word. I worked mainly in Adobe After Effects (AE) and Photoshop for video editing. For spotting and subtitling, I used VisualSubSync Enhanced. I translated the picture list and the .srt file in memoQ and the final QA was done in VLC Media Player.

Challenges

On-Screen Text

When I first decided to work on this project, I had the new Content-Aware Fill feature in AE in mind. It is a newly added feature in the 2019 release of AE powered by Adobe Sensei which is said to be magical. It can remove unwanted objects from any video, such as logos, text objects and even people. Unfortunately, I did not have any luck with this tool. I tried multiple times to get it working, while following the same steps discussed in this tutorial, but all in vain. The process would stop in the middle, and I had to start over from the beginning each time. Honestly, the tool seems to be very promising, but based on my experience I can say that it is not functional yet.

How I went about the on-screen text needs a blog post on its own, but let’s just say that I had to create a reference frame for each video frame
— which looked more like a still image— that contained text using another new feature in AE called Create Reference Frame. This feature opened in Photoshop, then I used the quick selection tool to select the text that I wanted to get rid of. Then using the Content-Aware Fill feature in Photoshop, I was able to fill in the empty spaces with the same background texture surrounding them. Next, I would save the new .psd file which automatically saved and created a new reference frame in the AE project.

I had to create a total of 83 reference frames for the first sentence “Respect brings us together”, 101 reference frames for “A message from” and 108 reference frames for “Fifty Years and Counting.”

As for the logos, I used the Arabic version of the “Sesame Street” logo that I found online and placed it wherever I needed.

The below screenshots demonstrate how I handled one OST:

Burning the Subtitles

During my coursework, I used HandBrake to burn-in any subtitles I created. Despite it being a handy tool, I was not quite happy about it because it does not support RTL languages, which just adds to my series of frustrations! But I did not stop there! By the time I discovered this technical issue with HandBrake, I was determined to find the one tool that has full support for Arabic and could burn-in AR subs, and potentially all RTL languages, without further technical problems.

I started my tool hunt as early as I started working on this project and even before. I tried a bunch of tools and none of them fixed the problem, including Format Factory, MeGUI, VobSub, and even Adobe Premiere Pro — I turned on the South Asian and Middle Eastern settings and tried to manually change the text direction and the text was still left aligned.

Just before I was about to give up, I found the tool I was looking for. Surprisingly, it is VLC Media Player. It is the same media player I have been using forever, and it never occurred to me that it has the feature to do this job. I was exploring some tutorials on YouTube when I accidentally came across this tutorial on how to hard-code subtitles in VLC. Since I have configured VLC to display Arabic subtitles correctly (right font and text direction), I was confident that it was going to work out, and it did!

Below are two screen shots from the localized video showing the AR burned-in subtitles:

Despite how frustrating the whole process felt, I was quite happy to finally know of a tool that would burn-in AR subtitles without further complexity.

Final Video

For all sorts of reasons, this was a complex project. I can say that I am quite happy with the final video even though it is not perfect. Despite that I was beyond my time budget, I am so grateful that I had the chance to try out so many tools and to get to experiment with each of them on my own. Now I have established my own workflow and I have the right tools to work with for any subtitling project. I am more aware of the limits and functionality of each tool I had to use and have definitely developed and honed my designing skills.

You can check out my final video here. I hope you enjoy it!

As part of the continuum of my hands-on and technology focused master’s degree, I took Professor Adam Wooten’s class on Translation Management Systems (TMS) during my second semester at MIIS. This course gave us the opportunity to explore different TMSs such as SDL WorldServer, GlobalLink, Lingotek, and others. We discussed the pros and cons of each TMS, and what are the factors and the needs to be considered in the process of choosing a TMS for a company. We worked on two main projects throughout the semester; one was a TMS comparison project for a hypothetical client, and the other was a TMS consulting project for a real client. Below I am going to give a brief overview of both projects.

Project 1: TMS Group Comparison

For our first project, we were required to compare between two TMSs; SDL WorldServer and GlobalLink, for our hypothetical client who is in the process of selecting a TMSs for their business. During this project, we wore many hats. We were the client, the project manager, the linguists and the marketers. We had the chance to evaluate each TMS from the different perspective of each stakeholder while considering their needs. We ran an example project through both TMSs, and we weighed the pros and cons of each TMS in a scorecard that my teammates and I established based on the needs of each stakeholder.  

Below are the basic business requirements we identified for our client:

Based on our results and assessment of the client’s needs and our own
online research, GlobalLink was what we recommended. We discussed our findings with the client in our final presentation.

You can check out our presentation on the findings and our scorecard, if you want to know more about the project.

Project 2: TMS Group Consulting Project

Our second project for this class was a consulting project for a real client, who have their own TMS but are looking to add improvements, and possibly new features, to make it ready to meet the competitive market standards.

For confidentiality reasons, I am not going to share the project files here, instead I am going to discuss the project in full.

Following a kick-off meeting with the client, we were divided up into groups based on different stakeholders. We were required to assess the current features and functions of the client’s TMS and identify areas of improvement.  

My team evaluated the TMS from the project manager standpoint. We were able to identify multiple areas for improvement, most of them had to do with project automation, such as:

• automatic passing of project files from translator to the PM and vice versa;
• assigning projects to translators during the project creation stage;
• notifications system; and
• quoting system.

One major area of improvement was the user help files and onboarding documentation. The client’s TMS lacked any kind of documentation which we identified as a major downside of this TMS, because it is quite unusual to find any tool without proper documentation.

All our recommendations were discussed in full in a recorded presentation that we delivered to our client.  

This course gave us a great overview of the TMSs available in the market. We learned how to carry out a selection process, from identifying basic business requirements to the actual implementation. We also had the chance to actually try out these TMSs which, I believe, prepared us to be able to use any TMS we haven’t tried before.

During my Spring 2019 semester at the Middlebury Institute of International Studies at Monterey, I took a course on advanced computer assisted translation (Advanced CAT). This course built upon the knowledge that we already had acquired during our Intro to CAT course the previous semester. A lot of focus was put on learning how to use or manipulate advanced features of many CAT tools like creating filter configuration for XML files and using regular expressions (regex) for quality assurance purposes.
We have also experimented with the Okapi suite like Okapi Olifant and Okapi Checkmate to maintain and clean up TMs and MT Corpora.

Language-specific Regex for Trados QA Checker

For this particular exercise, I had the chance to work with my teammates on coming up with some regular expressions to identify potential errors and verify translations in Chinese, a language I cannot read or speak. We used a handy online tool, Regex101, to test our expressions, and then we implemented them in SDL Trados QA Checker.

Here are some of the expressions we came up with:

SMT Engine Training

Our most challenging project by far was training a statistical machine translation (SMT) engine using Microsoft Custom Translator. The goal for this project was to design and implement a pilot project for a custom-trained SMT engine for translating the Annual Reports published by the European Court of Auditors (ECA) from English into French. This project involved many tasks, some of them were time consuming like document alignment and cleaning, especially that we were working mainly with PDFs. Other tasks involved file conversion, calculation of costs, post editing, manual quality evaluation (inspired by DQF and MQM error typology), and of course continuous troubleshooting and multiple iterations for each round of training. Our major key take-away from this project was the importance of risk management.  All our drawbacks combined were not accounted for in our timeline, and resulted in pushing us beyond our estimated time budget and the scale of the project greatly exceeded our initial expectations

To look at our project in detail, you can check out both our initial and updated proposal and presentation on lessons learned here.  

We also experimented a bit individually with Microsoft Translator Hub, the predecessor of Microsoft Custom Translator, before its retirement on May 17, 2019. I am grateful that I got the chance to see how training an SMT Engine looked like before Custom Translator came to existence.

Project Fluent

We also had a look at Project Fluent, an interactive software localization system developed by Mozilla. Project Fluent is designed to help translators produce natural-sounding translations by enabling them to use the entire expressive power of their language without asking developers for permission. Fluent makes it possible to cater to the grammar and style of many languages, independently of the source language, which in most cases is English.

For this exercise, I played around with Fluent to come up with customized rules for the gender and plural categories in Arabic. In addition to the masculine and feminine gender, Arabic has singular, plural and dual forms of pronouns, nouns, verbs, adjectives, etc. The dual form is used to refer to two people or two things. In the below rule, one line of code was included to account for the dual form. Also, in most cases, possessive pronouns in Arabic do not come in an isolate form. These pronouns are always attached to the objects they refer to, which is why the rule was customized, as shown below, by repeating the Arabic translation of the two words “comment” and “post” three times each with the corresponding pronoun attached to each instance.

Utility Training Video

One of the final mini projects was to record an instructional video for any utility tool of our choice that could help in a localization workflow. I chose to do mine on ClipMate— a clipboard manager that could potentially save hours of coping and pasting content everywhere. It also helps the user with relocating their saved files and folders by keeping clipboard history. You can check out my instructional video here.

My very first encounter with localization project management (LPM) was through a class that I took during my first semester at MIIS. In this class, which was in a workshop format, we looked in depth at every stage of the localization project life cycle and localization best practices. We covered a broad spectrum of topics, concepts, processes, and tools. We also visited many real-life examples through which we learned how to critically assess different situations and manage issues that might arise during the project.

Throughout the semester, we worked closely with ISO standards on project and quality management, especially ISO 17100 for Translation Services. Having these international standards at the backdrop and actively using them, made us aware of what to expect as project managers, what is expected from us as service providers, and what are the minimal requirements and competencies that should exist within any project to deliver quality translation and localization services. Students in my cohort, including myself, were divided into small groups of 6 students. Each group simulated the working environment of an actual LSP, and were hired by their first hypothetical client for a localization project. Through this simulation, we were exposed to the fundamental principles of LPM.

For our final project, we developed a Standardized Localization Project Management Office through a variety of project management tools like DokuWiki, TopTracker, and our primary tool, and friend along the way, Trello. Using Trello, we created a standardized workflow, going through several stages and iterations. We also created checklists for each stage of the localization project to help ourselves be on track. We created a quote, a timeline, and a profile for our client. Based on set criteria, we evaluated and hired qualified translators from professional networks like the ATA Online Directory or ProZ.com. Also, we had the chance to experiment with the CAT tool Memsource, and to simulate how working through an actual multilingual project would look like.

Our final deliverable for the course was to present our Standardized Localization Project Management Office and workflow in a postmortem presentation. It was a great opportunity to look back on what went right and what went wrong and point out some takeaways.

Takeaways

One of the things that I liked the most about this course was having the opportunity to recycle and apply other tools and knowledge I learned from other classes at MIIS like Advanced Business Applications and Intro to CAT.

One thing my team could have done differently, and would have helped us work more efficiently, was to have a clear role for each member of our team. We spent a considerable amount of “unbillable time” trying to communicate what is each person’s task for the week for instance. Even in a hypothetical situation like this one, it is better to have a clear role/s for each team member which would have established some sort of internal accountability.

Documentation is also very important in our context. We can not rely, in many situations, on just verbal agreements. Another takeaway was the importance of monitoring our time. We used TopTracker to track all the time we spent working on this project. We were very surprised at the end to see how much time we spent overall, and the breakdown of this time per job. This gave us something to think about for future projects; how to work efficiently and provide quality at the same time.

From the beginning of the semester to now, I realized I have developed new skills and strengthened other skills that I already possessed. I had a slightly rough idea about how translation and localization projects are managed, but taking this class changed a lot of misconceptions that I had, and provided me with the necessary resources to develop stronger project management skills for the future.