KantanMT vs Microsoft Translator Hub
Using the same training and testing data, the BLEU score from an engine trained in KantanMT was 45.45 while the BLEU score from an engine trained in Microsoft Translator Hub only yielded a score of 32.41.
However while post-edited machine translations (PEMT) only took an hour to evaluate a sample of 500 words from the Microsoft Translator Hub engine, PEMT took almost twice the time from the KantanMT engine. The reason is that more contents from the KantanMT engine were left untranslated. So even with a higher BLEU score, the quality of the translation that the KantanMT engine produced was less than the content the Microsoft Translator Hub engine produced.
That being said, KantanMT is not without its advantages. It took less than one hour to train the engine in KantanMT. While using the same materials, training an engine in Microsoft Translator Hub took almost two weeks. Also, KantanMT points out exactly what is wrong with the dataset if the training failed while Microsoft Translator Hub only displays a generic “error” message when the training failed, leaving the users clueless of what went wrong. However, the use of Microsoft Translator Hub is completely free for users translating less than two million characters per month. On the other hand, KantanMT charges a flat monthly subscription fee.
The most important feature all translators want in an SMT engine is for the engine to produce a good quality translation in a very short amount of time. The SMT engine I would recommend for translating Taiwanese Law would be the engine from Microsoft Translator Hub solely because of the quality it produced. Although training the engine took less time in KantanMT, there is no guarantee that a fully trained KantanMT engine can produce good enough of a quality to justify using the engine at all.
