These are notes from a breakout session at the AMTA 2010 Workshop on Collaborative Translation and Crowdsourcing, Denver, Colorado, October 31st 2010.
This breakout session focused on a cluster of issues related to motivating the crowd of translators. The issues which were raised during the initial brainstorming session were:
Haiti disaster relief efffort:
Going back to disaster related crowdsourcing, they found a link between the how Newsmaking the issue was, and how much interest there was in helping out.
There is a strong confound of incentive quantity and the standards of living in the countries where language is spoken. For Indian languages you can get away with paying less than, say, Japanese or Korean.
At Kiva.org they found that there are certain demographics for volunteers that they are recruiting...so middle aged professionals and retirees, as well as young mothers are great, but students tend to be quite bad in terms of long-term commitment.
Community is important. Even on Amazon Mechanical Turk people are forming communities on turkernation to discuss the work they do and other related matters. Giving people a good sense of community is very good.
Giving people a humanitarian reason for participating is great. Burch, when collecting data for Pakistani SMS translation, told people what the information would be used to help with disaster relief in Pakistan, and people seemed to really respond on a personal level to the work.
To what extent can Games with a purpose be used to motivate people for language translation tasks?
This breakout session focused on a cluster of issues related to motivating the crowd of translators. The issues which were raised during the initial brainstorming session were:
- What factors can help motivate people in a collaborative or crowdsourced translation situation?
- What factors can help with quality in a collaborative or crowdsourced translation situation?
- With these environments, is there a danger that translators will feel dehumanizedÉ
- Could Games with a purpose be used to good effect in a translation context?
- How can we make translation crowdsourcing TRULY collaborative (as opposed to being environments where individual translators act independantly of one another with little communication)?
- How to share translation expertise within a communityÉ
- What tools are needed for managing the community?
Haiti disaster relief efffort:
- it became hard to keep workers motivated as the process went on before funding came online. Personal relationships really helped keep people going
- Using social media helped both recruit people and provide a positive motivation for people coming into the work in a more personal manner
- Money is the main motivator
- Other sources of motivation besides money:
- community recognition
- personal achievement
- competition (who does the most tasks wins)
- stat maintenance keeps people shooting for a goal ("competing" with themselves going forward)
- marking of their ESL (English as a Second Language) output
- Note: This is applicable in situations where some "translators" in the crowd are in fact, second-language students of the target language.
- translation back and forth view for the Monolingual contributing
- Note: This refers to situations where a monolingual author "translates" into a language he doesn't know, using a MT system. Roundtrip Machine Translation (ex: from original English to Chinese, then back to English) is used to give the author clues about which part of the source sentence is causing problems for the MT system. The author can then rephrase that part of the sentence, to see if the roundtrip translation will come back looking better (which usually means that the Chinese generated by the MT system was also better).
- People on Livemocha want to learn a language and this is a great motivator
- positive feedback and access to more advanced work based on previous performance was a very good tool with people on Amazon Mechanical Turk
- “It’s really nice to remind me what I am”
- “I am hoping that Amazon Mechanical Turk will provide more opportunities for translations in Spanish!”
- “What about localizing Amazon Mechanical Turk in different languages so that any one can easily work in Amazon Mechanical Turk”
- “It is a very enjoyable thing to do research in linguistics.”
- “I’m an outlier, because I love languages so much, but you can count me in, I suppose. Good luck with your research.”
- “Used to be fluent in Latin, but it’s hard to stay in practice with a dead language.”
- “Tamil is my mother tongue ... the creativity will shine in mother tongue only.”
- “This is not only work. It can improve our knowledge also”
Going back to disaster related crowdsourcing, they found a link between the how Newsmaking the issue was, and how much interest there was in helping out.
There is a strong confound of incentive quantity and the standards of living in the countries where language is spoken. For Indian languages you can get away with paying less than, say, Japanese or Korean.
At Kiva.org they found that there are certain demographics for volunteers that they are recruiting...so middle aged professionals and retirees, as well as young mothers are great, but students tend to be quite bad in terms of long-term commitment.
Community is important. Even on Amazon Mechanical Turk people are forming communities on turkernation to discuss the work they do and other related matters. Giving people a good sense of community is very good.
Giving people a humanitarian reason for participating is great. Burch, when collecting data for Pakistani SMS translation, told people what the information would be used to help with disaster relief in Pakistan, and people seemed to really respond on a personal level to the work.
To what extent can Games with a purpose be used to motivate people for language translation tasks?
- thinking in terms of Re-Capture and Image labeling task
- Educational or fun game for translation? What are the annotations or other information that can be generated quickly and be quite useful?
- there has been work done (for example, CrowdFlower) with putting tasks into games like Farmville
- the payments here are in fake currency related to the games, so the game makers get money from the people posting the tasks but don't have to worry about passing real money to players. This allows people in other countries to participate more easily without needing a bank account or credit card.
