Released: Heuristics for crowdsourcing

Increasingly, libraries, archives, museums, galleries and academic institutions are crowdsourcing the processing of cultural heritage assets to unpaid volunteers. Crowdsourcing labour-intensive processes such as transcribing historical documents, recording personal histories, tagging paintings with keywords, correcting the OCR text of digitised newspapers, and cataloguing cultural heritage collections, are better enabling these institutions to create or enhance digitised data for public use and research, and engage the wider community.

The success of crowdsourcing initiatives relies on meeting two key objectives: sufficient participation and quality contribution. Meeting these objectives requires effective project and system design and evaluation, and an understanding of contextual factors such as the type of tasks being performed, and motivation and incentives to participate.

My PhD research addresses a gap in the crowdsourcing literature, by proposing an Information Systems Design Theory for websites for crowdsourcing the processing of cultural heritage assets, which better explains the website elements and aspects of design that can influence voluntary participation and contribution quality than existing design guidance. This knowledge can be applied in design and evaluation practice using the new set of specialized heuristics (design principles) developed in the study.

These heuristics, which are ranked according to likely level of influence on voluntary participation and contribution quality, enable crowdsourcers to better focus limited time and resources on those aspects of design likely to have the most influence on website goals. Organising the new set of heuristics by user-oriented categories encourages crowdsourcing practitioners to approach website design and evaluation from the perspective of the end user (the volunteer) with a view to providing an optimal user experience.

This new set of heuristics has been released in a PhD findings report prior to thesis submission to provide crowdsourcers with a practical tool to support website design and evaluation, and to gather feedback on its usability and utility. The report includes a research overview; a summary of heuristics development, evaluation, and application; and the complete set of heuristics.

Please send questions and feedback to donelle.mckinley@vuw.ac.nz or @nonprofitcrowd