posted by user: bnaderi || 2133 views || tracked by 2 users: [display]

QoMEX-CS 2019 : QoMEX 2019 Special Session: Quality of Experience and User Experience Research with and for Crowdsourcing


When Jun 5, 2019 - Jun 7, 2019
Where Berlin, Germany
Abstract Registration Due Jan 18, 2019
Submission Deadline Jan 22, 2019
Notification Due Feb 26, 2019
Categories    quality of experince   crowdsourcing   user experince   crowd

Call For Papers

QoMEX 2019 - Berlin, Germany
Call for paper: submission for QoMEX 2019 Special Session on Quality of Experience and User Experience Research with and for Crowdsourcing

Abstracts submission deadline: January 18, 2019
Paper submission deadline: January 22, 2019
Special Session page:

Motivation and objectives

Crowdsourcing has become a valuable tool in Quality of Experience (QoE) and User Experience (UX) research. In contrast to lab-based studies, crowdsourcing enables researchers to easily and quickly collect a large number of subjective ratings from a diverse set of participants. This diversity further enables researching cultural effects, influence factors generated by different end-user devices, and effects of different surrounding environments. However, these opportunities come at costs. Experimenters have only limited control of the test settings and in which environmental conditions the study takes place. Additionally, the remote test procedure can be error-prone as the test participants are not under the direct supervision of the test conductor.

In this context, the aim of the special session is twofold. On the one hand, we want to foster contributions following the traditional way optimizing and designing crowdsourced subjective studies for Quality of Experience and User Experience research. This includes novel methodologies for quality assurance and replicability, new fields of application like assessing the QoE of IoT devices in a crowdsources fashion, and using new technologies like wearable to collect additional environmental and user signals. On the other hand, we also want to raise awareness and foster research in a new research direction, the Quality of Experience and User Experience of crowdsourcing workers. Crowdsourcing has become mature in the academic and business usage, and much effort is put into a (cost-) efficient and quality optimizing the design of the tasks. However, no or only very little efforts are made to improve the working experience of the workers.

Topics of interest
We invite you to submit original papers in any area related, including but not limited to:

Crowdsourcing for subjective studies
-- Novel application
-- Limitations of current crowdsourcing systems
-- Quality control mechanism / Monitoring of test and environmental conditions
-- Large scale crowdsourcing studie
Reproducibility of result
-- Cross platform studies / Repeated studies
-- Assessment of hidden influence factors / Impact of hidden influence factors
-- Bias estimation and bias reduction
-- Standardisation of crowdsourcing test methods
Usability and User Experience of crowdsourcing tasks
-- Optimization of task interfaces and task workflows
-- Relation to result quality and worker motivation
-- Enhancing workers’ UX (e.g., by means of gamification of tasks)
-- Quality of complex crowdsourcing workflows (e.g. combination of AI and Crowds)
Interconnection of crowdsourcing and lab-based tests, e.g.,
-- Studies comparing results from lab and crowdsourcing
-- Adaptations of established lab-test standards to the crowdsourcing environment

About Organizers
- Babak Naderi (Technical University of Berlin)
- Matthias Hirth (University of Wuerzburg)
- Katrien De Moor (Norwegian University of Science and Technology)

Related Resources

QoMEX 2023   15th International Conference on Quality of Multimedia Experience
CBMI 2023   20th International Conference on Content-based Multimedia Indexing
UMAP 2023   User Modeling, Adaptation, and Personalization
IUI 2023   Intelligent User Interfaces
Narnia 2023   Sons & Daughters of Narnia: Tracing CS Lewis's Literary Influence into the 21st Century
ISPEC 2023   The 18th International Conference on Information Security Practice and Experience
NLPI 2023   4th International Conference on NLP & Information Retrieval
BESC 2023   10th International Conference on Behavioural and Social Computing
CQR 2023   IEEE International Communications Quality and Reliability Workshop
WOSCx 2023   Second International Workshop on Serverless Computing Experience (WOSCx2) 2023