posted by organizer: dustalov || 1127 views || tracked by 3 users: [display]

VLDB Crowd Science Workshop 2021 : Trust, Ethics, and Excellence in Crowdsourced Data Management at Scale


When Aug 20, 2021 - Aug 20, 2021
Where Hybrid / Copenhagen, Denmark
Submission Deadline Jul 5, 2021
Notification Due Jul 19, 2021
Final Version Due Aug 2, 2021
Categories    crowdsourcing   data excellence   data management   crowd science

Call For Papers

At VLDB 2021 Crowd Science Workshop experts in the field will meet together to present their work and discuss the future of the field. Submit your paper to join us at VLDB 2021!

Crowdsourcing has become a standard tool to obtain large amounts of carefully annotated data. Nowadays, its applicability has exceeded the classical purpose of collecting data to train AI algorithms. Indeed, crowdsourcing is often used to conduct research studies and to gain valuable insights from public.

The focus of this workshop is on the best practices of efficient and trustworthy crowdsourcing.

We welcome all submissions in the broad area of crowdsourcing: research papers that present new results, vision papers that discuss the future of the area, industry papers that describe an interesting use case, and any other papers in the area.

# Invited Speakers

- Michael Bernstein (Stanford University)
- Wuraola Fisayo Oyewusi (Data Science Nigeria)
- Ujwal Gadiraju (Delft University of Technology)

# Focus Areas

The list of focus areas is not exclusive and works on other aspects of crowdsourcing are also welcome!

We identify three key areas:

1. Large-Scale Data Excellence

Data is a crux of crowdsourcing. On the one hand, requesters want to annotate their data without compromising its privacy. On the other hand, workers produce large amounts of data that platforms can utilize to improve their services. Data-management practices are at the heart of our workshop and we welcome submissions in this area.

2. Trust and Ethics

Crowdsourcing is a multi-agent system in which hundreds of requesters interact with thousands of workers through the interface of the platform. To ensure that the system progresses in a fair and efficient manner, it is critical that all parties trust each other and adhere to certain ethical standards. We hope to discuss these standards and critically analyze the current state of affairs in our workshop.

3. Crowd-AI Interplay

Crowdsourcing is an important source of data for AI algorithms. But do we understand how does the crowd impact the development of AI? Human decision-making is known to be susceptible to various problems, including noise, bias, subjectivity, and miscalibration. These issues can adversely impact the properties of downstream AI methods that rely on crowdsourced data. Understanding and preventing such undesirable artifacts is a key goal for the whole crowdsourcing community and we will focus on this problem in our workshop.

# Guidelines and Submission

Find format guidelines and submit a paper!

A paper should be 5–15 pages. These limits are for main content pages, including all figures and tables. Additional pages containing appendices, acknowledgements, funding disclosures, and references are allowed.

Submission link:

Time zone for all deadlines is AoE (UTC-12).

# VLDB 2021 Crowd Science Challenge

In conjunction with the workshop, we organize a shared task on aggregation of crowdsourced texts. Consider participating and submit your report to the workshop!


# Organizers

- Daria Baidakova (Toloka)
- Fabio Casati (ServiceNow)
- Alexey Drutsa (Toloka)
- Nikita Pavlichenko (Toloka)
- Ivan Stelmakh (Carnegie Mellon University)
- Dmitry Ustalov (Toloka)

Related Resources

Sensors (MDPI) SI 2022   Ethics, Privacy and Trust in IoT-Based Smart Homes and Buildings
DBKDA 2023   The Fifteenth International Conference on Advances in Databases, Knowledge, and Data Applications
DSDM @ VLDB 2022   Workshop on Data Science for Data Marketplaces @ VLDB 2022
CPS AIBT 2023   CPS--2023 The 2nd International Conference on Artificial Intelligence and Blockchain Technology (AIBT 2023)
IEEE ICA 2022   The 6th IEEE International Conference on Agents
IEEE ICISPP 2023   IEEE--2023 4th International Conference on Information Security and Privacy Protection (ICISPP 2023)
IEEE CCNC STP-CPS 2023   5th International Workshop on Security Trust Privacy for Cyber-Physical Systems (STP-CPS'23) with IEEE CCNC 2023, 8-11 January, Las Vegas, NV, USA
ICIM 2023   IEEE--2023 the 9th International Conference on Information Management (ICIM 2023)
FLAIRS-36 ST XAI, Bias, and Trust 2023   FLAIRS Special Track on Explainability, Bias, and Trust in Artificial Intelligence
ICSD 2023   11th International Conference on Sustainable Development, 6 - 7 September Rome, Italy