posted by user: Jscholtz0416 || 4846 views || tracked by 10 users: [display]

Metrics for Visual Analytics 2007 : InfoVis Workshop: Metrics for the Evaluation of Visual Analytics

FacebookTwitterLinkedInGoogle

Link: http://www.cs.umd.edu/hcil/InfoVisworkshop/
 
When Oct 28, 2007 - Oct 28, 2007
Where Sacramento, CA
Submission Deadline Sep 21, 2007
Notification Due Sep 30, 2007
Categories    HCI   visualization   information technology
 

Call For Papers

The workshop is a full day and is scheduled for Sunday, October 28th

The field of visual analytics is now recognized as a research area in many universities and organizations. As new fields develop ways of assessing, progress in those fields also expands. In the field of visual analytics, we are fortunate in that we already have lessons learned about evaluating visualizations. Unfortunately, these lessons still point out that this is a difficult problem. Visual analytics compound this problem by adding more dimensions: not only are we concerned with some measure of the visualizations, but we are concerned with evaluating the impact these visualizations have in helping analysts in their work. User-centered evaluations are vital in visual analytics as they contribute greatly to adoption of research software. The issues we face in developing user-centered evaluations for visual analytics are selecting:
- The task: the tradeoff is between simple tasks that are easily evaluated and developing more realistic tasks that consume more time and are much less straightforward to evaluate.
- The corresponding dataset: the same issues as above plus the issues of developing a publicly releasable dataset that resembles a realistic dataset
- The system and environment: how much does the system or environment play a role in the utility or success of the task.
- The participants: access to senior analysts or junior analysts in evaluations and ensuring that analysts are open to new technology
- Training: how much training should be provided to analysts prior evaluations or whether analysts should be paired with technologists to operate the software
- The metrics: what combination of quantitative and qualitative measures will be accepted? How can we ensure that qualitative measures are collected with and meet some rigor? How can we measure insights that were derived from the visualization and interactions with the visualization? This is especially problematic as not all analysts approach problems in the same fashion. Most importantly, what measures are most helpful to the analytic community and to the research community?

. Selected participants will receive copies of all accepted position papers. These participants will present their ideas or current research during the morning (about 10-15 minutes each). Based on the position papers and these presentations, the organizers will develop a list of possible metrics. An initial list will be distributed to the participants prior to the workshop. After all the presentations, this list will be discussed and refined by the participants.

In the afternoon session, the organizers will provide representative examples of different types of visual analytics systems (The VAST 2007 contest winners have agreed to let us use their submissions) and the workshop participants will test the list of metrics by evaluating these systems using the metrics. A discussion session will follow to identify successes and difficulties, and refine the list of metrics. The organizers will generate a report evaluating the metrics based on the participants�?? usage and the discussions.

We will produce a poster from the workshop which will be included in the poster session. The poster will focus on the metrics used during the workshop and the lessons learned for each.

We will also consider a joint journal paper or future conference paper with the workshop participants contributing to the various metrics proposed by the workshop.


Submission of position papers:
Submissions should be no longer than 4 pages and should focus on metrics and methods for evaluating visual analysis environments. If participants have used these methods already, please include lessons learned and references. If the proposal has not yet been tried, please provide some estimates of the efforts that would be needed to implement these. Position papers should be submitted to the organizers (see e-mail addresses below) no later than September 15th. Please see http://www.cs.umd.edu/hcil/InfoVisworkshop/
for details - posted in early September. Participants will be notified of acceptance no later than September 30th.


Organizers:
Jean Scholtz
Pacific Northwest National Laboratory, 340 Northslope Way, Rockaway Beach, OR 97136
Jean.scholtz@pnl.gov

Georges Grinstein
University of Massachusetts Lowell, Lowell MA 01854
grinstein@cs.uml.edu

Catherine Plaisant
University of Maryland, College Park, MD 20742, U.S.A.
plaisant@cs.umd.edu


Related Resources

CGVCVIP 2024   18th International Conference Computer Graphics, Visualization, Computer Vision and Image Processing
ACM-Ei/Scopus-CCISS 2024   2024 International Conference on Computing, Information Science and System (CCISS 2024)
LREC-COLING 2024   The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation
CTISC 2024   2024 6th International Conference on Advances in Computer Technology, Information Science and Communications (CTISC 2024) -EI Compendex
ISVC 2024   19th International Symposium on Visual Computing
GreeNet Symposium - SGNC 2024   15th Symposium on Green Networking and Computing (SGNC 2024)
DaWaK 2024   26th International Conference on Big Data Analytics and Knowledge Discovery
DSIT 2024   2024 7th International Conference on Data Science and Information Technology (DSIT 2024)
CVIV 2024   2024 6th International Conference on Advances in Computer Vision, Image and Virtualization (CVIV 2024) -EI Compendex
AVI 2024   International Conference on Advanced Visual Interfaces