Title
Evaluating a scientific collaboratory: Results of a controlled experiment
Abstract
The evaluation of scientific collaboratories has lagged behind their development. Do the capabilities afforded by collaboratories outweigh their disadvantages? To evaluate a scientific collaboratory system, we conducted a repeated-measures controlled experiment that compared the outcomes and process of scientific work completed by 20 pairs of participants (upper level undergraduate science students) working face-to-face and remotely. We collected scientific outcomes (graded lab reports) to investigate the quality of scientific work, post-questionnaire data to measure the adoptability of the system, and post-interviews to understand the participants' views of doing science under both conditions. We hypothesized that study participants would be less effective, report more difficulty, and be less favorably inclined to adopt the system when collaborating remotely. Contrary to expectations, the quantitative data showed no statistically significant differences with respect to effectiveness and adoption.The qualitative data helped explain this null result: participants reported advantages and disadvantages working under both conditions and developed work-arounds to cope with the perceived disadvantages of collaborating remotely. While the data analysis produced null results, considered as a whole, the analysis leads us to conclude there is positive potential for the development and adoption of scientific collaboratory systems.
Year
DOI
Venue
2003
10.1145/772047.772051
ACM Trans. Comput.-Hum. Interact.
Keywords
Field
DocType
null result,qualitative data,post-questionnaire data,collaboration,scientific collaboratory,scientific collaboratory system,nanoscience,upper level undergraduate science,controlled experiment,data analysis,scientific outcome,scientific collaboratories,geographically-distributed work,scientific work,quantitative data,scientific research,repeated measures,information systems,statistical significance,evaluation
Information system,Data science,Collaboratory,Null result,Qualitative property,Computer science,Human–computer interaction,Controlled experiment,Scientific method
Journal
Volume
Issue
Citations 
10
2
37
PageRank 
References 
Authors
2.66
24
3
Name
Order
Citations
PageRank
Diane H. Sonnenwald149440.41
Mary C. Whitton21225110.86
Kelly Maglaughlin317012.21