Ph.D. Thesis Defense Announcement for James Michaelis

Printer-friendly version

Ph.D. Thesis Defense Announcement for James MichaelisOctober 21, 2014
The Tetherless World Constellation is proud to announce the successful completion of James Michaelis' Thesis Defense

Title: Towards a Methodology for Evaluation of Provenance-Based User Interfaces

Advisors: Deborah McGuinness, Jim Hendler

Abstract: The Merriam Webster English Dictionary defines provenance as: (i) the origin or source of something; (ii) the history of ownership of a valued object or work of art or literature. In its earliest usage, the provenance of physical objects - such as pieces of artwork - could be used to make assessments of their value. In more recent times, provenance has become an increasingly critical component for assessment of data in digital systems. From the perspective of digital artifacts, the World Wide Web Consortium's (W3C) Provenance Working Group defines provenance as: a record that describes the people, institutions, entities, and activities, involved in producing, influencing, or delivering a piece of data or a thing.

Particularly in the past decade, technologies for acquiring, recording, storing and representing provenance data have grown more sophisticated. Additionally, significant interest has been expressed by stakeholders of many digital systems in expanding their usage of provenance. However, in this time, limited work has been done to develop and rigorously evaluate tools for the exploration and analysis of provenance collections - particularly, for stakeholders with limited background in database querying languages and systems.

This dissertation aims to develop and validate a methodology for gauging usability of provenance-based user interfaces, taking the position that existing tools and techniques from multidimensional data analysis can be applied toward both the design of such a methodology as well as development of novice-friendly interfaces.

To advance this dissertation, three supporting contributions are made:

The ProvAnalytics Framework: This is a novel framework for exploring provenance record collections expressed as directed graphs. The core focus of ProvAnalytics is to provide a set of approaches for converting collections of provenance graphs into multidimensional datasets for subsequent review by interested users. Additionally, functionality is provided for generating synthetic provenance collections, intended to meet the needs of particular interface evaluations.

An Analysis and Classification for Provenance Querying Tasks: Currently, no benchmarks exist for gauging the usability of provenance-based systems for end users. This contribution seeks to build on prior computational benchmarks - particularly from the Provenance Challenge series - to establish a set of user-centric querying tasks oriented toward the multidimensional analysis paradigm.

A Proof-of-Concept Comparison of Querying and Presentation Interfaces: This contribution centers on demonstrating utility of the ProvAnalytics framework using a case-study based evaluation. Three interface configurations were compared in a 36 subject study, aimed at gauging performance across three types of established information retrieval tasks. Findings from this study are intended to demonstrate: (i) that novice users of provenance systems can quickly adapt to working with the developed tasks and data representations, and (ii) that statistically meaningful relationships can be obtained in support of routine usability hypotheses.