Annotating and embedding provenance in science data repositories to enable next generation science applications

Printer-friendly version

Abstract:

Recognizing the increased need for knowledge provenance in interdisciplinary eScience efforts, we have begun an effort to enhance a real-world data production pipeline and the resulting data services with semantic provenance. This work designing and implementing in an existing fielded virtual observatory setting has allowed us to collect key provenance requirements for a broad variety of end users. We have documented several image data pipelines for solar physics instruments at the Mauna Loa Solar Observatory and have documented almost 20 use cases covering usage from instrument scientists, observers, data analysts and managers, and end-user scientists. These use cases have guided our work developing an initial infrastructure that can be searched, queried, or browsed by these users. We use a multi-stage approach to provenance as data and information artifacts progress along processing pipelines. Our motivation, is that both the qualitative and quantitative measures of uncertainty may be vastly improved when treated in an end-to-end manner. This also reduces the likelihood that critical information is left behind or obscurely represented, making the later use of the data and information difficult or impossible. Another motivation is that provenance captured consistently at ingest time supports transparency of sources and propagation of credit for data generation, thereby increasing the likelihood of contribution and reuse. We present the current stages of implementation of our provenance infrastructure, tools and impact on what users are able to learn from the annotated information streams. The Semantic Provenance Capture in Data Ingest

History

DateCreated ByLink
July 18, 2011
14:34:19
Stephan ZednikDownload

Related Projects:

Inference Web Project LogoInference Web
Principal Investigator: Deborah L. McGuinness
Description: The Inference Web is a Semantic Web based knowledge provenance infrastructure that supports interoperable explanations of sources, assumptions, learned information, and answers as an enabler for trust. Provenance - if users (humans and agents) are to use and integrate data from unknown, uncertain, or multiple sources, they need provenance metadata for evaluation Interoperability - more systems are using varied sources and multiple information manipulation engines, thus increasing interoperability requirements Explanation/Justification - if information has been manipulated (i.e., by sound deduction or by heuristic processes), information manipulation trace information should be available Trust - if some sources are more trustworthy than others, trust ratings are desired The Inference Web consists of two important components: Proof Markup Language (PML) Ontology - Semantic Web based representation for exchanging explanations including provenance information - annotating the sources of knowledge justification information - annotating the steps for deriving the conclusions or executing workflows trust information - annotating trustworthiness assertions about knowledge and sources IW Toolkit - Web-based and standalone tools that facilitate human users to browse, debug, explain, and abstract the knowledge encoded in PML.
SPCDIS Project LogoSemantic Provenance Capture in Data Ingest Systems (SPCDIS)
Principal Investigator: Peter Fox
Co Investigator: Deborah L. McGuinness
Description: The goal of this project is to develop at the RPI Tetherless World Constellation, based within the NCAR High Altitude Observatory and in collaboration with the University of Texas at El Paso, the University of Michigan and McGuinness Associates a semantically-enabled data ingest capability.
DCO-DS LogoVirtual Solar Terrestrial Observatory (VSTO)
Principal Investigator: Peter Fox
Co Investigator: Deborah L. McGuinness
Description: VSTO is a collaborative project between the High Altitude Observatory and Scientific Computing Division of the National Center for Atmospheric Research and McGuinness Associates. VSTO is funded by a grant from the National Science Foundation, Computer and Information Science and Engineering (CISE) in the Shared Cyberinfrastructure (SCI) division.

Related Research Areas:

Knowledge Provenance
Lead Professor: Deborah L. McGuinness
Description: Knowledge Provenance
Concepts: Provenance,
Semantic eScience
Lead Professor: Peter Fox
Description:
Science has fully entered a new mode of operation. E-science, defined as a combination of science, informatics, computer science, cyberinfrastructure and information technology is changing the way all of these disciplines do both their individual and collaborative work.
As semantic technologies have been gaining momentum in various e-Science areas (for example, W3C's new interest group for semantic web health care and life science), it is important to offer semantic-based methodologies, tools, middleware to facilitate scientific knowledge modeling, logical-based hypothesis checking, semantic data integration and application composition, integrated knowledge discovery and data analyzing for different e-Science applications.
Partially influenced by the Artificial Intelligence community, the Semantic Web researchers have largely focused on formal aspects of semantic representation languages or general-purpose semantic application development, with inadequate consideration of requirements from specific science areas. On the other hand, general science researchers are growing ever more dependent on the web, but they have no coherent agenda for exploring the emerging trends on the semantic web technologies. It urgently requires the development of a multi-disciplinary field to foster the growth and development of e-Science applications based on the semantic technologies and related knowledge-based approaches.

Concepts: eScience
X-informatics
Lead Professor: Peter Fox
Description: In the last 2-3 years, Informatics has attained greater visibility across a broad range of disciplines, especially in light of great successes in bio- and biomedical-informatics and significant challenges in the explosion of data and information resources. Xinformatics is intended to provide both the common informatics knowledge as well as how it is implemented in specific disciplines, e.g. X=astro, geo, chem, etc. Informatics' theoretical basis arises from information science, cognitive science, social science, library science as well as computer science. As such, it aggregates these studies and adds both the practice of information processing, and the engineering of information systems.
Concepts: , eScience