Ph.D. Thesis Defense Announcement: Joshua Shinavier: Light, Sound, and Semantics: The Web of Data as a New Sensory Modality

Printer-friendly version


Ph.D. Thesis Defense Announcement: Joshua Shinavier: Light, Sound, and Semantics: The Web of Data as a New Sensory ModalityJuly 6, 2015
Ph.D. Thesis Defense Announcement

Light, Sound, and Semantics: The Web of Data as a New Sensory Modality

Joshua Shinavier
Department of Computer Science
Tetherless World Constellation
Thesis Advisor: Professor James A. Hendler

Tuesday, July 7, 2015
Winslow 1140 – 10:00 a.m.

This dissertation explores the use of machine-accessible knowledge about the objects in our environment to augment our perception at an immediate, preconscious level.

In everyday life, we combine simultaneous natural stimuli, such as the sound of a voice and the sight of the speaker’s moving lips, into percepts without thinking about them individually. Artificial stimuli may be combined into the same percepts if they are semantically congruent with the perceptual context and if they arrive within brief temporal windows of the natural stimuli, among other conditions. Insofar as a knowledge-based system can recognize and respond to new context quickly and appropriately enough, its feedback may offer an advantage over natural signals alone, as it allows us to draw attention to nonphysical and non-obvious properties of the world, such as abstract relationships.

In order to truly extend a person’s natural senses, we need to understand the psychophysics of semantics and perception as well as the technological challenges of building such a system. The main focus of this dissertation is on the latter set of problems. We will first translate the known perceptual constraints into a set of functional requirements, then introduce a concrete Semantic Web architecture which fulfills them. The architecture combines cooperative activity detection, a SPARQL-based complex event processor, a Linked Data client, and a body area network of sensing and feedback devices. A number of Semantic Wearable applications are provided as proofs of concept, and a simulation-based evaluation of the system is also described, illustrating the performance of the system, for non-trivial scenarios and at a significant scale, within its real-time constraints.

In this architecture, the Web of Data serves as a read-write repository of knowledge about people and objects; it is queried on demand and updated for new context with new knowledge, enabling a feedback loop of perception and interaction which is independent of any single environment.