Best Practices for Integrating IoT Data with Knowledge Graphs

Posted by Sam Chance on Apr 2, 2020 4:06:55 PM
Find me on:

IoTThe sheer difficulty of Internet of Things integrations presents issues for even the most accomplished data integration platforms, including modern data fabrics predicated on harmonizing data in knowledge graphs for inter-system interoperability.

The challenge is to deliver real-time integrations of data characterized by differences in structure, format, and models—many of which are highly dimensional and likely include proprietary formats—with typical relational sources. The standardization of the knowledge graph approach isn't a cure-all, but works best works when properly instituted.

Specifically, organizations must limit knowledge graphs' standardized data models (ontologies) and leverage Massively Parallel Processing (MPP). These best practices are instrumental for timely data integrations that grow with complexity as needed, effectively normalizing the IoT as just another enterprise data source.

Circumscribing Ontologies

Ontologies' advantages quickly become shortcomings if left unchecked. These models can express virtually any business requirement or data attribute, which can possibly overcomplicate models with details. Moreover, their natural mutable nature evolves to encompass new information and will include almost anything—if allowed to. Finally, the machine reasoning of the semantic standards powering these models doesn't always operate intuitively and may induce additional complexity, frustration, and disillusionment.

The first best practice for limiting ontologies is to be aware of their unrestrained potential. Organizations should reduce their descriptions to the minimum initial requirements. These typically include the bare essentials for data integrations and knowledge engineering. Ideally, organizations should employ a "middle-out" approach to building ontologies that defines a modest amount of common domain concepts, then expands to include information from the data sources. Top down approaches are too academic, while bottom up ones result in false starts.

The idea is to think big, start small, and scale fast. Perfection's not required; by starting small the data fabric grows organically because semantic standards expect change. In the Industrial Internet, for example, organizations can integrate IoT data from different devices in smart factories to monitor opportunities for optimization or predictive maintenance.

Massively Parallel Processing

A second best practice is to employ MPP to cope with data volume and complexity. The computational demands for knowledge graphs deployed at the IoT's scale are considerable. The larger these semantic graphs grow, the more computationally intensive they become. In fact, some can become so large that they decrease the quantities of data that can be loaded and queried, resulting in performance compromises and decreased value.

These issues are redressed by utilizing MPP. The near ubiquity of MPP is a direct consequence of the big data age, and all but standard in today's post big data age. When working with big data for IoT deployments, MPP is essential to account for the high speeds and data volumes that are routinely processed for swift integrations. MPP is critical to avoiding the latency and performance issues of knowledge graphs at IoT scale.

Moreover, it's a relatively inexpensive means of ensuring organizations get the rapid processing required for real-time IoT use cases. For example, using knowledge graphs to quickly integrate IoT data from remote patient monitoring systems with information in clinical facilities in healthcare can result in timely alerts and interventions to improve patient outcomes for a variety of conditions.

Harnessing the IoT

When properly implemented, the knowledge graph approach of comprehensive data fabrics makes integrating IoT data routine. The aforementioned best practice of limiting the scope of ontologies so they naturally evolve to include necessary requirements and utilizing the computational (and pricing) benefits of MPP enable organizations to profit from the IoT while eschewing its pitfalls. More importantly, these techniques will assist in democratizing the IoT, increasing its adoption rates, and making it a standard data management source in the coming decade.

Tags: IoT, Knowledge Graph

Subscribe to the Smart Data Blog!

Comment on this Blogpost!