Building distributed applications is typically a complex task. When building IoT type applications, the scope of distributing the application is not just limited to data center but across data centers, gateways, routers, edge devices. Building an application that spans this broad scope can become very daunting. One of the key requirements when building an IoT type application is the ability to converge OT and IT. See my presentation on this subject here.
Converging OT and IT is typically a streaming domain problem where data needs to be ingested, processed, analyzed and actions taken and then forwarded to the next stage. The computation graph can typically resemble a hierarchical computation model where enormous raw data is generated at multitude sources and decisions will need to be made at the so called edges which could be gateways or routers or micro-data centers and insights forward to the next stage.
Hitachi realized the complexity of convergence of OT and IT as they had built use cases across various vertical industries (see the HSDP FAQ blog for use case links) and during the process carefully documented the challenges of building such applications. Since the key aspect of the solution involved processing massive volumes and variety of streams of data Hitachi re-architected their streaming platform, HSDP, which was primarily focused on centralized stream processing (e.g. Finance, Banking) to address the requirements for building a OT-IT convergence solution.
HSDP's - Code Centrally and Distributed globally - is an unique solution architecture where the solution developer can build and test a complex geo-distributed solution and then hand it over to a deployment engineer to deploy the solution artifacts globally. Some of the key advantages with this solution architecture are:
- Solution developer can focus on building the processing and analysis that is required at the edges and core without having to worry about how to cascade insights from edge to core.
- After building and testing the solution the solution developer can use the CQL directives and determine based on the customer requirements which analysis needs to be done where (edge, core and anywhere in-between) and generate the artifacts.
- The deployment engineer can then take these artifacts, deploy them on the global solution network.
Using CQL as the primary language and simple directives, someone who is familiar with programming with SQL can build a geo-distributed solution.
Check out the HSDP Detailed Introduction and test drive the hands on tutorials to get an idea on how simple building a geo-distributed solution can be.