Real Time Data with Arcoflex Sensei and Microsoft Azure

ArcoFlex Sensei – Real Time Data with Azure IoT

 

Remote monitoring seems to have two completely diametric applications: offline and therefor latent storage of data and real time data. Latent data is fine. Collect, store and forget. No big deal: easy to obtain and cheap to collect. So let’s forget about this because everyone can do it. But what about real time. How ‘real time’ can we get, what does it cost and is it genuinely possible. Fortunately, Microsoft’s Azure IoT suite can provide exactly that.

Collecting data from remote sensors poses two problems: how fast can it traverse the various components of the cloud and what will it cost? Let us review the performance issue first and then analyse its impact on cost. If it is too expensive, we may not be able to accept the performance we desire. In order to get data from a device to a visualisation, these are the elements it must traverse:

·         Code at the sensor’s manager (Raspberry PI in our case)

·         IoT Hub where completely scalable input is available

·         Event hub where normalised JSON data needs to be available

·         SQL Azure DB where trend data is being persisted

·         Visualisation applications as a user end point

Arcoflex has established that traversing this entire gamut of processes and storage mechanisms can be as short as 600ms. This means that from the moment a sensor has recorded a value or assessed an alert condition, the resulting data is creating an SMS or email or updating a visualisation within one second. So we are able to boast sub-second responsiveness and can therefor claim real time data responses. This is a big call but it is quite easy to prove and demonstrate. Our Arcoflex Sensei Protyper can do this out of the box.

Is this important? Yes, it is. Bio-security breach conditions are one example where it is required under law to prove that two doors are not open at the same time. You cannot do this if your data recording solution cannot resolve to less than a second. But there are other examples: motor burnout prevention, temperature control, fluid mixing processes, alarms… the list is actually quite endless.

The big question is of course: how? The answer comes in two parts: an innovative time and value compression algorithm from ArcoFlex and Microsoft’s Azure platform. The two components combine to yield an utterly cost effective solution to provide real time data from your remote monitoring solution. The first part is ArcoFlex’s TVDC compression algorithm. What it does is to only send data that changes, set to configurable precision and frequency metrics. Because the 3G/4G solution environment sits directly on top of the sensors, this is what controls telecommunications provider costs. Keeping this low is important to the overall cost implementation.

The difficulty with customised transmission protocols however is one of incompatibility. No-one else can read it. So what ArcoFlex does is read the compressed data into Microsoft IoT Hub, a mechanism that has enormous capacity and almost infinite scalability. But it has a cost and needs to be managed so our compressed data packet management protocol ensures that IoT Hub expenses are minimised, as well as carrier costs for the 3G/4G connection. The targeted aim is less than $10/site per month for effectively unlimited amounts of trend data.

The problem though is that nothing else will read the compressed data so the next step is to expand the data set into full JSON, something almost everyone can consume. By contrast to IoT Hub, Microsoft’s Event Hub is very cheap. So we take all the IoT Hub traffic, expand it into JSON and throw it at Event Hub. Now the data is in a format that can be consumed by regular BI and Stream Analytic tools or dumped into a SQL Azure table. Suddenly everyone has access, including our own visualisation tools.

We have net latency down to sub second and this is impressive. Bandwidth costs are contained without compromising overall system performance. If your IoT monitoring solution is not yielding genuine real time responsiveness, then it isn’t using Arcoflex Sensei.

Geoff Schaller
@ArcoflexIOT