Wednesday, March 16, 2016

Watson IoT Platform and data storage

In December 2015, IBM opened new unit Watson Internet of Things, with its global headquarter in Munich, Germany. The core of the the new portfolio is Watson IoT Platform, formerly known as IBM IOT Foundation (IOTF).

Cloud data management model
Watson IoT Platform is handy way to connect IoT devices into the cloud. It provides device connectivity with MQTT - the standardized IoT protocol ISO/IEC 20922. Device management, token authentication, data storage and integration interfaces for further cloud applications.

Here comes the catch: Even if IOTF provides data storage, it should not considered as primary storage of your device data in production use. There are two main reasons:
  • Storing excess amount of data in IOTF is expensive
  • IOTF database does not provide QoS, data may be lost if delivered at high rate
So, what instead? Better way is to use IOTF only as an interface to cloud for your connected devices, and store your data in some No-SQL database like Mongo, Couch, Cloudant or Cassandra. This is much more robust solution as IOTF does provide QoS for data delivered.

Majority of device data generated in the world is deprecated just few seconds after it's created. Data aggregation may significantly reduce the amount of data that really needs to be stored for later use. However, if you wish to store all raw device data for later use like analytics not invented yet, it's better to use some long term data storage solution like Object storage, as data over there costs only a fraction of what it would cost to store it in the active No-SQL database.

This creates a three-tier data management model where Watson IOT Platform (IOTF) is the front-line to receive the data from devices. No-SQL database is the place where active application data is processed, and finally a Object storage or similar place is the attic for historical data perhaps never needed, but nobody dares to throw away.

Bluemix and Watson IoT Platform are tightly integrated and it's very easy to access the data in Bluemix application. Here is an example of simplest possible glue-logic to integrate IOTF and No-SQL database with each other ,Cloudant in this case. The example is implemented with Node-RED running in Bluemix Node.js application.

Integration of IOTF and Cloudant with Node-RED app in Bluemix.
The left-hand side IOTF node receives and data arriving in IOTF that meets the filtering criteria defined inside the IOTF configuration. The right-hand side cloudant node puts any data it receives into the defined cloudant database. If you need to do any filtering or data aggregation prior putting data into the database, just add a function node in between. If both IOTF and cloudant service instances are created inside the same Bluemix app, there is no need to configure any credentials here, those services are readily integrated.

In addition to being more robust than using IOTF database, this may be more economic as well. Free plan of IOTF provides storage space of 1GB for device data, where as Cloudant offers 20GB free quota per instance. Fetching data from Non-SQL database is also more comfortable with efficient query algorithms. Actually the REST API of IOTF only provides max 200 items at a time.

Reading data from Cloudant is as simple as writing it in, in Node-RED application. Here is an example of simplest possible implementation of REST API to fetch data from the database.

Implementation of REST API to fetch data from Cloudant in Node-RED.
With Watson IOT Platform, Bluemix, and Cloudant it's possible to build very quickly a robust, efficient, and cost effective IOT-solutions in cloud. The integration of IOTF and Cloudant overcomes all the pitfalls the current Watson IOT Platform has. 


  1. Great post Jaakko (as always). This "3 tier storage" architecture is exactly what I see in use with our customers. If serious analytics are required, the Cloudant DB is often replaced with DashDB or even a spark-based service.


      Are you Seeking for the Best Legit Professional Hackers online?
      Congratulations Your search ends right here with us.

      ★COMPOSITE HACKS is a vibrant squad of dedicated online hackers maintaining the highest standards and unparalleled professionalism in every aspect.
      We Are One Of The Leading Hack Teams in The United States With So many Accolades From The IT Companies. In this online world there is no Electronic Device we cannot hack. Having years of experience in serving Clients with Professional Hacking services, we have mastered them all. You might get scammed for wrong hacking services or by fake hackers on the Internet. Don't get fooled by scammers that are advertising false hacking services via False Testimonies, and sort of Fake Write Ups.
      COMPOSITE HACKS is the Answers To your prayers.

      We Offer Varieties Of LEGIT Hacking Services With the Help Of Our Root HackTools, Special HackTools and Our Technical Hacking Strategies Which Surpasses All Other Hackers.

      ★We Can help you Hack Or recover the password of any email, Facebook or any other Social Media accounts,
      ★Phone Hack (Which enables you to monitor your kids/wife/husband/boyfriend/girlfriend, by gaining access to everything they are doing on their phone without their notice),
      ★You Wanna Hack A Website or Database? ★You wanna Clear your Criminal Records?? ★You wanna Track Someone’s Location?

      Our Team accepts all types of hacking orders and delivers assured results to alleviate your agonies and anxieties.

      ★ Bank Accounts Loading ( Only USA & UK Banks)
      ★ Credit Cards Loading (Only USA & UK CC's)
      ★ Bitcoin Mining ⛏

      ★ You can also contact us for other Cyber Attacks And Hijackings, we do almost All★

      ★We have a trained team of seasoned professionals under various skillsets when it comes to online hacking services. Our company in fact houses a separate group of specialists who are productively focussed and established authorities in different platforms. They hail from a proven track record Called “HackerOne” and have cracked even the toughest of barriers to intrude and capture or recapture all relevant data needed by our Clients. Some Of These Specialist Includes PETER YAWORSKI, FRANS ROSEN, JACK CABLE, JOBERT ABMA, ARNE SWINNEN And More

      ★COMPOSITE HACKS understands your requirements to hire a professional hacker and can perceive what actually threatens you and risk your business, relationships or even life. We are 100% trusted professional hacking Organization and keep your deal entirely confidential. We are aware of the hazards involved. Our team under no circumstances disclose information to any third party. The core values adhered by our firm is based on trust and faith. Our expert hacking online Organization supports you on time and reply to any query related to the unique services we offer
      Time also depends on what exactly you want to hack and how serious you are. Enough time with social engineering is required for hacking. So if you want to bind us in a short time, then just DONT contact us because we can't hack within 10-15minutes, *sorry*.
      I have seen FAKE HACKERS claiming they can hack in 10min , but there is no REAL HACKER who can say this (AVOID THEM)

      ★COMPOSITE HACKS is available for customer care 24/7, all day and night. We understand that your request might be urgent, so we have a separate team of allocated hackers who interact with our Clients round the clock. You are with the right people so just get started.

      Hire a Hacker!
      Want faster service? Contact us!
      HackerOne©️LLC 2018.
      All Rights Reserved ®️

      ★We Treat Every Request With Utmost Confidentiality★ 🙌🏽

  2. rfid tags Hey what a brilliant post I have come a cross and believe me I have been searching out for this similar kind of post for past a week and hardly came across this.

  3. After reading your post. I am able to define this information in the dissertation writing services data storage is the place where data is held in an electromagnetic or optical form for access by a computer processor. I will do more research on the types of data storage.

  4. An author must have a vast knowledge of vocabulary. The dictionary of a writer must be full of new english vocabulary to make their work more attractive. Use of new words makes their work more valuable and graceful.QNAP TS ECx80 R2 Series

  5. • Nice and good article. It is very useful for me to learn and understand easily. Thanks for sharing your valuable information and time. Please keep updating IOT Online Training

  6. myTectra Placement Portal is a Web based portal brings Potentials Employers and myTectra Candidates on a common platform for placement assistance

  7. Selenium is one of the most popular automated testing tool used to automate various types of applications. Selenium is a package of several testing tools designed in a way for to support and encourage automation testing of functional aspects of web-based applications and a wide range of browsers and platforms and for the same reason, it is referred to as a Suite.

    Selenium Interview Questions and Answers
    Javascript Interview Questions
    Human Resource (HR) Interview Questions