Wednesday, March 16, 2016

Watson IoT Platform and data storage

In December 2015, IBM opened new unit Watson Internet of Things, with its global headquarter in Munich, Germany. The core of the the new portfolio is Watson IoT Platform, formerly known as IBM IOT Foundation (IOTF).

Cloud data management model
Watson IoT Platform is handy way to connect IoT devices into the cloud. It provides device connectivity with MQTT - the standardized IoT protocol ISO/IEC 20922. Device management, token authentication, data storage and integration interfaces for further cloud applications.

Here comes the catch: Even if IOTF provides data storage, it should not considered as primary storage of your device data in production use. There are two main reasons:
  • Storing excess amount of data in IOTF is expensive
  • IOTF database does not provide QoS, data may be lost if delivered at high rate
So, what instead? Better way is to use IOTF only as an interface to cloud for your connected devices, and store your data in some No-SQL database like Mongo, Couch, Cloudant or Cassandra. This is much more robust solution as IOTF does provide QoS for data delivered.

Majority of device data generated in the world is deprecated just few seconds after it's created. Data aggregation may significantly reduce the amount of data that really needs to be stored for later use. However, if you wish to store all raw device data for later use like analytics not invented yet, it's better to use some long term data storage solution like Object storage, as data over there costs only a fraction of what it would cost to store it in the active No-SQL database.

This creates a three-tier data management model where Watson IOT Platform (IOTF) is the front-line to receive the data from devices. No-SQL database is the place where active application data is processed, and finally a Object storage or similar place is the attic for historical data perhaps never needed, but nobody dares to throw away.

Bluemix and Watson IoT Platform are tightly integrated and it's very easy to access the data in Bluemix application. Here is an example of simplest possible glue-logic to integrate IOTF and No-SQL database with each other ,Cloudant in this case. The example is implemented with Node-RED running in Bluemix Node.js application.

Integration of IOTF and Cloudant with Node-RED app in Bluemix.
The left-hand side IOTF node receives and data arriving in IOTF that meets the filtering criteria defined inside the IOTF configuration. The right-hand side cloudant node puts any data it receives into the defined cloudant database. If you need to do any filtering or data aggregation prior putting data into the database, just add a function node in between. If both IOTF and cloudant service instances are created inside the same Bluemix app, there is no need to configure any credentials here, those services are readily integrated.

In addition to being more robust than using IOTF database, this may be more economic as well. Free plan of IOTF provides storage space of 1GB for device data, where as Cloudant offers 20GB free quota per instance. Fetching data from Non-SQL database is also more comfortable with efficient query algorithms. Actually the REST API of IOTF only provides max 200 items at a time.

Reading data from Cloudant is as simple as writing it in, in Node-RED application. Here is an example of simplest possible implementation of REST API to fetch data from the database.

Implementation of REST API to fetch data from Cloudant in Node-RED.
With Watson IOT Platform, Bluemix, and Cloudant it's possible to build very quickly a robust, efficient, and cost effective IOT-solutions in cloud. The integration of IOTF and Cloudant overcomes all the pitfalls the current Watson IOT Platform has. 

19 comments:

  1. Great post Jaakko (as always). This "3 tier storage" architecture is exactly what I see in use with our customers. If serious analytics are required, the Cloudant DB is often replaced with DashDB or even a spark-based service.

    ReplyDelete
    Replies
    1. ★★HIRE A LEGIT HACKER★

      Are you Seeking for the Best Legit Professional Hackers online?
      Congratulations Your search ends right here with us.

      ★COMPOSITE HACKS is a vibrant squad of dedicated online hackers maintaining the highest standards and unparalleled professionalism in every aspect.
      We Are One Of The Leading Hack Teams in The United States With So many Accolades From The IT Companies. In this online world there is no Electronic Device we cannot hack. Having years of experience in serving Clients with Professional Hacking services, we have mastered them all. You might get scammed for wrong hacking services or by fake hackers on the Internet. Don't get fooled by scammers that are advertising false hacking services via False Testimonies, and sort of Fake Write Ups.
      COMPOSITE HACKS is the Answers To your prayers.

      ★ OUR HACKING CAPABILITIES:
      We Offer Varieties Of LEGIT Hacking Services With the Help Of Our Root HackTools, Special HackTools and Our Technical Hacking Strategies Which Surpasses All Other Hackers.

      ★We Can help you Hack Or recover the password of any email, Facebook or any other Social Media accounts,
      ★Phone Hack (Which enables you to monitor your kids/wife/husband/boyfriend/girlfriend, by gaining access to everything they are doing on their phone without their notice),
      ★You Wanna Hack A Website or Database? ★You wanna Clear your Criminal Records?? ★You wanna Track Someone’s Location?

      Our Team accepts all types of hacking orders and delivers assured results to alleviate your agonies and anxieties.

      * SOME OTHER SPECIAL SERVICES WE OFFER INCLUDES:
      ★ Bank Accounts Loading ( Only USA & UK Banks)
      ★ Credit Cards Loading (Only USA & UK CC's)
      ★ Bitcoin Mining ⛏

      ★ You can also contact us for other Cyber Attacks And Hijackings, we do almost All★

      ★We have a trained team of seasoned professionals under various skillsets when it comes to online hacking services. Our company in fact houses a separate group of specialists who are productively focussed and established authorities in different platforms. They hail from a proven track record Called “HackerOne” and have cracked even the toughest of barriers to intrude and capture or recapture all relevant data needed by our Clients. Some Of These Specialist Includes PETER YAWORSKI, FRANS ROSEN, JACK CABLE, JOBERT ABMA, ARNE SWINNEN And More

      ★COMPOSITE HACKS understands your requirements to hire a professional hacker and can perceive what actually threatens you and risk your business, relationships or even life. We are 100% trusted professional hacking Organization and keep your deal entirely confidential. We are aware of the hazards involved. Our team under no circumstances disclose information to any third party. The core values adhered by our firm is based on trust and faith. Our expert hacking online Organization supports you on time and reply to any query related to the unique services we offer
      Time also depends on what exactly you want to hack and how serious you are. Enough time with social engineering is required for hacking. So if you want to bind us in a short time, then just DONT contact us because we can't hack within 10-15minutes, *sorry*.
      I have seen FAKE HACKERS claiming they can hack in 10min , but there is no REAL HACKER who can say this (AVOID THEM)

      ★COMPOSITE HACKS is available for customer care 24/7, all day and night. We understand that your request might be urgent, so we have a separate team of allocated hackers who interact with our Clients round the clock. You are with the right people so just get started.

      ★CONTACT:
      E-mail: compositehacks@gmail.com
      Hire a Hacker!
      Want faster service? Contact us!
      HackerOne©️LLC 2018.
      All Rights Reserved ®️

      ★We Treat Every Request With Utmost Confidentiality★ 🙌🏽

      Delete
    2. I am here to testify about how total company change my life,it was like this,i read about online trade and investment,when searching on how to trade with legit company,i ment a review which said i can invest in oil and gas company,where i can invest $5000 USD into oil and gast ,which i did after reaching total company,out end of my trade,i make a profit worth $65,0000 USD after investing with now through out thirty one days ,it real great i ment this company which change my life,i said to myself,it is better to share this on topix and forum,so that financial problem can reduce,what have your government offer to reduce financial problem,it is better you help yourself,because the system of the countries are not after our dealy bread,you can invest and get rich too,email total company at: total.company@aol.com

      Delete
  2. rfid tags Hey what a brilliant post I have come a cross and believe me I have been searching out for this similar kind of post for past a week and hardly came across this.

    ReplyDelete
  3. After reading your post. I am able to define this information in the dissertation writing services data storage is the place where data is held in an electromagnetic or optical form for access by a computer processor. I will do more research on the types of data storage.

    ReplyDelete
  4. An author must have a vast knowledge of vocabulary. The dictionary of a writer must be full of new english vocabulary to make their work more attractive. Use of new words makes their work more valuable and graceful.QNAP TS ECx80 R2 Series

    ReplyDelete
  5. • Nice and good article. It is very useful for me to learn and understand easily. Thanks for sharing your valuable information and time. Please keep updating IOT Online Training

    ReplyDelete
  6. Thankful to you for this amazing information sharing with us. Get website designing and development services by Ogen Infosystem.
    SEO Service in Delhi

    ReplyDelete
  7. Your content is really awesome and understandable, thanks for the efforts in this blog. Visit Mutual Fund Wala for Mutual Fund Schemes.
    Mutual Fund Distributor

    ReplyDelete
  8. Visit lifestyle magazine for Fashion, Dining, Health, Beauty and Culture parties in India.
    Lifestyle Magazine

    ReplyDelete
  9. Nice blog, keep more updates about this related blog. Visit Kalakutir Pvt Ltd for Floor Marking Paint, Warehouse Zebra Painting and Caution & Indication Signage’s.
    Caution & Indication Signages

    ReplyDelete
  10. I am here to testify about how total company change my life,it was like this,i read about online trade and investment,when searching on how to trade with legit company,i ment a review which said i can invest in oil and gas company,where i can invest $5000 USD into oil and gast ,which i did after reaching total company,out end of my trade,i make a profit worth $65,0000 USD after investing with now through out thirty one days ,it real great i ment this company which change my life,i said to myself,it is better to share this on topix and forum,so that financial problem can reduce,what have your government offer to reduce financial problem,it is better you help yourself,because the system of the countries are not after our dealy bread,you can invest and get rich too,email total company at: total.company@aol.com

    ReplyDelete
  11. Very good and interesting article, really helpful for the confirmation, I really like to visit this site because it provides very useful information.

    Accident Attorney Fort Lauderdale

    ReplyDelete
  12. Use this diet hack to drop 2 lb of fat in just 8 hours

    Well over 160k men and women are utilizing a simple and secret "liquid hack" to drop 1-2lbs every night while they sleep.

    It's painless and works every time.

    Just follow these easy step:

    1) Take a clear glass and fill it up with water half full

    2) Then follow this strange hack

    and you'll become 1-2lbs lighter the very next day!

    ReplyDelete
  13. Your article is very interesting.
    Want to make your own website!!!
    Contact Essencesoftwares the best website development company in Gurgaon.
    Essence Softwares provides: Website Development Services, App Development Services, Digital Marketing Services, IoT Solutions Services and Startup Consulting Services.
    Web Development Company in India
    App Development Company in India
    Web Development Company in Edinburgh
    App Development Company in Edinburgh
    Web Development Company in Gurgaon
    IOT Services Company in Gurgaon

    ReplyDelete
  14. putlockerss.info we have provide the best fridge repair service.

    ReplyDelete
  15. Very nice this blog . this blog is very informative.such a great blogging website.

    "woman wine expert in usa
    "

    ReplyDelete
  16. WORK WITH A HACKER

    When it comes to investing in general, the old proverb "if something sounds too good to be true, then it probably is" still holds true and should be kept in mind. This holds particularly true for digital currency. I invested a total of $280,000 since I saw that many profit-seeking speculators go to deceptive websites that offer so-called guaranteed returns or other arrangements that need investors to deposit big sums of money in order to receive even larger assured returns.
    Unfortunately, these false promises resulted in financial ruin when I tried to withdraw my money and discovered that I couldn't. To make a long tale short, I informed my husband, who then employed Captain Jack Crypto, a hacker, to help. Fortunately, we were able to restore what I've lost with the wizard's assistance. If you are interested in making an online investment, I advise you to conduct a thorough examination before committing to any kind of investment. If you have already fallen victim to an investment scam and are searching for a professional hacker, I suggest that you contact Captain Jack Crypto. Here are his contact details:

    Captainjackcrypto@europe.com is the email address he uses.

    ReplyDelete