Internet of Things (IoT) Testing: Challenges, Tools and Testing Approach

Testing Internet of Things (IoT):

When a question such as “What are the basic necessities of life” is thrown at people, most of them would answer, “Food, Shelter, Clothing”.

But, that was the case before a century. Human beings have evolved to develop a handful of extra necessities for living. We have evolved to make our life, simpler, better, easier. 

We have stopped using switches to control lights, have stopped paying at kiosks to submit toll charges, we have been monitoring our health status smartly, tracking vehicular movements more efficiently and a lot in the list.

How do we do it now? How is it so different? 

We need to understand these first after which we can learn how to test them.

IoT Testing

What is Internet of Things (IoT)?

The IoT is the connection of vehicles, home appliances, medicinal equipment using embedded electronics, microchips etc. to collect and exchange data of a different kind is called as IoT. This technology allows the user to control devices remotely over a network.

IoT Examples

Some of the examples of IoT in real life implementation are as follows:

#1) In wearable tech:

Wearable gadgets such as Fitbit bands and Apple watches synchronize easily with the mobile devices.

IoT real examples

These help in capturing necessary information such as health, heart rate monitoring, sleeping activity etc. These also help in displaying data, notifications from the mobile devices onto them.

#2) Infrastructure and development:

With the use of an application such as CitySense, it is easier to get the real time outdoor lighting data and based on these, the street lights are switched on or off. There are also various applications so as to control traffic signals and parking availability in a sophisticated city layout.

#3) Healthcare:

There are numerous applications so as to monitor the health conditions of the patients.

Based on the benchmarked data, the services control the dosage of medicine at different times in a day. There are applications such as UroSense which can monitor the fluid levels in the patient’s body and based on the need can initiate the fluid transfer. At the same time, the data can be transmitted wirelessly to various stakeholders.

Technology used in IoT

Following are the few of the most used technologies in IoT:

  1. RFID [Radio Frequency Code] tags and EPC [Electronic Product Code]
  2. NFC [Near Field Communication] is used to enable two-way interactions between the electronic devices. This is basically for the smartphones and is mostly used to do the contactless payment transactions.
  3. Bluetooth: This is used where short range communications are enough to get away with the problem. This is mostly used in wearable technologies.
  4. Z-Wave: This is a low power RF comm technology. This is primarily used for home automation, lamp controlling etc.
  5. WiFi: This is the most commonly used choice for IoT. When on a LAN, this helps in transferring files, data and messages seamlessly.

Testing IoT

Let’s take an example of a Medical healthcare tracking system in which the instrument monitors the health, heart rate, fluid intake details and sends out a report to the physicians. That data is recorded in the system and the historical data can be viewed whenever required.

The physicians can initiate drug intakes, fluid supplements based on the data. This can be triggered remotely from any of the devices [computers or mobile devices] to which the medical device is connected to.

Testing IoT

Now to test such an architecture, we need to go through multiple testing approaches such as;

IoT Test Approaches

#1) Usability:

  • We need to make sure the usability of each of the device used here.
  • The medical healthcare tracking device used should be portable enough to be moved into different segments of the medical.
  • The equipment should be smart enough to push not only the notifications but also the error messages, warnings etc.
  • The system should have an option to log all the events to provide clarity to the end users. If it is not capable of doing that, the system should push those as well to a database to store it.
  • The notifications should be shown and handling of the display should be done properly in the devices [computers/mobile devices].
  • Usability in terms of displaying data, processing data, pushing job tasks from the devices should be tested thoroughly.

=> Read more about general Usability testing here

#2) IoT Security:

  • IoT Security challenges: IoT is data centric where all the devices/system connected operate based on the data that is available.
  • When it comes to the data flow between devices, there is always a chance that the data can be accessed or read when getting transferred.
  • From a testing standpoint, we need to check if the data is protected/encrypted when getting transferred from one device to the other.
  • Wherever, there is an UI, we need to make sure there is a password protection on it.

=> Read more about general Security Testing here

#3) Connectivity:

  • As it is a healthcare solution, connectivity plays a vital role.
  • The system has to be available all the time and should have seamless connectivity with the stakeholders.
  • As per connectivity, two things are very important to test;
    • Connectivity, transfer of data, receiving job tasks from the devices should be seamless when the connection is UP and running.
    • The other condition is the connection down scenario. Doesn’t matter how robust is the system and the network, there are chances that the system will go offline. Being a tester, we should test the offline conditions as well. Once the system is not available on the network, there has to be an alert which can prompt the physicians so that they can start to monitor the health conditions manually not depending on the system till it is up. On the other hand, there has to be a mechanism in the system which can store all the data in it during the offline period. Once the system comes online, all that data should get propagated. Data loss should not be there in any condition.

#4) Performance:


  • When we are talking about a system for a healthcare domain, we need to make sure the system is scalable enough for the whole hospital.
  • When the testing is carried out, it is done for 2-10 patients at a time and the data is propagated to 10-20 devices.
  • When the whole hospital is connected and 180-200 patients are connected to the system, the data that is propagated is much bigger than the tested data.
  • As testers, we need to make sure the system performs the same even though the added data is propagated.
  • We should also test the monitoring utility to display the system usage, power usage, temperature etc.

#5) Compatibility Testing:

  • Looking at the complex architecture of an IoT system, compatibility testing is a must.
  • Testing items such as, multiple operating system versions, browser types and respective versions, generations of devices, communication modes [For e.g. Bluetooth 2.0, 3.0] is necessary for IoT compatibility testing.

=> Read more about general Security Testing here

#6) Pilot Testing:

  • As far as the IoT is concerned, Pilot testing is a must.
  • Only testing in Lab makes sure the product/system works fine. But, this may backfire badly when exposed to real-time conditions/steps/scenarios.
  • During pilot testing, the system is exposed to a limited number of users in the real field. They use the application and give feedbacks on the system.
  • These comments come in handy making the application robust enough for the production deployment.

#7) Regulatory Testing:

  • This being a healthcare system needs to pass through multiple regulatory/compliance checkpoints.
  • Think of a scenario where the product passes through all the testing steps but fails in the final compliance checklist [testing performed by regulatory body].
  • It is a better practice to get the regulatory requirements in the starting of the development cycle itself. The same should be made part of the testing checklist.
  • By doing that, we make sure the product is certified for the regulatory checklist as well.

#8) Upgrade testing:

  • IoT is a combination of multiple protocols, devices, operating systems, firmware, hardware, networking layers etc.
  • When an upgrade is performed, be it for the system or for any of the involved items as stated above, thorough regression testing should be carried out/strategy should be adopted so as overcome upgrade related issues.

IoT Testing Challenges

IoT Challenges

The challenges a tester faces in IoT are as follows:

#1) Hardware-Software Mesh

IoT is an architecture, which is closely coupled among various hardware and software components. It is not only the software applications that makes the system but also the hardware ones, sensors, communication gateways etc. too play a vital role.

Only functionality testing does not help in completely certifying the system. There is always a dependency on each other in terms of the environment, data transfer etc. So, it becomes a tedious job as compared to testing a generic system [only software/hardware component].

#2) Device Interaction module

As this is an architecture between different set(s) of hardware and software, it becomes mandatory that they talk to each other in real time/near real time. When they both integrate with each other, things such as security, backward compatibility, upgrade issues becomes a challenge for the testing team.

#3) Real-time data testing

As we have discussed earlier that a Pilot testing/regulatory testing is mandatory for a system such as this, it also becomes very tough to get such data.

Being in testing team, getting regulatory checkpoints or getting the system deployed in the pilot is very tough. The step becomes even tougher if the system is related to Healthcare as per our example. So, that stays as a big challenge for the testing team.

#4) UI

The IoT is spread across devices belonging to every platform [iOS, Android, Windows, linux]. Now, testing that out on devices can be done but testing it on all possible devices is almost impossible.

We cannot omit the possibility of the UI being accessed from a device which we don’t possess or simulate. That’s a challenge which is tough to overcome.

#5) Network availability

Network connection plays a vital role as IoT is all about the data being communicated in faster speeds all the time. IoT architecture has to be tested in all kinds of network connectivity/speeds.

To test this, virtual network simulators are mostly used to vary the network load, connectivity, stability etc. But, real time data/network is always a new scenario and testing team doesn’t know where the bottleneck would develop in the long run.

IoT Testing Tools


There are various tools which are used during testing IoT systems.

They can be classified based on the target and are given as below:

#1) Software:

  1. Wireshark: This is an Open source application used to monitor the traffic in the interface, source/destination host addresses etc.
  2. Tcpdump: This does a similar job that of the Wireshark except, this does not have a GUI. This is a command line based utility which helps the user in displaying the TCP/IP and other packets that are transmitted or received over a network.

#2) Hardware:

  1. JTAG Dongle: This is similar to a debugger in PC applications. This helps in debugging the target platform code and show variable step by step.
  2. Digital Storage Oscilloscope: This is used to check various events with time stamps, glitches in power supply, signal integrity check.
  3. Software Defined Radio: This is used to emulate receiver and transmitter for a large range of wireless gateways.

To the developing world around us, IoT is a growing market and has a lot of opportunities. The time is not far when IoT becomes essential for the testers to survive in the development world.

The IoT-enabled gadget, smart device application, and communication module play a vital role in studying and evaluating the performance and behavior of various IoT services.

Poor design of IoT-enabled devices and services can hamper the correct functioning of the application and in turn negatively affect the end-user experience.

To Summarize

IoT testing approach can be different based on the system/architecture involved. Testers should concentrate more on the Test-As-A-User [TAAS] approach rather than testing based on the requirements.

One more major player in IoT testing is the Integration testing. IoT is successful if the Integration test plan is accurate and robust enough to catch flaws in the system.

IOT testing may be a tough/challenging job but, it is also very exciting as well for the testing team to certify such a complicated mesh of devices, protocols, hardware, operation systems, firmware etc.

About the author: This is a guest post by Subhasis. He is working as a team lead with over 8 years of corporate experience working for Fortune 500 IT companies in the field of Software Quality Assurance, Software Development and Testing experience.

Post your IoT queries/comments below.

Happy testing!

Recommended reading


#1 Aman Gupta

Very useful tutorial on IOT testing.
please can you elaborate is there is good career in this?

#2 Danh Do

Thanks a lot.

#3 ganesh

Hi, IOT involves Manual or Automation ?

If both required, Upto which extent those are used?

Specify in %.

#4 Gaurav Khanna

Fantastic Article…!!

#5 md rahman

Awesome ! There is a real time information in this topics.
Thanks to Share With Us.

#6 Anshu Khandelwal

Hi ,

Really informative article.I am assigned to test an IOT wearable for healthcare sector.I would like to know more about how exactly bluetooth testing can happen in that scenarios.Also we are putting data in cloud so how to build test harness for such a Application which has wearable,Mobile Apps,Web portal.How to define a test strategy for same.

Really appreciate your response.

#7 Vivek

Informative content related to IoT.

#8 Phuong

Very helpful, Thanks alot!

#9 naga sekhar

STH is given in-detail about IOT testing including many options which are invlived in this. Thanks a lot.

#10 Subhasis

@Aman: Analysts predict the increase in market for IoT is around 20-25% in next 5 years. That makes IoT a good career prospect!
@Ganesh: IoT can involve automation along with manual when needed. Strictly based on the application and implementation. Looking at our example, automation can be carried on for the Apps/Web based implementation. However, automation on medical devices can be tricky!
@Anshu: For Bluetooth testing, you can look into PTS. You would need a PTS dongle for this. For your scenario, start treating all the items [Mobile Apps, Wearable, Web Portal] individually and then think of the integration.

#11 Wendy Hebert

You may want to correct the section for #3) Healthcare. You have duplicated some of the sentences from paragraph one.

#3) Healthcare:

There are numerous applications so as to monitor the health conditions of the patients. Based on the benchmarked data, the services control the dosage of medicine at different times in a day. There are applications such as UroSense which can monitor the fluid levels in the patient’s body and based on the need can initiate the fluid transfer. At the same time, the data can be transmitted wirelessly to various stakeholders.

Based on the benchmarked data, the services control the dosage of medicine at different times in a day. There are applications such as UroSense which can monitor the fluid levels in the patient’s body and based on the need can initiate the fluid transfer. At the same time, the data can be transmitted wirelessly to various stakeholders.

#12 Subhasis

Thank you Wendy.

Leave a Comment