Internet of Things (IoT) Testing: Challenges, Tools and Testing Approach

Testing Internet of Things (IoT):

When a question such as “What are the basic necessities of life” is thrown at people, most of them would answer, “Food, Shelter, Clothing”.

But, that was the case before a century. Human beings have evolved to develop a handful of extra necessities for living. We have evolved to make our life, simpler, better, easier. 

We have stopped using switches to control lights, have stopped paying at kiosks to submit toll charges, we have been monitoring our health status smartly, tracking vehicular movements more efficiently and a lot in the list.

How do we do it now? How is it so different? 

We need to understand these first after which we can learn how to test them.

What You Will Learn:

What is Internet of Things (IoT)?

The IoT is the connection of vehicles, home appliances, medicinal equipment using embedded electronics, microchips etc. to collect and exchange data of a different kind is called as IoT. This technology allows the user to control devices remotely over a network.

IoT Examples

Some of the examples of IoT in real life implementation are as follows:

#1) In wearable tech:

Wearable gadgets such as Fitbit bands and Apple watches synchronize easily with the mobile devices.

These help in capturing necessary information such as health, heart rate monitoring, sleeping activity etc. These also help in displaying data, notifications from the mobile devices onto them.

#2) Infrastructure and development:

With the use of an application such as CitySense, it is easier to get the real time outdoor lighting data and based on these, the street lights are switched on or off. There are also various applications so as to control traffic signals and parking availability in a sophisticated city layout.

#3) Healthcare:

There are numerous applications so as to monitor the health conditions of the patients.

Based on the benchmarked data, the services control the dosage of medicine at different times in a day. There are applications such as UroSense which can monitor the fluid levels in the patient’s body and based on the need can initiate the fluid transfer. At the same time, the data can be transmitted wirelessly to various stakeholders.

Technology used in IoT

Following are the few of the most used technologies in IoT:

  1. RFID [Radio Frequency Code] tags and EPC [Electronic Product Code]
  2. NFC [Near Field Communication] is used to enable two-way interactions between the electronic devices. This is basically for the smartphones and is mostly used to do the contactless payment transactions.
  3. Bluetooth: This is used where short range communications are enough to get away with the problem. This is mostly used in wearable technologies.
  4. Z-Wave: This is a low power RF comm technology. This is primarily used for home automation, lamp controlling etc.
  5. WiFi: This is the most commonly used choice for IoT. When on a LAN, this helps in transferring files, data and messages seamlessly.

Testing IoT

Let’s take an example of a Medical healthcare tracking system in which the instrument monitors the health, heart rate, fluid intake details and sends out a report to the physicians. That data is recorded in the system and the historical data can be viewed whenever required.

The physicians can initiate drug intakes, fluid supplements based on the data. This can be triggered remotely from any of the devices [computers or mobile devices] to which the medical device is connected to.

Now to test such an architecture, we need to go through multiple testing approaches such as;

IoT Test Approaches

#1) Usability:

=> Read more about general Usability testing here

#2) IoT Security:

=> Read more about general Security Testing here

#3) Connectivity:

#4) Performance:

#5) Compatibility Testing:

=> Read more about general Security Testing here

#6) Pilot Testing:

#7) Regulatory Testing:

#8) Upgrade testing:

IoT Testing Challenges

The challenges a tester faces in IoT are as follows:

#1) Hardware-Software Mesh

IoT is an architecture, which is closely coupled among various hardware and software components. It is not only the software applications that makes the system but also the hardware ones, sensors, communication gateways etc. too play a vital role.

Only functionality testing does not help in completely certifying the system. There is always a dependency on each other in terms of the environment, data transfer etc. So, it becomes a tedious job as compared to testing a generic system [only software/hardware component].

#2) Device Interaction module

As this is an architecture between different set(s) of hardware and software, it becomes mandatory that they talk to each other in real time/near real time. When they both integrate with each other, things such as security, backward compatibility, upgrade issues becomes a challenge for the testing team.

#3) Real-time data testing

As we have discussed earlier that a Pilot testing/regulatory testing is mandatory for a system such as this, it also becomes very tough to get such data.

Being in testing team, getting regulatory checkpoints or getting the system deployed in the pilot is very tough. The step becomes even tougher if the system is related to Healthcare as per our example. So, that stays as a big challenge for the testing team.

#4) UI

The IoT is spread across devices belonging to every platform [iOS, Android, Windows, linux]. Now, testing that out on devices can be done but testing it on all possible devices is almost impossible.

We cannot omit the possibility of the UI being accessed from a device which we don’t possess or simulate. That’s a challenge which is tough to overcome.

#5) Network availability

Network connection plays a vital role as IoT is all about the data being communicated in faster speeds all the time. IoT architecture has to be tested in all kinds of network connectivity/speeds.

To test this, virtual network simulators are mostly used to vary the network load, connectivity, stability etc. But, real time data/network is always a new scenario and testing team doesn’t know where the bottleneck would develop in the long run.

IoT Testing Tools

There are various tools which are used during testing IoT systems.

They can be classified based on the target and are given as below:

#1) Software:

  1. Wireshark: This is an Open source application used to monitor the traffic in the interface, source/destination host addresses etc.
  2. Tcpdump: This does a similar job that of the Wireshark except, this does not have a GUI. This is a command line based utility which helps the user in displaying the TCP/IP and other packets that are transmitted or received over a network.

#2) Hardware:

  1. JTAG Dongle: This is similar to a debugger in PC applications. This helps in debugging the target platform code and show variable step by step.
  2. Digital Storage Oscilloscope: This is used to check various events with time stamps, glitches in power supply, signal integrity check.
  3. Software Defined Radio: This is used to emulate receiver and transmitter for a large range of wireless gateways.

To the developing world around us, IoT is a growing market and has a lot of opportunities. The time is not far when IoT becomes essential for the testers to survive in the development world.

The IoT-enabled gadget, smart device application, and communication module play a vital role in studying and evaluating the performance and behavior of various IoT services.

Poor design of IoT-enabled devices and services can hamper the correct functioning of the application and in turn negatively affect the end-user experience.

To Summarize

IoT testing approach can be different based on the system/architecture involved. Testers should concentrate more on the Test-As-A-User [TAAS] approach rather than testing based on the requirements.

One more major player in IoT testing is the Integration testing. IoT is successful if the Integration test plan is accurate and robust enough to catch flaws in the system.

IOT testing may be a tough/challenging job but, it is also very exciting as well for the testing team to certify such a complicated mesh of devices, protocols, hardware, operation systems, firmware etc.

About the author: This is a guest post by Subhasis. He is working as a team lead with over 8 years of corporate experience working for Fortune 500 IT companies in the field of Software Quality Assurance, Software Development and Testing experience.

UPDATE: We have added a useful Infographics on IoT topic. Thanks to for sharing this with us.  

Post your IoT queries/comments below.