This Tutorial Explains What is Multi-Tenancy, the Types of Testing to Perform for Multi-Tenant Systems & Database Configurations for Multi-Tenant Architecture:
A multi-tenant system is a software architecture that serves multiple users referred to as tenants.
The multi-tenancy concept can be compared to a single house built on land that can accommodate only one family versus an apartment that can accommodate more families on the same piece of land.
So the available resources are optimized and tuned to perform the task. In the below figure, the left part is based on the concept of single-tenant whereas the right part describes the multi-tenant concept.
Table of Contents:
Need For Multi-Tenancy
At present, many companies are tweaking their products to support multi-tenancy because of the following reasons:
- It is cost-effective for the end-user.
- Maintenance-friendly for the software provider.
- It optimizes the physical resources available.
Multi-Tenant Database Architecture
The architecture of a multi-tenant database would be economical if there is only one database catering to multiple clients. In this case, the hardware resources required to maintain would be minimal.
There are multiple ways a database can be used/configured in a multi-tenant system. These are:
#1) Separate database per tenant/client:
In this approach, each tenant has its own database. Hence they have their own set of data which will remain isolated from other data that other tenants own, that is, logically the database will be isolated.
Pros: High security and can be customized according to the tenant’s needs.
Cons: High maintenance and hardware costs.
#2) Shared database with a different table grouped by schema per tenant/client:
Here, all the tenants will be configured under the same database but they will have their tables grouped by a schema. Isolation will be at the table level.
Pros: Low in cost and addresses security concerns to an extent.
Cons: Data restoration of a single tenant is difficult.
#3) All are shared – database, table, and schema:
In this case, the database and the tables are shared. So here, the tables are the same and will be storing multiple tenants’ data in any order. They might be identified by a column that would indicate the source i.e. tenant information. This would be validated when any tenant places a call request. Here, the isolation will be at a column level.
Pros: Lowest cost
Cons: Data security needs to be addressed appropriately.
Development costs are higher if the operational cost is intended to be lower and vice versa.
Testing Of Multi-Tenant Systems
Let us see various types of testing to conduct while testing multi-tenant systems.
Data Leak Testing
The major factor while testing multi-tenant systems is the way the data is shared/stored in addition to the regular testing. So in a multi-tenant system, the resources can be shared but not data.
Let’s take a multi-tenancy example to find out the usual flaws in a multi-tenant system.
Consider the above example of a form builder application, where the client is allowed to submit a form and retrieve a form. The application server validates for fields in the form and commits/retrieves from the database accordingly.
Now, when three clients are performing the same action simultaneously say commit (submit), the data flow of each of the clients should be observed closely and monitored such that the flows are as expected.
The probability of the occurrence of the issue is more common in an application with a shared Database. A deadlock error in the database happens when more than two database sessions have some locked data. The sessions would wait continuously in a loop and nothing happens till timeout.
Hence, there is a need to detect a deadlock and address a way to handle them if it exists in a multi-tenant system. This is in addition to the observation of the data flow.
Now, consider a database call action being performed by all the clients at the same time. Say, the clients are retrieving data from a form at the same time. Since the resources are shared, there might be a possibility of client 1 receiving the data of client 2. If this happens this is a serious security breach as the integrity of the data is lost.
It is understood that too many database calls written in the system lower the performance. Hence it is essential to have limited calls to get all the information required that is there is a necessity to design the system in such a way that it solves the purpose.
Another way this can be addressed is by having the objects cached but the cost of development to isolate those goes up in comparison to the single-tenant system which assumes a single process at a time with no concurrency. In a multi-tenant system, resources like disk input/output, memory, connection pools, CPU cycles, etc. are shared.
It would be better if validation is done to ensure the uniqueness of the object being used across tenants. It is done so that the tenants should not be able to modify or delete objects. To sum up, data isolation must be clearly defined at all levels.
Multi-tenant System Under Load
Performance and load tests should be performed on multi-tenant systems. This helps in identifying the bottleneck situations in the application. This would validate the stability of the system under load as sometimes data leak happens when the system is under load which would be difficult to find during a manual test.
Endurance tests should also be performed where the system under test is validated for performance over an extended period.
Also, the resource utilization and response times can be measured and benchmarked to determine the metrics of the system under test. System readiness can be measured by performing the above tests. Load balancers are commonly used in this type of environment.
Monkey testing can be done in addition to the above which will check for the reliability of the software. The reason for exhaustive testing is because the number of tenants is more and the reputation will be at a greater risk.
Hence, the system can be tuned for a better experience and the resources can be optimized based on the report.
License/Feature Testing
There must be adequate testing done to validate features/licenses configured for clients. To brief on this, if client 2 is licensed to a particular feature then client 2 should only be able to access them and client 3 should get an appropriate message/warning when it tries to access the same feature.
Also, there should be tests to validate the data isolation associated with the feature. Data isolation is being emphasized as data leak is one of the major issues being faced in a multi-tenant system.
Some applications might have jobs/services running in the background associated with updating the database. These have to be monitored to ensure they update only for the tenant it is configured.
Security Testing
Security testing is another component that must be done as this plays a key role due to the nature of the system.
The major emphasis should be on SQL injection. This is quite vulnerable because if the database server/databases/tables are shared then it would be easier for one tenant/external user (more vulnerable if the application is not restricted to the internal network) to inject queries in the application and exploit another tenant’s data.
It would be good to have some validations at the UI level such that one tenant cannot attempt to view another tenant’s data without disturbing the usability of the system.
Automation Testing
To run automation in a multi-tenant system, it is recommended to run the tests parallel across tenants as this would mimic the production system to an extent. Draw a data flow map and automate tests at each node to check for the data leak. By doing this there would be enough confidence to track feature/functional failures arising.
Conclusion
To summarize, there are regulations, such as the GDPR, that are being passed by some countries where data protection is taken seriously. GDPR is passed by the European Parliament which addresses the data protection for all individuals including the data which is exported out of Europe. (Click here to read more)
These are more vulnerable in a multi-tenant system and have to be addressed on priority during testing.
Multi-tenant architecture is designed in such a way that every tenant may get to share databases, tables, functionality/features, and some non-functional items. The major risks that are involved in this type of architecture are data leaks, security, and the isolation level of the system.
Testing checklist:
- Functional/feature test: To test for the availability of features licensed by tenants.
- Performance test: To benchmark resource utilization and the response time for smooth operability. Have a broader set of performance indicators like memory, processor capacity, disk, bandwidth, response time, etc.
- Data leak issue: It is recommended to draw a map of the data flow for the system under test which includes caching if applicable and to test the points for any possible data leak/data corruption. Validation is required for the usage of objects, file systems, and data purge. Actions involving database commit/call can be done simultaneously which might make it more effective depending on the system.
- Security test: Emphasis on SQL injection due to the nature of the system.
- Automated test: To spend more on creating automated tests is a good investment as all the above testing has to be repeated for each major/minor release. Although it entirely depends on the logic of the framework which has to be developed in such a way that it is scalable to changes and identifies data leaks.
An exhaustive set of tests needs to be performed before releasing a multi-tenant system into production as any failure in production would create a huge impact in terms of reputation and cost due to the involvement of many tenants. On the contrary, maintenance would be easy if they were taken care of by defining/adhering to the process.