Let us explore an interesting and challenging topic “How to Test ERP Software”. We’ll learn the challenges involved in ERP testing (e.g SAP Testing), guidelines and road-map for successful ERP testing.
Before going into more details about ERP testing, first, we need to understand some key information about ERP. For testing purposes, understanding the features, functionalities, and workflow of ERP software is most important. Without the proper knowledge, testing ERP modules is very critical and cumbersome.
What You Will Learn:
What is ERP and why do companies need ERP Software?
ERP means Enterprise Resource Planning. A more practical thought is that if an “Enterprise” (an organization or a company) is “Planning” to increase productivity and efficiency by integrating the “Resource”, then it is ERP.
What are the resources? It is called as 4M’s in any type of organization like Man, Machine, Method, and Materials.
Every ERP Software in the globe is built based on this concept in mind. Every company needs an Accounting or ERP solution which is a fact of any business.
An organization may use one or more software as per the requirements of the individual departments. For example, the Accounts department may use accounting software developed in Oracle technologies, the finance department may use budgeting software developed in Microsoft technologies, the personnel department may use payroll software developed in Java and inventory department may use warehousing software developed by IBM and so on.
Apart from the technologies, each software is different. The database used for storing the data is also different from Oracle, SQL Server, DB/2, etc.
Most importantly, the data available in one department is not available for the other department, as the data is not stored in a centralized location. This situation raises many complexities in a business scenario of an enterprise organization and reduces the productivity of the company as well as impacts the efficiency of the employees.
For instance, the sales manager doesn’t know the inventory stock in the warehouse for committing the sales order with a customer and the Vice President of the company doesn’t know the financial status of the company until unless visiting the department or have a phone call with them.
ERP Software considers all these various business situations and functional flow scenarios and gives the solution to the enterprise customers by integrating the multiple department’s workflows inside the organization and centralizing the database through a server infrastructure.
So, every department’s data will be stored in a centralized server on a daily basis. Any authorized employee will be able to view live data online without the need to interact with anyone to proceed with their daily operations. Finally, the VP of the company can review the complete organization data with a single mouse click. Great work, right?
The worldwide ERP software market per year is more than $25Billion which grows 10-20% every year and the demand will be increased in small and medium enterprise business segments through SaaS (Software-as-a-Service) and Cloud (Off-Premises Infrastructure) platforms.
Subsequently, the demand for ERP testing professionals is also increasing in the IT industry. Many ERP implementation projects spent more than 50% of their budget on testing and related activities.
Even though we call it ERP Software, it is not “just software” and the ERP vendor’s job is not completed by selling ERP software to the customer. ERP is actually a “business solution” to an organization by automating the business process and workflow inside the company. This will not be achieved by just installing the software and setting up the infrastructure, rather the success of ERP implementation requires a lot of dedicated workaround and people involvement.
In this ERP world, how to select and evaluate the best ERP software, which is having the quality and performance as expected is a big question and dilemma amongst the enterprise customers.
A snapshot of the ERP enabled business organization will be captured as below.
Importance of ERP Testing
The majority of ERP Implementation projects fail because of improper test planning and testing. To understand the importance of ERP testing, testing professionals need to understand the variations of the ERP systems.
ERP systems may be available in multiple versions for multiple domains targeting multiple customers. For example, major ERP vendors are having different versions and flavors of their ERP software as below.
- Microsoft Dynamics NAV: Designed for small and medium enterprise customers
- Microsoft Dynamics AX: Designed for large enterprise customers
- System testing, performance testing, and load testing are important, apart from functional testing.
- SAP Insurance: Designed for Insurance companies
- SAP Banking: Designed for Banking sectors
- SAP Chemical: Designed for Chemical companies
- Domain knowledge and industry experience are important, apart from functional testing.
Similarly, the products from Oracle viz., E-Business Suite, PeopleSoft, Siebel and JD Edwards etc, have different features, target customers, and industries, where the testing team needs to understand the purpose and functionalities of the complete systems.
There are two types of testing teams available in ERP companies:
- “Core testing team” to test the ‘base ERP software’ with the static functionalities.
- “Implementation testing team” to test the ‘implementation project’ with dynamic and customized functionalities.
When a feature and functionality is customized as per the customer requirements, the modified impact will be on multiple modules and systems. The testing team should capture each and every change and the related impact for the complete traceability.
Automated ERP testing
ERP companies have a dedicated testing force to test their ERP systems. Testing such a large system requires the test execution to be “automated”. Depending upon the requirements, the ERP companies design their automated testing framework and set up their testing infrastructure.
Automated testing is suggested for ERP testing as manual testing requires a lot of time to complete one round of testing. It is evident, that without testing each and every component of a system and ensuring 100% quality, ERP implementation fails.
Below is the list of different types of ERP testing as most of the ERP systems are being built in an SOA (Service Oriented Architecture).
- Functional testing
- Data Handling testing
- Integrity testing
- System testing
- Security testing
- Reliability testing
- Adaptability testing
- Scalability testing
- Usability testing
- Performance testing
- Load testing
- Interface testing
- Interoperability testing
- Regression testing
- Infrastructure testing
- Image testing
- Installation testing
- Parallel testing
There are many ERP vendors in the International market.
Below is a quick list of ERP software and its vendors based on market share, cost, implementation model, average time for deployment, platform, and delivery, technology, features, and functionalities. Each vendor offers different flavors of the product depending on the customer’s domain and industry verticals.
- SAP (#1) – From SAP since 1972, to major market shareholders and millions of users worldwide.
- Microsoft Dynamics (#2) – From Microsoft Corporation, millions of users worldwide.
- Oracle E-Business Suite (#3) – From Oracle Corporation to millions of users worldwide.
- Epicor ERP
- Sage ERP
- Infor ERP
- Ramco ERP
- Focus ERP
- Tally ERP
What are the ERP Modules? The module in ERP is a group of features and functionalities for a specific group of process or departmental workflows.
For example, sales order process, payment process, and customer management can be grouped as a unique functional workflow in a company and can be formed under “Sales & Distribution” module and similarly employee attendance, payroll process, personnel management can be grouped as a unique functional workflow in a company and can be formed under the “HR & Payroll” module.
For testing purposes, understanding the features, functionalities, and workflow is most important. Without the proper knowledge, testing ERP modules is very critical and cumbersome.
ERP vendors may have different module names for grouping the functionalities, but the concept will be the same. For example, let us have a quick look at a few SAP modules to have an idea if we are performing SAP Testing.
- SAP FICO (for Financial and Cost Accounting)
- SAP HR (for Human Resources)
- SAP MM (for Materials Management)
- SAP PP (for Production Planning)
- SAP SD (for Sales and Distribution)
- SAP CRM (for Customer Relationship Management)
- SAP SCM (for Supply Chain Management)
Similarly, all ERP software have such modules as per their functionality grouping and naming conventions.
Why is ERP Testing Critical?
ERP Testing is really critical if the testing plans don’t cover the complete road-map of the implementation phase. All the modules within the system are completely integrated and inter-related, meaning all the functionalities are dependent on the functionalities of other modules or other systems.
For example, to create a sales order in “Sales” module, the system has to check the available stock in the “Inventory” module and check the credit limit of the customer in “Customer” module.
The functionality is not only dependent on other modules but also dependent on external systems like a multi-currency enabled system retrieves the current exchange rate from an external live system, the shipping rate for the sales order may be required from an external live system, the sales tax of the inventory items may be required from a government agency etc.,
When a test case is written based on the above business use case, a tester needs to understand the complete workflow of the functionality and the dependency of the workflow, otherwise, the test case will fail or remain incomplete.
Below diagram gives an overview of a sales order process and the dependency of internal and external systems.
Apart from the functional testing like in the above business use case, the other phase of testing is on the “ERP Implementation” which is more critical. The implementation phase of ERP involves a lot of modifications and customization as per the customer requirements and infrastructure.
This kind of testing not only involves normal functional testing but also the performance and user acceptance testing. There are multiple components that interact with the live system from internal and external sources.
Understanding all the components and their functionalities is more important for testing professionals to prepare the test plans and the test suite with all the feasible use cases to be covered for the application under testing.
The successful ERP testing professionals are trained and worked on a lot of different ERP implementations for many years and gather experience to become a capable ERP tester with an additional credential as a “subject matter expert” or a “domain expert”.
The heart of many ERP systems is in Accounting or Finance modules, where most of the testers lack accounting knowledge and find it hard to understand the financial concepts and the general ledger system. The complexity in the inter-related and the outer-related workflow of the ERP systems demotivates the testing professionals in writing a proper ERP test suite.
In particular, the implementation of ERP systems requires various forms of system testing, multiple user acceptance testing, multiple levels of performance and load testing with millions of “live test data” from the existing systems which need to be imported to the new ERP system.
The main challenge is in collecting input data for testing the feature and analyzing the correctness of the output data, which requires domain knowledge.
For example, it is not that easy to collect data to test the general ledger transactions or sales order transactions in ERP systems. Even though the data is collected for input, it is highly critical to understand and analyze the correctness of output data as per the financial standards like IFRS.
For example, a “balance sheet” generated from the general ledger transactions cannot be understood by normal testers without the knowledge of a balance sheet or trial balance. The reason being the “balance sheet” is generated from the general ledger based on multiple transactions from all the modules.
Can I Test ERP Software?
The important question for testing professionals is “Can I test the ERP system with my available knowledge and experience?”. The answer to this question is not simple “Yes or No”, but requires self-assessment.
ERP systems are available in multiple languages, supporting multiple currencies, adaptable for multiple platforms, targeting multiple domains and designed for multiple users. And, the testers are not going to test the complete ERP system individually or alone.
No one is expected to have the complete and full knowledge and experience of all the modules in ERP or multiple languages expertise or multiple industrial exposures. However, testing professionals need to ask themselves a few questions as below, which will help them to prepare for the ERP road-map.
- May I know the ERP and ERP Modules?
- Have I written test cases for ERP before?
- Have I executed test cases for ERP before?
- Do I have any technical knowledge like Scripting, HTML, XML, and SQL?
- Do I have any industrial exposure like Pharmaceutical, Chemical, and Banking, etc?
- How many ERP modules am I going to test?
- Do I need training for ERP testing?
- Is the ERP testing manual or automated?
- Am I part of the Core testing team or the Implementation testing team?
ERP Testing Road-map
Let’s prepare ourselves with a road-map with a checklist for ERP testing. The point to note is that the “testing effort” might be almost equal to “development effort” for ERP testing and the ERP companies are trying to set up a platform and infrastructure to reduce the effort, but increasing the efficiency and productivity.
This is a quick checklist and may vary on a case-to-case basis.
ERP Testing Checklist:
|1.||Identify the scope of testing and check the ERP testing is for the base product or for the implementation project.|
|2.||Identify the total modules available / released in the product for the application under testing.|
|3.||Identify the target customers, system users, and industry verticals of the product to be implemented.|
|4.||Estimate the testing effort for writing testing plan, test suite, test cases, test execution and test reports.|
|5.||Check the status of previous testing strategy, test execution plans for the application under testing.|
|6.||Identify the business process to be tested and identify the business process to be automated.|
|7.||Identify the test scenarios to be tested and identify the test scenarios to be automated.|
|8.||Analyze the existing testing framework for manual and automated testing for the ERP application under testing.|
|9.||Check the existing / proposed automated testing tool is configured and the framework is well tested for execution.|
|10.||Setup the Quality Assurance strategy with the quality metrics identified for the ERP testing for both manual and automation.|
|11.||Analyse the automation test scripts for all the business use case scenarios are well reviewed and approved.|
|12.||Identify the best practices for improving the efficiency and productivity of testing efforts for both manual and automation.|
|13.||Form a testing team with the group of experienced professionals with the right skill set for both manual and automated ERP testing.|
|14.||Setup the testing framework for maintaining the test scripts which can be reusable in automation process.|
|15.||Escalate the test scenario / business use cases which are not feasible to test or not feasible for automation framework.|
|16.||Identify the process of collecting test data for both input and output and check the availability of domain expert to analyze the output data.|
Before getting into ERP testing, the testing professionals need to understand the concepts, process and strategies and the criticalities of ERP testing. Yes, we are now ready to start the testing process with a quick road-map in hand.
The next ERP testing tutorial: ERP Testing Test Plan – So, our next step is to initiate the ERP testing process by preparing the QA and Test Plan, preparing the Test Suite with module wise Test Cases, identifying the best practices for manual and automated Test Execution and preparing the template format for Test Reports.
We will cover all these topics in our next tutorial. So don’t miss it.
Let us know if you have any questions about ERP or SAP Testing.