Review of Jmeter Components (Part- II):
=> This is part of JMeter training series. See the list of all tutorials in this series here.
Hope you all must have gone through the JMeter Introduction and Installation by now. As we continue with next one in the series, it is highly recommended that you all must install JMeter and practice side by side.
In this tutorial, readers would get themselves familiar with all components of JMeter and how to use them in the test plan to cover all possible performance testing scenarios to test the AUT (Application Under Test).
Elements of Jmeter had been listed down in the previous article.
What You Will Learn:
Components of JMeter
Penning down again for your reference:
- Test Plan
- Config Element
- Logic Controllers
All major components of Jmeter such as Thread Group, Samplers, Listeners and Config Elements are explained in details later in the article.
Please refer to the below flow diagram to understand each component and their relation to specific modules of JMeter.
Now we would start touching each component of Jmeter along with use cases just to know how it works and how can testers implement these in their testing. Please note that we won’t be covering all Samplers, listeners in this article. We will work on the most used ones and will take up rest in the next article when we create real time Test Plans.
Just as a simple test plan in Software Testing consists of all steps which execute the script, JMeter's Test plan has the same purpose. Everything which is included in a test plan is executed in a sequence which is top to bottom or as per the defined sequence in the test plan.
Test Plan can be as simple as it could be, with Just ThreadGroup, Sampler, and Listener and it starts getting more complex as soon as you start adding more elements like config elements, preprocessors or controllers.
As we all know that JMeter measure performance by generating Virtual Users or Threads which hits server under test as if real users are sending requests to a server. Therefore, every Test Plan should have virtual users or Thread Group as we call them in JMeter's terms.
Important Points about Test Plan:
- The test plan should be saved before running
- Jmeter files or test plans are saved in form of. JMX extension files
- You can also save parts of Test Plan as the different selection. For example, If you want to save HTTP Request Sampler with Listener, you can save it as Test Fragment so that it can be used in other test scenarios as well
- Elements of WorkBench are not saved with Test Plan
Thread Group is a group of users which will be hitting the server under test either concurrently or in some predefined sequence. Thread Group can be added to Test Plan by right clicking the test plan. JMeter is all “Right Click stuff”, you get all the options on the right click.
You can rename Thread Group name to your own. Just change the name and click anywhere outside the Test Plan window, you would see the name getting changed.
Please see below screenshot for adding Thread Groups
(Note: Click on any image for enlarged view)
It is very important to configure your thread group as per the test conditions.
For example, if you want to test how a web server behaves when 100 user hits it concurrently, you can set Thread Group as below:
Basically, there are three main parameters which must be configured to generate actual load or virtual users:
- Number of Thread(Users) – It defines the number of virtual users. For testing purpose, we should generate only a limited amount of load as generating huge volume at once would mean consuming lot many threads which can ultimately lead to high CPU utilization.
- Ramp Up Period – This field is very important in controlling the load generation. Ramp up period defined the amount of time in which the total load will be generated.
- It means all 10 users will be hitting servers concurrently as soon as a test is run
- You all must have noticed the “Scheduler” checkbox in above screenshot. In case you want your test to run at a specific time later then you can set the timings as you can also see in the below screenshot. It means that every 1 sec, a new user will be hitting server. The load will not be concurrent but will be in increments. By the 10th second, all user would have hit the request.
- Loop Count – It defines the number of times Thread Group will execute. If you check the Forever check box your test will run forever unless you manually stop it. This can be used to test something like “If your server doesn’t crash on continuous load for few minutes”.
So, how does a Jmeter knows what type of request has been sent to server???
- It is through Samplers. Samplers are a must to add to a Test Plan as only it can let Jmeter know what type of request need to go to which server and with any predefined parameters or not. Requests could be HTTP, HTTP(s), FTP, TCP, SMTP, SOAP etc.
Samplers can only be added to Thread Group not directly under Test Plan as Thread Groups need to use a sampler to send a request to server URL under test. The sampler can be added by path Thread Group -> Sampler -> HTTP Request.
These are the most common requests sent to the servers. Say, for example, we want 100 users to hit https://www.google.com concurrently, this can be done as described in below screenshot:
- The path is the navigation inside the main website. For example, if we want to hit http://www.google.com/gmail then we can set “/Gmail” in path and rest remains the same
- Need not type “www” in the server name
- Port Number is used if you are using any proxy server
- Timeout Connect and Response can be set if you want to have a benchmark on your server connection time and response time. Your request will fail if your server takes more time to send response than configured one
- You can also configure parameters to send with your request. For example: In some cases, you may need to send Authorization Token with your request, so you made add them in HTTP Request as below:
Path-> Test Plan-> Thread Group-> Sampler-> FTP Request
FTP stands for File Transfer Protocol and it is used to upload or download a file from the server. JMeter’s threads send requests to FTP servers to upload or download files from there and measures the performance.
- Local File is the location where you need to save the downloaded file
- Use GET option if you are downloading from FTP server
- User POST option if you are uploading any file on FTP server
We have lot many listeners which will be covered when we go through some real Test Plans consisting Samplers, listeners, config elements etc.
So, till now we have seen few samplers sending requests to the server but haven’t analyzed the response yet. Performance testing is all about analyzing server responses in various form and then presenting the same to the client.
Listeners are used to display the results of test execution so that testers get to know the stats. We have around 15 listeners in Jmeter but mostly used ones are table, tree, and Graph.
View Results in Table:
This is the most commonly used and easily understandable form of listeners. It displays the result in form of table with some important performance parameters.
Listeners can be added directly under Test plan or under a sampler. The difference is, when you add listener under a sampler, it will show the results of that sampler only. If we add sampler directly under the test plan, it displays the result for all the Sampler up in the hierarchy.
The screenshot below for your reference:
You see the results something like displayed below:
- Latency: It is the time when the first piece of information is received i.e. the first byte of data is received
- Connect Time: It is the time taken to establish connection with the server
- Sample Time: It is the time taken to receive complete data
- Sample – Sequence of sample number
- Bytes – Size of data received.
View Results in Tree:
This is another most commonly used listeners and provides detailed information with request and response. One can also view the HTML page rendered in response apart from viewing Json, XML, Text, RegEx.
It is very useful as testers can put assertions on the response received to be ensured that the test passed. Jmeter results still show “Pass” even if the response is not desired.
For Example: Say, we hit HTTP request on any website www.xyz.com and in response we expect <title> XYZ </title> or in simple words, when we hit this page company’s home page opens with its name. If we haven’t put assertion, Jmeter will still display results since the hit has gone to the server.
Please see below to know the format of results:
For viewing HTML page in response, click on the drop-down in the left pane and then select “HTML”, Navigate to the response tab and check the page which is returned as server’s response.
A workbench is a place where you can store those elements which are not in use in your current test plan but which can be later copy pasted in it. When you save JMeter file, the components which are present in workbench are not automatically saved. You must save them separately by right clicking and choose “Save as “option.
You all might be thinking then what is the use of workbench, anyways it is easy to add any component directly to a Jmeter’s Test Plan.
The reason of having workbench was that user could do some experiments and try out new scenarios. As we already know elements in workbench are not saved so a user can literally use anything and then throw away. But, there are some “Non-Test Components” which are only available in WorkBench.
They are listed here:
- HTTP Mirror Server
- HTTP(s) Test Script Recorder
- Property Display
HTTP(s) Test Script Recorder is the most important Non-Test Element used in JMeter. It helps testers in recording the script and then configuring the load for each transaction.
Jmeter only records the request sent to the server. Don’t get confused with “Record and Play” functionality of QTP/Selenium. All the requests are recorded and testers can apply the desired load on them to see the behavior.
This element is very important for scenarios where testers don’t know what all requests are going on from their application. They can use Http(s) script recorder to record the application under testing.
Performance Testing of mobile applications can also be done in this way by setting up JMeter proxy and then recording the requests which our mobile app sends to the server. Step by Step procedure for mobile performance testing will be explained in next article.
Till now, we have covered how JMeter hits the server and how the responses are displayed via listeners. To ensure that the response received is correct and as per expectation, we need to add assertions. Assertions are simply validations which we need to put on responses to compare the results.
Below are the types of assertions commonly used:
- Response Assertion
- Duration Assertion
- Size Assertion
- XML Assertion
- HTML Assertion
In Response Assertion, we can add our own pattern strings and then compare them with the responses received from a server. For example, you all know response code is 200 when any request returns some response successfully. So, if we add pattern string “Response Code = 202” then the test case should fail.
Please refer below screenshots to add Response Code assertion.
Now, when the test runs it shows the result with Red Color indicating that the Assertion results are failed.
Duration Assertion is very important and validates that the server responded within a given amount of time. This can be used in scenarios where we need to sample 100 requests and ensure that every time response is received within the benchmarked limit.
Case: 10 users are concurrently hitting “google.com” server and Duration Assertion is set to 1000ms.Please see below screenshots:
XML Assertion validates if response data has correct XML document in it and HTML Assertion verifies the HTML syntax of the response received from a server.
Requests sent to the server can be further parameterize using some config elements which are executed before the actual request. A simple example of it could be reading values of a variable from a CSV file for which CSV Data Set Config is used.
Below are some of the important config elements used in the performance testing of the web and mobile applications
- CSV Data Set Config.
- User Defined Variables
- HTTPS Requests Default
- HTTPS Cache Manager
- HTTPS Cookie Manager
CSV Data Set Config
CSV data set config helps Jmeter in picking values of some parameters from a CSV file rather than passing different parameter in each separate request. For example, if we need to test login functionality with a different set of users and passwords, then we can create two columns in a CSV file and enter the values there so that JMeter can pick one for each request sent to the server.
Below is the flow of using CSV data set the config to test weather API for different cities in India.
- Adding CSV data set config element to test plan
- Creating CSV file
- Passing variable in the request parameter. APPID parameter can be generated dynamically from http://openweathermap.org/appid
- Running the test and viewing results.
User Defined Variables
It helps Jmeter to pick values from a pre-defined variable. For example, support that you need to create a test plan in which you need to add many HTTP requests on the same URL and there could be a scenario in which client plans to migrate it later to some different URL.So, to avoid updating URL in each request we can tell JMeter to pick the URL from a UDV (User Defined Variable) which can be later updated to handle all requests to updated URL.
So, to avoid updating URL in each request we can tell JMeter to pick the URL from a UDV (User Defined Variable) which can be later updated to handle all requests to updated URL.
HTTP Request Defaults
This config element is very useful for specifying the default values of https requests. To guide you more, take an example where we need to hit 50 different requests at google server.In this scenario, if we add an HTTP Request Default then we need not specify a server name, path or any other properties like port number, connection time out properties. Whatever is specified in the HTTP Request Default config element is inherited by all HTTP requests.
In this scenario, if we add an HTTP Request Default then we need not specify a server name, path or any other properties like port number, connection time out properties. Whatever is specified in the HTTP Request Default config element is inherited by all HTTP requests.
Please see below how to add HTTP Request Default and specify server and path.
HTTP Cache Manager and HTTP Cookie Manager are used in making JMeter behave as a real-time browser. HTTP Cache manager can clear cache after each request whereas the other one can manage the cookies settings.
Logic Controllers And Timers
Logic controllers and timers help Jmeter control the flow of transactions. Timers ensure the delay in each thread if need to test any server. For Example, if we need FTP request to wait for 5 secs after HTTP request is completed, we can add timer there.
Logic Controllers are used to defining the flow of requests which are sent to the server. It can also let you store requests for each module separately such as login and logout.
By now, you all must have got yourself familiar with the components of JMeter and have tried using it and must be facing some issues. In the next article, we will be covering some real-time performance testing scenarios covering mobility domain so that you all get more hands-on knowledge on JMeter.
Stay Tuned! Next article will help you managing your requests as well as analyzing results and comparing with the benchmarks of performance testing.
=> Continue reading part-III: JMeter Processors and Controllers