Manual and Automation Testing Challenges

Software Testing has a lot of challenges both in a Manual as well as in Automation.

Generally in Manual Testing scenario developers through the build to test team assuming the responsible test team or tester will pick the build and will come to ask what the build is about? This is the case in organizations not following so-called ‘Processes'.

Tester is the middleman between developing a team and the customers, handling the pressure from both sides. And I assume most of our readers are smart enough to handle this pressure. Aren't you?

Manual and Automation Testing Challenges

This is not the case always. Sometimes testers may add complications in a testing process due to their unskilled way of working. In this post I have added most of the testing challenges created due to testing staff, developing staff, testing processes and wrong management decisions.

So here we go with the Top Challenges:

#1) Testing the complete application
Is it possible? I think impossible. There are millions of test combinations. It’s not possible to test each and every combination both in the Manual as well as in Automation Testing. If you try all these combinations you will never ship the product ;-)

#2) Misunderstanding of company processes
Sometimes you just don't pay proper attention to what the company-defined processes are and these are for what purposes. There are some myths in testers that they should only go with company processes even these processes are not applicable for their current testing scenario. This results in incomplete and inappropriate Application Testing.

#3) Relationship with Developers
Big challenge. Requires very skilled tester to handle this relation positively and even by completing the work in testers way. There are simply hundreds of excuses developers or testers can make when they do not agree with some points. For this tester also requires Good Communication, Troubleshooting and analyzing skill.

#4) Regression Testing
When a project goes on expanding the regression testing work simply becomes uncontrolled. The pressure to handle the current functionality changes, previous working functionality checks, and bug tracking.

#5) Lack of Skilled Testers
I will call this a ‘wrong management decision’ while selecting or training testers for their project task in hand. These unskilled fellows may add more chaos than simplifying the testing work. This results in incomplete, insufficient and ad-hoc testing throughout the Testing Life Cycle.

#6) Testing always under Time Constraint
Hey tester, we want to ship this product by this weekend, are you ready for completion? When this order comes from the boss, tester simply focuses on task completion and not on the test coverage and quality of work. There is a huge list of tasks that you need to complete within the specified time. This includes writing, executing, automating and reviewing the test cases.

#7) Which Tests to execute first?
If you are facing the challenge stated in point no 6, then how will you make a decision which test cases should be executed and with what priority? Which tests are important over others? This requires a good experience to work under pressure.

#8 ) Understanding the Requirements
Sometimes testers are responsible for communicating with customers for understanding the requirements. What if tester fails to understand the requirements? Will he be able to test the application properly? Definitely No! Testers require good listening and understanding capabilities.

#9) Automation Testing
Many sub-challenges – Should automate the testing work? Till what level automation should be done? Do you have sufficient and skilled resources for Automation? Is time permissible for automating the test cases? The decision of Automation or Manual Testing will need to address the pros and cons of each process.

#10) The decision to stop the Testing
When to stop testing? Very difficult decision. Requires core judgment of testing processes and the importance of each process. Also requires ‘on the fly’ decision ability.

#11) One test team under multiple projects
Challenging to keep track of each task. Communication challenges. Many times results in failure of one or both the projects.

#12) Reuse of Test scripts
Application development methods are changing rapidly, making it difficult to manage the test tools and test scripts. Test script migration or reuse is very essential but difficult task.

#13) Testers focusing on finding easy bugs
If the organization is rewarding testers based on a number of bugs (very bad approach to judging Testers Performance) then some testers only concentrate on finding easy bugs those don't require deep understanding and testing. A hard or subtle bug remains unnoticed in such a testing approach.

#14) To cope with attrition
Increasing salaries and benefits making many employees leave the company at very short career intervals. Management is facing hard problems to cope with the attrition rate. Challenges – New testers require project training from the beginning, complex projects are difficult to understand, delay in shipping date!

These are some top Software Testing challenges we face daily. Project success or failure depends largely on how you address these basic issues.

For further reference and detailed solutions to these challenges refer to book “Surviving the Top Ten Challenges of Software Testing” written by William E. Perry and Randall W. Rice.

Over to you
Many of you are working on the Manual and/or Automation Testing field. Though I have addressed many of above challenges in our previous articles, I want your views on handling these Software Testing challenges.  Feel free to express your views in the comment section below.

Recommended Reading

148 thoughts on “Manual and Automation Testing Challenges”

  1. This Post is wonderful, I got some realtime experience from this. More over I need some information about web testing(website). Could you please explain about the web testing and web testing check list. And also explain from where I have to start test a website.

    Reply
    • How we use Selenium script for testing web applications?
      Here we generate selenium script using Eclipse editor.
      Using this script, exchange data from Excel spread sheets and relevant data base through web developed under testing.
      What is needed to be a good Selenium Tester?
      For very detailed testing – there are as many as 24 tests to be performed.
      One can generate Selenium script right from using Web Driver class.
      Automation is a very useful technique in web testing over manual testing. But it has limitation that it will not replace manual. But when test case design, beauty is that manual test case can be automated.
      Testing tools helps in automation with minimum scripting knowledge but good handle on scripting from scratch will help in better utilization of tools.
      For achieving this one should have clear knowledge of html, css needed for testing, detailed knowledge of java script, dom and java.
      In selenium/java scripting heart is –
      a) Clear knowledge of variable types, converting from one to another (Ex: Casting, parse integer), arrays, conditional statements.
      b) Through knowledge of Java jar, package, sub-packages and classes like java-util, java-sql, org-apache, selenium etc., is essential.
      c) Creating user defined classes and packages with constructor, procedures and functions.
      d) Calling or using developed classes and packages when needed in any testing scripts.
      This will save enormous amount of time and effort as they can be repeatedly used for all future testing scripts and by other testers as well.
      IDE: Similar to MS plug and play, IDE comes as add- on for fire-fox driver which is plug and record.
      When application is ready for testing, run IDE with recording for planned test. If this runs smooth, application as a whole of part is ready for detailed testing. Also it can be used for finding element locators easily.
      Beauty of selenium will be appreciated when through with Intelligent frameworks(6).
      Hybrid as the name implies uses more than one framework and transacting from one to other.
      Selenium will not recognize windows:
      Hence it will not handle files and Excel documents. But can be handled in Selenium using third party tools like Auto It for file uploading.
      Similarly Excel documents can be managed with POIs (org apache package). Hard coded data will generally saved in Excel spread sheets. To avoid duplication, POIs will help to use hard coded data from Excel for using in Selenium – import data(input stream) and exporting results back to Excel(Output Stream).
      Data base connectivity:
      Much of data provided by customer and future data collected will be stored in data base. Selenium can interact with data base using job (java sol package) and class for name classes. Data stored in data base is handled using sql queries. Mostly we use data manipulation language. This is also used to delete data entered into data base while testing one test is complete.
      Most skill is needed in creating Grid which simulates multiple users across different platforms browsers, web pages concurrently. This is used for Load testing for finding memory space requirements on server.
      Meticulous use of wait time given is a definite skill – Selenium is much faster tool than web application. Hence short interval needs to be provided between steps like Thread sleep etc., or using java script web driver wait class. It will slow down testing time but ensures errors minimized and test stopping.
      In debugging try catch block will help in providing step by step issues that help in finding solutions.
      Dealing with frames, pop ups is definite requirement.
      Selenium will show test results in console in mercury but will not generate any report.
      Test cases written in excel. These steps in case are accessed through poi in through Input Stream and performed.
      One way is to write test results in Excel using out-put-Stream and saving in Excel preferably with CSS effects.
      Other most accepted way is using TestNG that generates test performance as html document for sharing with either internally or with Customer. One will be considered as automation tester who is good in using TestNG.
      THIS IS THE KNOWLEDGE OR SKILL SET ESSENTIAL FOR SELENIUM AUTOMATION TESTING – this will give enough ammunition for good working.
      NEVERTHLESS HANDS-ON EXPERIENCE IS THE FINAL – similar to battle field experience.
      Most common problems encountered in running test?
      1) Web page not found?
      2) Element id, name, link text (partial or full), x-path not found?
      3) File not located?
      4) Data base not connected?
      5) Null point exception?
      These are most frequently encountered problems.
      File not found exception shows as null point – means either a) path file to be corrected or extra spaces also can cause this.
      b) With pois, type of class (HSSF/XSSF etc.,) and file format may not match.
      c) Similar exception is with url of web.
      d) Same thing may happen with data base connectivity.
      Locator not found: Selenium runs much faster than web most of the times.
      a) Selenium searches for locator before another page is loaded. Needs providing extra time for page display through thread sleep, either implicit or explicit methods or using web driver wait?
      b) Locator changed by programmer in versions or locator being randomized rather than static for security reasons.
      c) Similar issue happens with variables and declarations.
      Integer declared may be required to initialize at the beginning.
      Variable declaration as public static or private may cause errors.
      Automation scripting from scratch for Web Applications:
      Murty Peri, Ph.D., Chemistry
      Hydrocarbon, Polymer, health care research scientist back ground
      Worked in India, Saudi Arabia, USA.
      Selenium Automation tester

      Reply
  2. As u said it’s most important how tester handle all these challenges.
    Some challenges are not in control of QA team like selection and appointment of testers over the projects ;-)
    Managements are really facing hard problems retaining the talent especially in countries like India where supply and demand is becoming critical.

    Reply
    • Not really.
      We have excellent pool of people who can serve but identifying and process is more complicated.
      I am Murty Peri, Ph.D., Chemistry Hydrocarbon, Polymer, health care research scientist back ground
      Worked in India, Saudi Arabia, USA. and Selenium Automation skill added. Right now I am Vizag, AP India. I am ready to participate work with any IT firm for another year while I am here.

      Reply
  3. Excellent Article. I also in the testing field for more than two years, Manual testing. Recently i joined in a new company. Their interviews were standard, having 4 rounds such as aptitude test, technical test, techincal interview and hr interview. What happened is, only after joined there i found i was mistaken. Actually they didn’t have testing stratagey. didn’t bug tracking tools etc. What they had is some untested softwares, which they developed for some of their own companies, all their modules constitutes an ERP package. Now i am in delima what to do. Can anyone find a solution, whether continue with this company for 1 yr or look for another. Plssss help me..

    Reply
  4. @Nisha
    You dont worry, you are in the right pool. You can download a free bug tracking tool from the net say for an example ‘Qengine issuemanager’ and configure it for your company needs and thereby you can impress your company people. And if you can place some ERP experience, it will be an advantage in your further job changes. You are getting a chance to implement testing srategies in your company, that agian will be a plus. Then now you are thinking either they are not willing for your suggestions for the new quality plans or how can i implement all these and more in a new environment. God bless you… leave your doubts today itself

    Reply
  5. Hi readers,
    Though I have addressed many above mentioned challenges in my previous posts, I want your experience to handle such challenges.

    @Simson – I agree, there are some challenges that are not in testers control like assigning testers to projects, training from company etc..As these challenges are handled at management level.

    Still most of other challenges are faced by QA teams. So share your experience which will be helpful to others..

    Reply
  6. hi,vijay
    whatever information or details your giving its really added to our knowledge but I am having some queries regrding how to face HR round and technical round so pls
    give more details if you have some idea.

    Reply
  7. @susan
    form tester prospective..
    stub–simulation of not yet implemented functionality.. used in top down

    driver –simulation of not yet implemented functionality
    used in bottom up..

    Reply
  8. Thanks rahul,
    HOW how we can simulate the not yet implemented functionality, if by using the code then who will write the code, or the template “underconstruction” is this stub and driver. Please clarify me rahul, you are genius i saw your other postings…
    susan

    Reply
  9. I have read so many articles and books but,notting as like this website is providing.It is doing wonders for people who want to hone there skills in testing.I think many people would not be able to list out some of the testing challenges listed above.Great going ..keep up with your good work …

    Reply
  10. Iam not able to understand what is this APACHE AND TOMCAT SERVER is all about..are they different ?
    Some say they are not and they are called as webapplication servers.By the way do application server and web server reside on the same machine or in different machines.How they are configured.
    How a application server contacts webserver or vice versa..again are they configured on differnet machines .I am totally confused please help me understand this.Respond me to this e mail with your answers norman_g40@yahoo.com

    Reply
  11. hi,
    questions asked in test.

    1.write any 5 test metrics.

    what is test metrics.

    2. following artifacts r baselined.
    a.SRS b. Test cases c. minutes of meeting d.agenda for meeting e. defect reports.

    3. configuration reviews r not needed if regression test has been regourously applied during software integration.

    a. True b. False c. Incorrect Stmt d. None

    4. customers perspective view of quality is measured by

    a. surveys b. service level agreements c. recommend & loyalty d. all of these

    5. in the change control procedure major changes should be approved by

    a. change control board b. configuration control board c. senior manager d. It manager e. none

    6. what is test design and test procedure

    7. QC process involves

    a. defect prevention b. casual analysis c. review walkthrough, inspection d. statistical process control.

    and also send me related questions which v can predict in the interview by looking into these types of questions.

    thanx in advance
    rahini

    Reply
  12. @Rahini
    1>defect mat2>req . mat 3>tracebility mat 4>
    2>it could be a.SRS b. Test cases but in most of cases SRS gets priority
    4>b
    5>CCB
    6>test proce–order in which test cases are executed
    test design–we focus in designing TC
    7> d. statistical process control.

    Reply
  13. hi,
    i had read urs manual about testing challenges.
    it realy covers all aspects related 2 s/w testing challenges.really it is best effort……………………

    thanx..

    Reply
  14. thank you for ur reply.
    what about agenda for meeting. what exactly it means.

    as per my knowledge QA is process oriented and QC is product oriented. clarify me if am wrong.
    so i chose the option defect prevention. is it wrong.

    hi rahul,
    whats the expansion of ccb which u mentioned coz 2 CCB r there. ambiguous here.

    and also i need information about ISO 9000 and CMM.
    and for what kind of organizations these certifications been give. this is also interview question

    options r here for ISO 9000.

    a. development organizations b. product based organi..
    c. service based orga.. d. hardware companies. e. none

    Reply
  15. hi,
    questions asked in test.

    1.write any 5 test metrics.

    what is test metrics.

    2. following artifacts r baselined.
    a.SRS b. Test cases c. minutes of meeting d.agenda for meeting e. defect reports.

    3. configuration reviews r not needed if regression test has been regourously applied during software integration.

    a. True b. False c. Incorrect Stmt d. None

    4. customers perspective view of quality is measured by

    a. surveys b. service level agreements c. recommend & loyalty d. all of these

    5. in the change control procedure major changes should be approved by

    a. change control board b. configuration control board c. senior manager d. It manager e. none

    6. what is test design and test procedure

    7. QC process involves

    a. defect prevention b. casual analysis c. review walkthrough, inspection d. statistical process control.

    and also send me related questions which v can predict in the interview by looking into these types of questions.

    thanx in advance

    Reply
  16. VERY GOOD POSTING! GREAT WORK!! KEEP THE MOMENTUM GOING!! WE ARE EAGERLY EXPECTING YOUR FUTURE GOOD POSTS…. CHEERS…JK

    Reply
  17. really its very useful for testing people who is not clarity on their doubts. I got so many clarifitions.
    articles r very interesting n used simple words to undstand nice work.
    thnks a lot. keep it up!

    Reply
  18. i would like to know onething that a tester when and which test, actually testing will start. .ex: he studied requirments sepcification docts then what is further test as a tester.will u plz answer for this question

    Reply

Leave a Comment