Software companies adapting an Agile methodology very fast & automation is one of the key factor which enable the team to align with the scrum process.
Entire Scrum Team needs to evaluate the automation requirement & automation should run for entire SDLC which Includes automated unit tests, nightly builds and scripts that can validate build files, database, C LI,GUI, configurations and functionalities
Let us look into below scenario
In ABC company, there is a team which work on a desktop application having GUI & CLI interface, decides to create an automation framework which should completely cover there test plan.
They call a meeting & discuss about the number of test cases & their types which are executed. Everyone provides their input & they decide about the automation framework.
In the below article, I am trying to explain how QA team came up with their test plan input which has all the type of test cases then how automation team responded & proposed the automation framework & management’s analysis of automation ROI
I would try to explain you below scenarios
- Automation team’s initial Question to QA team
- QA team’s detailed requirements to Automation team
- Automation team’s further questions to limit the sizing of framework to avoid spending time & resources on unnecessary test automation
- QA team’s response
- Automation Team proposal
- Going on activities while running the test cycle
- Automation ROI analyzed by management
Automation team’s initial Question to QA team
- What is the scope they are looking?
- How many test cases must be automated?
- How complex is the initial testbed, do they want testbed to be created automatically
- What is the scope of GUI , CLI & API automation
QA team’s detailed requirements to Automation team
| Functional test case | 200 | Non-functional test case | 65 |
| GUI test case | 35 | Installation/Upgrade-Downgrade test case | 15 |
| CLI test cases | 30 | Load test case | 20 |
| Integrated/System/functionality test cases | 60 | Performance test case | 20 |
| Smoke tests | 20 | ||
| Sanity tests | 30 | ||
| Regression tests | 25 |
Automation team’s further questions to limit the sizing of framework to avoid spending time & resources on unnecessary test automation
- What are the most repetitive test cases
- Which modules are frequently changed, as automation of those modules would always need lot changes & that should be avoided & those type of test cases should be run manually
- Which functionalities have very high complexity, we shouldn’t automate those test cases as that would need lot of resource effort & time
- If application has stable GUI & CLI & there is only new functionalities introduced in new release than we can avoid the GUI/CLI automation & should use API for automation framework
QA team’s response
QA team had come up with the below information & asked automation team to act accordingly
| Priority | ||
| 1 | Smoke test | As soon as team gets build they want to run a test suit which should execute all the smoke test case so that they can quickly understand weather this is testable good build or not |
| 2 | System/Integrated test | This are the main functionality test & need most focus hence automation is needed |
| 3 | Regression test | As & when there are fixes introduced in code, a regression automation suit must be run |
Automation team proposal
Automation team proposed that they will create a test suit & implement all the scenarios following below points
- Automation code will be written in Java, It will directly use the product API to check any functionality e.g. if we want to login in to application we will not use GUI login page or CLI login terminal but will be login using the “connection” API. This approach will be faster &efficient;however, it won’t test anything on GUI/CLI interface
- Team will work on TestNG/Junit framework , which can prioritize test cases to be run, check the error code, pass/fail the test case as per the error code & create the detailed report
- After executing each test case logs, screenshots & results would be generated & once complete test suit is run, an email will be sent with the test report to all team members & management
Ongoing activities while running automation suit:
Team faced setup issues like network problem, application issues which affect test automation, cleanup issues like necessary changes in the configuration after any failure & timeout problem, which automation team keep fixing & improve the framework
Analyze the code coverage with the current automation framework & try to leverage it by automating new test cases or including more scenarios in current suit.
Automation ROI analyzed by management
Area where company invested on automation
- Paid tool: (Though in the above scenario whatever language or tools (Java & TestNG) I considered are free 🙂
- Training on different automation tools
- Product Application training to automation team
Returns after successfully implanting automation framework for entire SDLC
- Overnight tests could be run which saved time
- Testing reports were emailed to team immediately after test suit completion
- More test coverage in same or less time
- More repeatable tests
- Faster test coverage
Now after considering the above points, it gives us the idea how initial planning is done for automation, how QA team contribute in it. What are the areas of more focus?
There are many types of Automation tools & scripting languages are present in market. While automation we may follow many other approaches &there can be many other testing requirements. It is not possible to cover each & everything here, I hope above scenario has given you some insight about the automation framework
Thanks for reading the article, if you have any questions please ask in comments or send email on gagandeep@gsbitlabs.com