Agile vs DevOps (CI/CD) – Introduction & Comparison

 In today’s competitive world, the customers look forward to the latest releases of software with new and improved features incorporated. The organizations strive to deliver the best to their customers in the shortest possible time. This is only possible when there is a process in place to incorporate changes on an ongoing basis.

DevOps is a practice of continuously integrating and deploying the code changes in short cycles in order to facilitate faster builds and release versions. This gives rise to the need for Continuous Testing to validate all the changes implemented.

DevOps is a term formed from two concepts – Development and Operations. It is not just a term. It is an ideology that encompasses various concepts and development practices which are prevalent in Software development world. The concept of agile development was introduced in the 1990s. With changes in development approach, the testing approach needed a shift too, in order to align with the changes. Testing process also evolved to be in sync with the entire software development ecosystem. You can read our blog on “Evolution of software testing” for a quick overview of this evolution process.

Continuous Regression or Continuous Testing plays a key role in CI/CD tool chain. Infact, we have written a detailed analysis on how Webomates CQ effortlessly adds Continuous Testing to any system. Click here to read more about it. In this particular article, we will talk about DevOps and Agile methodology.

DevOps (CI/CD)

Definition: DevOps is a religion, a culture, a set of good practices, usage of right frameworks and tools which enable us to achieve Continuous Integration and Deployment. The objective is to strive towards 100% automation between code check-in and bug-free Deployment on production.

Description: Broadly there are following stages in CI/CD: Build-Code Review-Deploy-Test (where last two, ‘Deploy’ and ‘Test’ are repeated for several environments like test, pre-production, production, etc.). As a part of CI/CD as soon as the code is checked in by the developer, It’s automatically picked up for build and code review.

The successful build is then deployed on a test environment followed by running a suite of automated tests. Based on the quality gates (e.g. zero Priority 1 issues, two Priority 2 issues), the build is then deployed on the next environment, say pre-production and henceforth. The result of each step is generally mailed to the relevant stakeholder for information/action.

Tools: The most commonly used tools for CI/CD framework are Jenkins, TeamCity, and Bamboo. Common tools that one must be aware of are Jenkins, GIT, Ant, Maven, SonarQube, Ansible, Check, Puppet, Docker, Kubernetes, etc.  Read for more Information : Agile vs DevOps

While DevOps and Agile go hand in hand, it’s imperative for the organization to understand the importance of each of these methodologies independently and based on their eco-system and need one should emphasize the desired area appropriately.

Various organizations are moving towards ‘Agile DevOps’, where Agile and DevOps enable people, processes and technology working together to meet business goals. In addition to select a validation method to add Continuous Testing to the CI/CD toolchain click here for a demo of the Webomates CQ platform.

Examples of Ad-hoc testing

A very random testing plan which can be executed to find random defects is known as ad hoc testing. Know more about ad hoc testing in this article – Ad Hoc testing explained

Lets look at some of the Examples of Ad-hoc testing

• To test the web application responsiveness, tester can randomly open the web application in various browsers like Firefox, Internet explorer or any mobile devices.
• Most common example is entering random input in the text boxes to test validation of the input field.
• Testing the application on different platform to test cross-platform compatibility of the system.
• For mobile app testing, the perfect example of ad hoc testing will be opening up the app and clicking random buttons or text to perform random functions.
• Checking the web application functionality with respect to back button of the browser.
• Random power off the mobile / system while doing any activity on the application being tested.

There can be many more such examples of ad-hoc testing

Webomates has integrated solution Webomates CQ which helps companies to test the Mobile app properly and with effectiveness

Get in touch with our experts @webomates

Test automation: Are you taking a holistic approach

 In an increasingly agile world, test automation aids in improving feature velocity and reducing the duration of release cycles. 

Test automation is a technique to automate predefined repetitive testing tasks, using various test automation tools and testing scripts. 

Test automation is the key to continuous testing and has marked benefits in terms of accuracy, scalability, dependability, enhanced test coverage, time and efforts saving.

Pros and cons of test automation

Besides, the well-known benefits attached with test automation, let us list down what extra it has to offer to the teams and the customers, without losing the focus on what can be done to improve the automation process.

In order to reap maximum benefits from test automation, we need to address the issues that have been setting roadblocks in achieving success.

Initial setup issues – time consuming and expensive

Automation testing is not some magic. It involves careful analysis, design and development of a framework, keeping in mind licensing, development and operational costs. It is no mean feat getting all the stakeholders on the same page, to see the long term benefits of test automation vis-à-vis the initial setup cost.

Talent acquisition – finding the right talent

QA team should be well equipped with the necessary technical skills to design and maintain the test framework. It is a time consuming exercise to acquire the right skill set or train the existing team. But once done, it adds significant value in terms of designing and test execution.

Keeping the tests up to date – outdated data scripts

Keeping the test case repository up to date is a herculean task, especially for large scale projects. For every change in functionality, the related test cases need to be updated. Any change in the test environment or third party tools leads to modification of test cases, to reflect any environmental or configuration impact. Identifying the affected test cases, modifying and verifying them again is a complex and error prone process.

Time intensive exercise – Automating correctly

Identifying, automating and then executing the right test cases for a test cycle is an important step of test planning. However, it is often overlooked and as a result testing turns into a time intensive process.  and it then takes hours to execute the whole test suite. 

Also, the regression cycles have to be effective and optimized, especially when there are changes in test cases.

Triaging through false positives

False positives are one of the major challenges in automation testing, especially with higher release frequencies with agile and CI/CD. It undermines the value of automation and introduces a tremendous amount of effort to triage the failures. The effort and time lost in segregating the false positives from real results is taxing on the whole process. For more details, read our article “The bane of automation – false failures”.

Finding the right automation tools

Identifying the right tool for automation is a challenging task. It has to be a customizable multifaceted tool which covers all possible aspects of testing, and of course, is pocket friendly. There are many commercially available tools in the market. But which one ticks all the boxes? Read for more : Test automation

Webomates’ AI defect predictor ensures that no false positives are reported, thus saves precious man hours that are spent in triaging the defects.If this has piqued your interest regarding Webomates’ CQ service and you want to know more, then please click here and schedule a demo, or reach out to us at info@webomates.com

Testing Defect

 Over the years, software development and testing have undergone several stages of evolution. Every new era dawned with new hopes, but at the same time was dotted with new challenges. However, one thing that has remained constant – the software defects. Be it a small-scale project or a colossal application, defects are everyone’s nightmare. 

A software defect is an error, flaw, or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways — Wikipedia.

An efficient defect management ecosystem, with an effective defect(Testing Defects) prevention process, pre-testing defect removal, intensive software testing can significantly bring down the overall development costs and can also save a lot of time.

Ward Cunningham coined the term “technical debt”, which essentially refers to the increasing cost for defect removal accrued over a period of time, due to oversights during the design and development process. The overall cost of identifying and rectifying defects down the line can grow exponentially if the loopholes in the process of software development are not plugged.

What is the main origin of defects?

Debugging is a painful exercise, therefore identifying the origin of the software defects is extremely important in order to take timely corrective measures. The rate at which the new features and modifications are introduced, it becomes even more important to ensure that the builds are defect-free before any new changes are incorporated. 

Traditionally, the focus has been more on code defects, but the fact is that the defects can originate as early as requirements gathering/analysis, and can be introduced at the time of software design too. If the base is defective, how can an end product be of high quality? The worst is to realize this much later and then start firefighting, which obviously is an expensive exercise.

In this article, we will explore the possible origins of defects and how we can help you in handling them.

Requirements misinterpretation

Understanding customer needs and translating them to technical requirements lays the foundation of an application. If done incorrectly, the resulting design will be faulty. The code may be defect-free, but it was built on the wrong premise. Hence, it is a good idea to introduce thorough requirement-related reviews to nip any potential defect at the onset only. 

Requirement understanding and misinterpretation can be due to either poor communication between the teams and clients or due to immature requirement gathering processes.

Design issues

There are times when software architects end up translating the functional requirements to design specifications incorrectly, thus introducing a defect inadvertently. Besides, a poor design with not enough focus on basic software architecture, inaccurate concepts, database efficiency, leads to various issues, which further result in multiple defects at different points of time. Also, testers may not take into account all the possibilities and design not enough test cases, or they may misunderstand the functionality and prepare a faulty test plan. Hence, it is important to have an inclusive approach and ensure that both, the developers and the testers, are a part of the requirement understanding and designing process.

Execution flaws

Translation of design to a working code gets the maximum scrutiny when it comes to looking for defects. There is nothing called flawless development. It is marred with several coding issues like,

• Using outdated legacy plugins/libraries 
• Using an untested set of code
• Bad fixes for previously detected defects, inadvertently introducing other issues
• Not adhering to standards

All these coding issues are breeding grounds for long-term issues.

Testing flaws

Software testing itself can have multiple issues, either due to an oversight or lack of understanding. These issues can further introduce defects, which may prove expensive. Some of them are listed below. 

• Missing tests for a functionality
• Not all aspects of testing are covered for a particular functionality
• False failures
• Incorrectly designed test cases

Reporting issues

What end purpose will testing serve if the reporting is not done correctly and efficiently? 

Outdated reports and lack of information sharing are the bane of the software testing process. Following are few areas of concern during test results reporting.

• Reports are not updated after every test cycle
• The defect is recorded incorrectly
• Incorrect reporting and sharing of test results 
• Inefficient defect triaging Read for more : Testing defects

Webomates CQ helps organizations in bridging the gap between the dream and reality of effective testing.If this has piqued your interest and you want to know more, then please click here and schedule a demo, or reach out to us at info@webomates.com. If you liked this blog, then please like/follow us Webomates or Aseem.

Top 5 Benefits of Codeless Testing

 Software testing has evolved rapidly over the last few decades. The transition from manual testing to automation testing was one of the key turning points in the history of software testing.

 When Selenium was first introduced in the early 2000s, it immediately gained popularity and became one of the most favoured tools for test automation. However, it came with a condition that the testers need to have knowledge of programming in order to reap maximum benefits from test automation. So, even though automation was supposed to “reduce” the testing workload, the testers ended up spending a lot of time writing code scripts for testing.

With the rising demand for efficient software with faster time to market, software testing became even more stringent. Automation, though helpful, came at a price in terms of resources and time. The year 2020, heralded the era of codeless testing, which ensures a high-quality deliverable while reducing the testing time significantly.

What is Codeless testing?

Codeless testing essentially means conducting automated testing without writing a code for the test script.

Codeless testing addresses the issues of regular test automation i.e.

The steep learning curve for testers
Mandatory to know to program
Zeroing on a suitable programming language to use
Extra time consumed in debugging in case there is a bug in the test script itself

Test case management is difficult in case of multiple changes
With Continuous testing being an integral part of the CI/CD process, the testing task becomes repetitive and time-consuming. Codeless testing accelerates the overall testing process, with a significant reduction of burden from the development and QA teams.

What makes codeless testing special?

Most codeless testing tools are built on top of Selenium and have a plethora of inbuilt features which aid the testing teams. The test engineers can leverage the power of these tools to conduct testing efficiently. Let us take a look at the benefits attached with codeless testing.

benifits.jpg

Easy to conduct with minimum learning

Easy-to-conduct-with-minimum-learning.jpg

Codeless testing enables test engineers with little or no programming knowledge to create, execute and maintain the test cases without investing time in learning to code. This not only saves a lot of time and efforts but also makes the whole testing process much more efficient.

Efficient test case management

Efficient-test-case-management.jpg
Earlier, a major chunk of time was spent in writing and reviewing the test scripts. Needless to say, it required a specific skill set too. With codeless testing, this is no longer a hurdle. The test engineers can generate test cases easily. Also, the non-technical stakeholders can easily understand the test cases and give their feedback. 

Any modifications in the functionality can be quickly incorporated, thus increasing the reusability and maintainability quotient of the test cases. Usage of AI/ML in managing, maintaining, healing and executing the test suites, with little or no human intervention is the way forward.

Resource saving

Resource-saving.jpg

Hiring the right fit for a testing job used to be a challenge. A candidate had to “know” coding besides having a thorough knowledge and good experience in testing. A lot of time and energy was spent looking for that perfect match. Codeless testing eases the burden of programming know-how.

Continuous testing, though a good process to incorporate, comes with its own baggage. 

The test engineer ends up spending a lot of time in regression testing, that there is little time left for important aspects like exploratory testing, which sometimes becomes a hurriedly done activity. With codeless testing, efforts and time involved are reduced, thus freeing up the major resources to focus on other important aspects of software development and testing processes.Read for more Information : codeless-testing

Is load testing stressful for you?

 Holidays – It is that time of the year when everybody travels to be with family. Online ticketing portals are under heavy load due to rampant booking. What if a site crashes during an ongoing transaction due to excessive traffic and server’s inability to manage it? Not only is it an embarrassment, but loss of business as well. This is a very basic example, but there are mission critical software which cannot afford to have such glitches. Such scenarios are precisely the reason that load testing the application is extremely important.

Load testing is a subset of performance testing which gauges the functioning of a system under different load conditions. It essentially identifies the bottlenecks and potential breaking point by checking for response time, throughput rates, and resource utilization when subjected to peak load.

Why is it so important to conduct load testing?

Why is it important to have load testing protocols in place?

Challenges of load testingLoad testing has certain challenges which, if not addressed, reduce the efficacy of the testing process.

How Webomates helps in addressing these challengesThere are many tools available in the market which aid in performing load testing.

• Some organizations develop it in house and customize as per their needs.
• Some use open source tools. They are rudimentary but fulfil the required needs.
• Enterprise level tools which have more capabilities.

It is important to consider the following factors while selecting a load testing tool.

• Ease of Use
• Ease of integration 
• Scalability 
• Accuracy and accessibility

Webomates CQ checks all these criteria, and even more. Let us explore how.

Ease of Use: There is no need for having skilled resources putting in weeks of coding to write load test scripts. CQ converts functional test scenarios to performance/load testing cases just with the click. CQ users can effortlessly convert any functional test to load test just by enabling “performance check” for that particular test case. It allows you to configure the number of virtual users for testing. Once this configuration is done, the load testing process can be invoked.

Ease of integration: Our tool seamlessly integrates in your CI/CD pipeline and has a facility for a complete package for functional and performance tests. It provides codeless and effortless load testing, which is completely on demand and can be invoked as and when needed. Read for More Information : functional testing and non functional testing

If you are interested in learning more about Webomates’ CQ service please click here and schedule a demo, or reach out to us at info@webomates.com.

At Webomates, we continuously work to evolve our platform & processes in order to provide guaranteed execution, which takes testing experience to an entirely different level, thus ensuring a higher degree of customer satisfaction.

Crowdsourced Testing Vs Manual Testing

Crowdsourced testing is an emerging trend in the field of software testing. It is an approach which involves different people with different backgrounds across the globe in testing process. 

The testers need not be mandatorily from testing field or with an IT background, but having a basic knowledge of using a computer and access to the internet is sufficient. The testers are connected through a crowdsourcing platform and tests are conducted by multiple individuals at different locations.

It adds multiple dimensions to manual testing process by providing a diverse and temporary workforce, thus aiding the organizations with limited testing bandwidth in achieving quick results.

It is natural to assume that crowdsourcing is similar to manual testing and sometimes deemed as an extension of manual testing, but they have certain fundamental differences.
This blog helps in understanding how crowdsourced testing is different from manual testing approach.

Crowdsourced Testing Manual Testing Execution Speed Fast Slow Scalability HighLess Device Coverage High Limited Localization testing High Limited Varied End user testingHighLimitedQuality of TestingMediumHighQuality of Exploratory TestingLowHighTest BrittlenessLow ( breaks easily)HighPeople

 ManagementLowHighCostLessHighTest Case ContentVery detailedTo the pointCommunication between testersVery limitedHighConfidentiality issuesYesNoFraud TrackingThorough auditsModerate audits
Conclusion

Crowdsourced testing is not a replacement for in house testing. It accentuates the quality of testing by providing additional dimensions to the whole process.

Crowdsourcing offers different values as compared to manual and automation testing. Combining all three forms of execution offers a higher quality, faster overall execution time and the highest quality of defects. However, it comes with some challenges and for better understanding of how Webomates handles them, read “Overcoming the challenges of crowdsourced testing” on our website.

Read for more : Crowdsourced testing

If you are interested in learning more about the services offered by Webomates then please click here and schedule a demo, or reach out to us at info@webomates.com. You can also avail a free trial by clicking here.

Crowdsourced Testing Vs Automation Testing

Crowdsourced testing is an innovative way to reap the benefits of crowdsourcing and cloud platform. It enlists the services of a community of testers, who collaborate online to test an application in addition to already existing testing processes of an organization.

 

 

 

Crowdsourced testing involves leveraging on benefits of having a heterogeneous mix of people using diverse testing environments, which results in low cost, efficient and fast testing process.

 

 

 

Automation,on the other hand, achieves precision and speed that is difficult to match. Furthermore, integration with a CI/CD is easy with automation. However, automation, though efficient, does not have the creativity and vision of humans.

 

 

 

Human supervision and intervention cannot be completely excluded from the testing process. And there may be scenarios where in house resources are scarce and organizations look for suitable alternatives like Crowdsourcing as it is easier to adapt to modified test cases.

 

 

 

In our previous article we talked about how crowdsourcing is different from manual testing approach. We will now examine how Crowdsourced testing is different from automation testing approach

 

 

 

Crowdsourced TestingAutomation TestingScalabilityHighHighRe-run CostHighLow Device Coverage HighHigh Localization testing HighLowVaried End user testing HighLowQuality of TestingMedium HighExecution SpeedMediumFastCost of TestingLowHigh Test Case ContentVery detailedTo the pointAdaptability to change in test casesHighLessSupport for Exploratory Testing

 

 

 

YesNoNeed for proficiency in programming NoYesConfidentiality issuesYesNone Fraud TrackingThorough auditsNot required

 

 

 

Conclusion

 

Both types of testing, i.e. Automation and Crowdsourced, have their own range of benefits at the same time they pose certain challenges as well.

 

 

 

Initial cost of setting up an automated process for testing acts as a deterrent to many cost conscious organizations. However, it does prove beneficial in the long term, especially with CI/CD. It is important to identify and choose the right automation tool for long term benefits.

 

Crowdsourced testing is a powerful approach which harnesses the power of crowd talent and wisdom.It offers different values as compared to manual and automation testing. It scales as compared to manual testing and is more flexible than automation testing .

 

 

 

 

 

However, it comes with some challenges and for better understanding of how Webomates handles them, read “Overcoming the challenges of crowdsourced testing” on our website. Read for more Information : crowdsourced testing

 

 

 

If you are interested in learning more about the services offered by Webomates then please click here and schedule a demo, or reach out to us at info@webomates.com. You can also avail a free trial by clicking here.