Case Studies


 
What we noticed from the first
day with Subject7 was the ease
of use, and how fast testers
were able to ramp up, even
without any prior knowledge of
testing…. We reduced our
regression time by more than
90% and freed up our testers for
exploratory testing.
— Dovel
 

Significantly Reduced Regression Testing Time

Dovel empowers customer missions with innovative technology solutions. By applying proven software development methodologies combined with leading edge technologies and approaches, customers are provided solutions that will meet the needs of today and into the future. Dovel has been instrumental in the design and implementation of some of the most mission critical and innovative systems in government today. They bring strong experience across the software development life cycle as well as specialized experience in Health IT, Education,
Grants Management, and Regulatory markets. Dovel has been appraised at CMMI® Maturity Level 3, and its annual quality report receives accolades from industry leaders such as Capers Jones, who has listed Dovel in the category of “Companies That Utilize Best Practices” along with IBM, Amazon, Google, and Apple.

Challenge

Dovel has supported several projects within the federal government. One specific project required 936 man hours of regression testing time, and six testing resources were allocated towards this regression effort. Dovel delivered development, operations, maintenance, and enhancement support for this project and continues to provide ongoing regression testing on the system. The project was a series of web-based applications integration into a single management systems servicing twenty (20) Federal agencies. Dovel, as a cutting-edge technology company, employed testers with a broad range of skill sets, but preferred and sought a solution that did not require such a substantial manual effort, so that testers could focus on providing subject matter expertise rather than spending time learning a proprietary programming language to conduct automated testing. It was critical for the Dovel team to have a cost-effective solution that met the following criteria: easy to install and configure, scaled out of the box, provided cross-browser and cross-environment testing,
and did not require any knowledge of coding or scripting.

Solution

After evaluating a series of test automation products, the Dovel team decided to pilot Subject7's platform. The team was quickly impressed with the functionality and ease of use. With only introductory training, the team was creating and running test cases almost immediately. Subject7 delivered:

  • A Cost-effective solution that could grow with Dovel as their needs evolved

  • The convenience of Software as a Service (SaaS), with no installation necessary and seamless upgrades

  • Data-driven testing out-of-the-box

  • Scalable execution where additional virtual machines could easily be added to encompass all of the regression testing over different browsers and development environments

After pilot, Dovel selected Subject7 for all of their automated and load testing needs.

Results

After an initial investment of time to create a library of repeatable test cases, the team observed dramatic increases in the speed with which they could fully implement regression testing. Production level defects were reduced significantly, which was directly attributed to test automation. The testing team is now able to focus their energy on testing new functionality, while the Subject7 test automation platform runs all the regression tests. Dovel reported the following, as a result of working with Subject7:

  • Reduced regression testing time from 936 man hours to only 7 hours

  • Reduced 6 testers to 4 virtual machines

  • Executes regression tests nightly as opposed to only once per release

Today, with more than a year as a successful partner working directly with Dovel, the organization has initiated the use of Subject7’s automation, load, and manual
testing capabilities across additional products and services. Subject7 continues to train and provide support to the Dovel team, and has provided open dialogue about future product enhancements. Dovel has continually been impressed with Subject7’s commitment to customer service and speed in which feature enhancements are introduced.

 

We increased our productivity 90% and saved 42% on adding headcount.
— Leidos
 

Increased Productivity and Efficiency in Software Test Automation for the Department of Health and Human Services

Leidos, formerly known as Science Applications International Corporation (SAIC), is a FORTUNE 500® science and technology solutions leader working to address some of the world's toughest challenges in national security, health and engineering. The Company's 21,000 employees support vital missions for our government and the commercial sector, develop innovative solutions to drive better outcomes and defend our Nation's digital and physical infrastructure from 'new world' threats. Headquartered in Reston, Virginia, Leidos reported annual revenues of approximately $5.77 billion for its fiscal year ended January 31, 2014 after giving effect to the spin-off of the company's technical services and information technology business.

Business Situation

Leidos is supporting several projects with the Department of Health and Human Services as part of its Health Services contracts. These projects and others require more than 2,000 test cases. Leidos is providing development, operations, maintenance, and enhancement support for these programs and provides ongoing regression testing on these systems. It was critical for the Leidos team to have a cost-effective solution that was easy to install and configure, scaled out of the box, provided cross-browser and cross-environment testing, and did not require any significant knowledge of coding or scripting.

Technical Situation

These Leidos projects included 160 applications written on a variety of different technologies including, but not limited to, .NET and Java technology stacks. Some of the test servers for these projects were available over the Internet, while others were not. Leidos had testers with various skillsets but preferred a solution that did not require extensive technical experience so that its testers could focus on providing subject matter expertise rather than spending time learning proprietary programming language to conduct automated testing.

Solution

After evaluating several test automation COTS products in the market, the Leidos team decided to pilot Subject7's platform for a few months. Subject7 made the necessary enhancements to its offerings to better support Leidos and its clients. The Leidos Health team later established its Testing Excellence Center and standardized on Subject7 as its primary automation tool. Subject7 trained the Leidos Health team and provided support throughout this timeframe.

Results

Subject7's solution provided these core features and capabilities:

  • Deployed as Software as a Service (SaaS) with no installation is necessary and seamless upgrades.

  • No technical knowledge or scripting experience needed.

  • Data-driven testing, out-of-the-box.

  • Scalable through the power of its cloud backbone -- eliminates the need to maintain different combinations of operating systems, browsers, and versions.

  • Portable exports on demand, using industry standards such as Selenium 2, xPath, and regular expressions.

  • Cost-effective and grows with Leidos on an as-needed basis.

The Leidos team experienced the following results, using Subject7:

  • Productivity went up by more than 90%.

  • Provided 42% savings in test automation resources.

  • Helped Leidos pursue additional opportunities.


moso_logo.png
Once we found Subject7, we were able to radically change our QA process. We focused 75% of our team’s time on stabilization in 2018, reducing defects by 66% by the end of the year.

- CTO of Motionsoft
 

Motionsoft Executes 1,600 Automated Test Cases Daily with Subject7

Motionsoft is the leading provider of software and billing silutions for the active lifestyle industry,  with more than 2,000 health and fitness customees  throughout the Americas and the UK. The software provides all the capabilities needed to manage a fitnesss center, including contract management, check-in, point-of-sale, financial reporting, business analytics and more.

Grant Otto is Motionsoft’s CTO, and Arinze Ukeh manages the company’s team of 16 testers and automation engineers.

Challenge

When Grant joined the company in early 2016, there was an existing automation project in place for the MoSo 3.0 product build and rollout. The company had 2,000 manual test cases that took weeks to run. In addition, the test cases had been built by an offshore team that wasn’t fully integrated with the company’s in-house team. Grant cancelled the project and continued to use internal resources for manual testing throughout the remainder of the product build.

In early 2018, MoSo 3.0 had been live for two customers for several months, when the need arose to harden and stabilize the platform more. The company needed to be able to find more defects because of the limitations of manual testing, which prevented them from running certain test cases. Grant needed to automate approximately 800 to 1,000 smoke test cases.

“Among other things, our clients use MoSo 3.0 for billing their customers, so we take QA automation very seriously to ensure a high quality of service on our platform,” Grant noted.

Solution

Motionsoft considered several test automation options, including a home-grown solution, as well as Subject7. After seeing an impressive demo and checking customer references, Grant chose Subject7.

He identified several initial test automation requirements as a pilot, including the automation of 350 test cases within six months, and an ultimate goal of automating 26,000 test cases over time.

Results

Thanks to Subject7, Motionsoft was able to make product quality a high priority in 2018.

“Once we found Subject7, we were able to radically change our QA process,” said Grant. “We focused 75% of our team’s time on stabilization in 2018, reducing defects by 66% by the end of the year. I’m frankly shocked at our overachievement in the automation area.”

After easily surpassing the original goal of automating 350 test cases, Motionsoft now executes 1,600 automated cases on a daily basis.

“We do a monthly release,” said Grant. “We have one cycle of full regression testing and one smoke test after the deployment.”

“With Subject7 we can run 1,200 cases every day, Monday through Friday,” Grant added. “If these cases had to be run manually, we could only run them once a month, with each scheduled release.”

No Coding Skills Necessary

“Subject7 is a very user-friendly test automation solution,” Grant added “There’s simply no coding needed to write and execute automation tests. If you know basic Xpath, you can do your own test scripts. It’s just very intuitive and certainly not difficult to learn.”

Ensuring Quality in an Agile World

“You really can’t adopt an agile methodology without test automation,” said Grant. “You simply don’t have time to keep running tests in the waterfall method. Anyone trying to do releases monthly will find that it’s just not possible to release a quality product without testing automation.”

“In this this day and age, there’s enough technology out there with integration into your development process, so that all of these tests can be triggered automatically by code merges,” Grant concluded.


csg_logo.png
 

80% Reduction in Regression Testing Time

Cobb Systems Group (CSG) is an award-winning, minority-owned business enterprise headquartered in Montgomery County, Maryland. CSG provides both technology for talent and talent for technology.

CSG’s patented talent engagement and analytics platform, ProForm™, delivers the features of traditional job boards, resume builders, and applicant tracking systems along with robust assessment, matching, and insight capabilities. ProForm™-enabled practices reduce time and cost to hire, enhance candidate pipeline quality and job fit, increase inclusion, and lower ongoing workforce costs.

Challenge

Within ProForm™, privileges determine what menus, screens, and features are accessible to each user. ProForm™ has over 250 privileges. This level of granularity allows for thousands of custom configurations. These configurations must be regression-tested across multiple datasets and browsers, further expanding the number of test scenarios to be run with each code release or infrastructure change.  CSG needed an automated test platform that would ensure test scenario coverage within release timeframes.

Solution

CSG evaluated several commercially installed, software as a service (SaaS), and open source solutions.  However, all other solutions effectively required test automation resources to be software developers. In Subject7, CSG found a product that, out-of-the-box is:

  • Easy to implement (deployed as SaaS), with no installation necessary and seamless upgrades

  • Scalable with the power of Amazon cloud

  • Simple to use (wizard-driven test script creation), requiring no software development or scripting skills

  • Powerful in its ability to consistently replicate the execution of multiple test suites across multiple browsers with real-time status reporting

  • Robust in its ability to support data-driven testing wherein one test scenario is automatically re-run with varying inputs

Results

Using Subject7, CSG was able to automate privilege regression testing and:

  • Decrease regression testing cycle time from 2-3 weeks to 3-4 days (this time can further be reduced by employing additional virtual machines)

  • Enhance team productivity

  • Free valuable test resources for other activities 

  • Ensure test execution consistency and coverage 

    CSG continues to improve its continuous integration processes using Subject7 as a critical component in its test and validation efforts.