Case Studies


 
What we noticed from the first
day with Subject7 was the ease
of use, and how fast testers
were able to ramp up, even
without any prior knowledge of
testing…. We reduced our
regression time by more than
90% and freed up our testers for
exploratory testing.
— Dovel
 

Significantly Reduced Regression Testing Time

Dovel empowers customer missions with innovative technology solutions. By applying proven software development methodologies combined with leading edge technologies and approaches, customers are provided solutions that will meet the needs of today and into the future. Dovel has been instrumental in the design and implementation of some of the most mission critical and innovative systems in government today. They bring strong experience across the software development life cycle as well as specialized experience in Health IT, Education,
Grants Management, and Regulatory markets. Dovel has been appraised at CMMI® Maturity Level 3, and its annual quality report receives accolades from industry leaders such as Capers Jones, who has listed Dovel in the category of “Companies That Utilize Best Practices” along with IBM, Amazon, Google, and Apple.

Challenge

Dovel has supported several projects within the federal government. One specific project required 936 man hours of regression testing time, and six testing resources were allocated towards this regression effort. Dovel delivered development, operations, maintenance, and enhancement support for this project and continues to provide ongoing regression testing on the system. The project was a series of web-based applications integration into a single management systems servicing twenty (20) Federal agencies. Dovel, as a cutting-edge technology company, employed testers with a broad range of skill sets, but preferred and sought a solution that did not require such a substantial manual effort, so that testers could focus on providing subject matter expertise rather than spending time learning a proprietary programming language to conduct automated testing. It was critical for the Dovel team to have a cost-effective solution that met the following criteria: easy to install and configure, scaled out of the box, provided cross-browser and cross-environment testing,
and did not require any knowledge of coding or scripting.

Solution

After evaluating a series of test automation products, the Dovel team decided to pilot Subject7's platform. The team was quickly impressed with the functionality and ease of use. With only introductory training, the team was creating and running test cases almost immediately. Subject7 delivered:

  • A Cost-effective solution that could grow with Dovel as their needs evolved
  • The convenience of Software as a Service (SaaS), with no installation necessary and seamless upgrades
  • Data-driven testing out-of-the-box
  • Scalable execution where additional virtual machines could easily be added to encompass all of the regression testing over different browsers and development environments

After pilot, Dovel selected Subject7 for all of their automated and load testing needs.

Results

After an initial investment of time to create a library of repeatable test cases, the team observed dramatic increases in the speed with which they could fully implement regression testing. Production level defects were reduced significantly, which was directly attributed to test automation. The testing team is now able to focus their energy on testing new functionality, while the Subject7 test automation platform runs all the regression tests. Dovel reported the following, as a result of working with Subject7:

  • Reduced regression testing time from 936 man hours to only 7 hours
  • Reduced 6 testers to 4 virtual machines
  • Executes regression tests nightly as opposed to only once per release

Today, with more than a year as a successful partner working directly with Dovel, the organization has initiated the use of Subject7’s automation, load, and manual
testing capabilities across additional products and services. Subject7 continues to train and provide support to the Dovel team, and has provided open dialogue about future product enhancements. Dovel has continually been impressed with Subject7’s commitment to customer service and speed in which feature enhancements are introduced.

 

We increased our productivity 90% and saved 42% on adding headcount.
— Leidos
 

Increased Productivity and Efficiency in Software Test Automation for the Department of Health and Human Services

Leidos, formerly known as Science Applications International Corporation (SAIC), is a FORTUNE 500® science and technology solutions leader working to address some of the world's toughest challenges in national security, health and engineering. The Company's 21,000 employees support vital missions for our government and the commercial sector, develop innovative solutions to drive better outcomes and defend our Nation's digital and physical infrastructure from 'new world' threats. Headquartered in Reston, Virginia, Leidos reported annual revenues of approximately $5.77 billion for its fiscal year ended January 31, 2014 after giving effect to the spin-off of the company's technical services and information technology business.

Business Situation

Leidos is supporting several projects with the Department of Health and Human Services as part of its Health Services contracts. These projects and others require more than 2,000 test cases. Leidos is providing development, operations, maintenance, and enhancement support for these programs and provides ongoing regression testing on these systems. It was critical for the Leidos team to have a cost-effective solution that was easy to install and configure, scaled out of the box, provided cross-browser and cross-environment testing, and did not require any significant knowledge of coding or scripting.

Technical Situation

These Leidos projects included 160 applications written on a variety of different technologies including, but not limited to, .NET and Java technology stacks. Some of the test servers for these projects were available over the Internet, while others were not. Leidos had testers with various skillsets but preferred a solution that did not require extensive technical experience so that its testers could focus on providing subject matter expertise rather than spending time learning proprietary programming language to conduct automated testing.

Solution

After evaluating several test automation COTS products in the market, the Leidos team decided to pilot Subject7's platform for a few months. Subject7 made the necessary enhancements to its offerings to better support Leidos and its clients. The Leidos Health team later established its Testing Excellence Center and standardized on Subject7 as its primary automation tool. Subject7 trained the Leidos Health team and provided support throughout this timeframe.

Results

Subject7's solution provided these core features and capabilities:

  • Deployed as Software as a Service (SaaS) with no installation is necessary and seamless upgrades.
  • No technical knowledge or scripting experience needed.
  • Data-driven testing, out-of-the-box.
  • Scalable through the power of its cloud backbone -- eliminates the need to maintain different combinations of operating systems, browsers, and versions.
  • Portable exports on demand, using industry standards such as Selenium 2, xPath, and regular expressions.
  • Cost-effective and grows with Leidos on an as-needed basis.

The Leidos team experienced the following results, using Subject7:

  • Productivity went up by more than 90%.
  • Provided 42% savings in test automation resources.
  • Helped Leidos pursue additional opportunities.