Total Wine & More Reduces eCommerce Site Testing time by 60% with Subject7
Total Wine & More is the country‘s largest independent retailer of fine wine, with 193 superstores in 23 states and a workforce exceeding 4,000 employees. Each store typically carries more than 8,000 different wines from every wine-producing region in the world, as well as more than 2,500 beers, as well as more than 3,000 different spirits.
Srinivasa Telukutla (“Sri”) at the time was the Senior Manager, ecommerce platform at Total Wine & More, responsible for the company’s ecommerce platform.
In 2015, Total Wine ECommerce team spent a significant amount of time keeping code updated for Total Wine’s online store and mobile app. Testing was always a laborious, time-consuming task for Total Wine team, and they knew there must be a better way.
Lured by the initial appeal of open source, Total Wine tried building a home-grown testing automation tool that used a Selenium framework with Java.
After spending two years of trying to get this “free” solution to work, Total Wine team came to the stark conclusion that it just wasn’t a viable solution.
Total Wine initially learned about Subject7 via a referral, who worked as a contractor for another Subject7 customer. Total Wine team immediately concluded that it was exactly what they needed, after seeing how simple but effective it was as a test automation tool. He ran a pilot for a few weeks, and was convinced that Subject7 was exactly what he needed.
“It’s a very simple interface,” said Sri. “I can train anyone on the Subject7 tool. This really changes what kind of skillset I need with an automation tester.”
Ongoing use of Subject7 is also a snap for Total Wine. “It’s very easy to maintain our testing automation with Subjec7, particularly when compared to the direct open source Selenium code we tried using before,” said Sri.
Since implementing Subject7 in 2016, Total Wine & More has seen several business benefits to the improved speed and amount of testing QA team is now able to perform. These benefits include:
Decreased time to market – with automation, Total Wine team has reduced testing time by roughly 60%, and is running more tests in less time.
Improved quality of releases
Reduced cost – fixing issues is much less expensive now, since tests can be run easily and more frequently to find and correct issues quickly.
Having worked with Subject7 now for more than 2 years, Total Wine is very happy with the level of attention and service they have received from the company.
“The team at Subject7 has been really easy to work with,” said Sri. “They’re very nimble, collaborative, and quick to respond to our feature requests. Subject7’s CEO has given us his personal commitment to work with us on any potential issues, and that’s extremely valuable to us.”
Significantly Reduced Regression Testing Time
Dovel empowers customer missions with innovative technology solutions. By applying proven software development methodologies combined with leading edge technologies and approaches, customers are provided solutions that will meet the needs of today and into the future. Dovel has been instrumental in the design and implementation of some of the most mission critical and innovative systems in government today. They bring strong experience across the software development life cycle as well as specialized experience in Health IT, Education,
Grants Management, and Regulatory markets. Dovel has been appraised at CMMI® Maturity Level 3, and its annual quality report receives accolades from industry leaders such as Capers Jones, who has listed Dovel in the category of “Companies That Utilize Best Practices” along with IBM, Amazon, Google, and Apple.
Dovel has supported several projects within the federal government. One specific project required 936 man hours of regression testing time, and six testing resources were allocated towards this regression effort. Dovel delivered development, operations, maintenance, and enhancement support for this project and continues to provide ongoing regression testing on the system. The project was a series of web-based applications integration into a single management systems servicing twenty (20) Federal agencies. Dovel, as a cutting-edge technology company, employed testers with a broad range of skill sets, but preferred and sought a solution that did not require such a substantial manual effort, so that testers could focus on providing subject matter expertise rather than spending time learning a proprietary programming language to conduct automated testing. It was critical for the Dovel team to have a cost-effective solution that met the following criteria: easy to install and configure, scaled out of the box, provided cross-browser and cross-environment testing,
and did not require any knowledge of coding or scripting.
After evaluating a series of test automation products, the Dovel team decided to pilot Subject7's platform. The team was quickly impressed with the functionality and ease of use. With only introductory training, the team was creating and running test cases almost immediately. Subject7 delivered:
A Cost-effective solution that could grow with Dovel as their needs evolved
The convenience of Software as a Service (SaaS), with no installation necessary and seamless upgrades
Data-driven testing out-of-the-box
Scalable execution where additional virtual machines could easily be added to encompass all of the regression testing over different browsers and development environments
After pilot, Dovel selected Subject7 for all of their automated and load testing needs.
After an initial investment of time to create a library of repeatable test cases, the team observed dramatic increases in the speed with which they could fully implement regression testing. Production level defects were reduced significantly, which was directly attributed to test automation. The testing team is now able to focus their energy on testing new functionality, while the Subject7 test automation platform runs all the regression tests. Dovel reported the following, as a result of working with Subject7:
Reduced regression testing time from 936 man hours to only 7 hours
Reduced 6 testers to 4 virtual machines
Executes regression tests nightly as opposed to only once per release
Today, with more than a year as a successful partner working directly with Dovel, the organization has initiated the use of Subject7’s automation, load, and manual
testing capabilities across additional products and services. Subject7 continues to train and provide support to the Dovel team, and has provided open dialogue about future product enhancements. Dovel has continually been impressed with Subject7’s commitment to customer service and speed in which feature enhancements are introduced.
Increased Productivity and Efficiency in Software Test Automation for the Department of Health and Human Services
Leidos, formerly known as Science Applications International Corporation (SAIC), is a FORTUNE 500® science and technology solutions leader working to address some of the world's toughest challenges in national security, health and engineering. The Company's 21,000 employees support vital missions for our government and the commercial sector, develop innovative solutions to drive better outcomes and defend our Nation's digital and physical infrastructure from 'new world' threats. Headquartered in Reston, Virginia, Leidos reported annual revenues of approximately $5.77 billion for its fiscal year ended January 31, 2014 after giving effect to the spin-off of the company's technical services and information technology business.
Leidos is supporting several projects with the Department of Health and Human Services as part of its Health Services contracts. These projects and others require more than 2,000 test cases. Leidos is providing development, operations, maintenance, and enhancement support for these programs and provides ongoing regression testing on these systems. It was critical for the Leidos team to have a cost-effective solution that was easy to install and configure, scaled out of the box, provided cross-browser and cross-environment testing, and did not require any significant knowledge of coding or scripting.
These Leidos projects included 160 applications written on a variety of different technologies including, but not limited to, .NET and Java technology stacks. Some of the test servers for these projects were available over the Internet, while others were not. Leidos had testers with various skillsets but preferred a solution that did not require extensive technical experience so that its testers could focus on providing subject matter expertise rather than spending time learning proprietary programming language to conduct automated testing.
After evaluating several test automation COTS products in the market, the Leidos team decided to pilot Subject7's platform for a few months. Subject7 made the necessary enhancements to its offerings to better support Leidos and its clients. The Leidos Health team later established its Testing Excellence Center and standardized on Subject7 as its primary automation tool. Subject7 trained the Leidos Health team and provided support throughout this timeframe.
Subject7's solution provided these core features and capabilities:
Deployed as Software as a Service (SaaS) with no installation is necessary and seamless upgrades.
No technical knowledge or scripting experience needed.
Data-driven testing, out-of-the-box.
Scalable through the power of its cloud backbone -- eliminates the need to maintain different combinations of operating systems, browsers, and versions.
Portable exports on demand, using industry standards such as Selenium 2, xPath, and regular expressions.
Cost-effective and grows with Leidos on an as-needed basis.
The Leidos team experienced the following results, using Subject7:
Productivity went up by more than 90%.
Provided 42% savings in test automation resources.
Helped Leidos pursue additional opportunities.
Motionsoft Executes 1,600 Automated Test Cases Daily with Subject7
Motionsoft is the leading provider of software and billing silutions for the active lifestyle industry, with more than 2,000 health and fitness customees throughout the Americas and the UK. The software provides all the capabilities needed to manage a fitnesss center, including contract management, check-in, point-of-sale, financial reporting, business analytics and more.
Grant Otto is Motionsoft’s CTO, and Arinze Ukeh manages the company’s team of 16 testers and automation engineers.
When Grant joined the company in early 2016, there was an existing automation project in place for the MoSo 3.0 product build and rollout. The company had 2,000 manual test cases that took weeks to run. In addition, the test cases had been built by an offshore team that wasn’t fully integrated with the company’s in-house team. Grant cancelled the project and continued to use internal resources for manual testing throughout the remainder of the product build.
In early 2018, MoSo 3.0 had been live for two customers for several months, when the need arose to harden and stabilize the platform more. The company needed to be able to find more defects because of the limitations of manual testing, which prevented them from running certain test cases. Grant needed to automate approximately 800 to 1,000 smoke test cases.
“Among other things, our clients use MoSo 3.0 for billing their customers, so we take QA automation very seriously to ensure a high quality of service on our platform,” Grant noted.
Motionsoft considered several test automation options, including a home-grown solution, as well as Subject7. After seeing an impressive demo and checking customer references, Grant chose Subject7.
He identified several initial test automation requirements as a pilot, including the automation of 350 test cases within six months, and an ultimate goal of automating 26,000 test cases over time.
Thanks to Subject7, Motionsoft was able to make product quality a high priority in 2018.
“Once we found Subject7, we were able to radically change our QA process,” said Grant. “We focused 75% of our team’s time on stabilization in 2018, reducing defects by 66% by the end of the year. I’m frankly shocked at our overachievement in the automation area.”
After easily surpassing the original goal of automating 350 test cases, Motionsoft now executes 1,600 automated cases on a daily basis.
“We do a monthly release,” said Grant. “We have one cycle of full regression testing and one smoke test after the deployment.”
“With Subject7 we can run 1,200 cases every day, Monday through Friday,” Grant added. “If these cases had to be run manually, we could only run them once a month, with each scheduled release.”
No Coding Skills Necessary
“Subject7 is a very user-friendly test automation solution,” Grant added “There’s simply no coding needed to write and execute automation tests. If you know basic Xpath, you can do your own test scripts. It’s just very intuitive and certainly not difficult to learn.”
Ensuring Quality in an Agile World
“You really can’t adopt an agile methodology without test automation,” said Grant. “You simply don’t have time to keep running tests in the waterfall method. Anyone trying to do releases monthly will find that it’s just not possible to release a quality product without testing automation.”
“In this this day and age, there’s enough technology out there with integration into your development process, so that all of these tests can be triggered automatically by code merges,” Grant concluded.
80% Reduction in Regression Testing Time
Cobb Systems Group (CSG) is an award-winning, minority-owned business enterprise headquartered in Montgomery County, Maryland. CSG provides both technology for talent and talent for technology.
CSG’s patented talent engagement and analytics platform, ProForm™, delivers the features of traditional job boards, resume builders, and applicant tracking systems along with robust assessment, matching, and insight capabilities. ProForm™-enabled practices reduce time and cost to hire, enhance candidate pipeline quality and job fit, increase inclusion, and lower ongoing workforce costs.
Within ProForm™, privileges determine what menus, screens, and features are accessible to each user. ProForm™ has over 250 privileges. This level of granularity allows for thousands of custom configurations. These configurations must be regression-tested across multiple datasets and browsers, further expanding the number of test scenarios to be run with each code release or infrastructure change. CSG needed an automated test platform that would ensure test scenario coverage within release timeframes.
CSG evaluated several commercially installed, software as a service (SaaS), and open source solutions. However, all other solutions effectively required test automation resources to be software developers. In Subject7, CSG found a product that, out-of-the-box is:
Easy to implement (deployed as SaaS), with no installation necessary and seamless upgrades
Scalable with the power of Amazon cloud
Simple to use (wizard-driven test script creation), requiring no software development or scripting skills
Powerful in its ability to consistently replicate the execution of multiple test suites across multiple browsers with real-time status reporting
Robust in its ability to support data-driven testing wherein one test scenario is automatically re-run with varying inputs
Using Subject7, CSG was able to automate privilege regression testing and:
Decrease regression testing cycle time from 2-3 weeks to 3-4 days (this time can further be reduced by employing additional virtual machines)
Enhance team productivity
Free valuable test resources for other activities
Ensure test execution consistency and coverage
CSG continues to improve its continuous integration processes using Subject7 as a critical component in its test and validation efforts.