These days, even twenty-something billionaires and their hyper-growth startups need to move fast without breaking things.
When Facebook launched back in 2004, things were much simpler. The world had one browser, Internet Explorer, with over 90% share of global usage. The Internet of Things was only a thing if you read Scientific American. It had been a year after Android Inc. had been founded, a year before Google acquired them, and three years before anyone would buy an iPhone. At the time, even Thoughtworks was contemplating whether Agile would be a practice limited to star developers, and if you used “Agile” as a noun on Madison Ave, you could bet on never being invited to a client pitch…for at least a decade.
Those were the good old days. Since then, we’ve seen an explosion of devices and platforms resulting in a dramatic increase in complexity. Clients and users today believe that everything should work perfectly on every device at all times, and have zero tolerance for any friction or bug. A recent Think Google study found that 66% of users will take a negative action towards a brand after a single less-than-optimal mobile experience. We’ve also witnessed the mass adoption of the idea that everything should be built faster, resulting in unreasonably short timeframes for design, development and testing. The need for speed and quality has never more important in the minds of CDOs and CTOs.
On the topic of quality, a 2015 survey by PWC found that CTOs considered testing and quality assurance (QA) the #1 barrier preventing their teams from shifting towards faster delivery, given the time required to do effective QA and the challenge of integrating QA throughout the development process vs leaving it to the end. But it’s not only enterprise leaders who are concerned with QA. A 2016 survey of digital agencies conducted by SoDA and ULTRA found that 60% of respondents considered QA to be a critical function – more so than software development (58%), system architecture (51%) or data science (15%).
The same SoDA survey also unearthed widespread challenges with QA experienced by agency leaders and producers alike:
* 43% were not satisfied with their current approach to QA
* 52% felt poor QA had undermined their organization’s reputation and/or client relationships
* 69% felt poor QA had caused unnecessary stress for their PMs and producers
Never mind moving fast without breaking things, if you’re sprinting this Holiday to launch a campaign on time with zero show-stopper defects, it may feel more like you’re trying to move fast, on a moving track, in a time warp, while hoping not to destroy your reputation or wreck your team! While constantly changing requirements and ever-shrinking timelines may be outside your control, poor QA is not.
And when it comes to QA, you can set your team up for success by planning ahead and following these four simple steps:
* Scope for QA at project onset. Our experience at ULTRA has been that QA hours for typical digital agency projects run at 20% of development hours. That said, it’s important to define the optimal coverage matrix upfront for all your projects, including types of testing needed (functional, design, etc.), areas to focus (features, pages, etc.) and browser / device needs based on each client’s preferences. That ratio goes up when the dev team is overworked and quality suffers, which tends to happen whenever there are massive spikes in work.
* Define your expectations for mobile coverage early on. New and updated devices continue to fragment the market making mobile coverage nearly impossible to stay on top of. In fact, a recent Perfecto Mobile report indicates for optimal coverage, you should consider 32 devices and OS combinations. This is the new reality, one that’s very different from how things were just a few years ago. At ULTRA, we recommend defining the optimal coverage matrix starting with your client’s analytics and adding in current usage data. That will surface any gaps you have in your testing requirements and device library while you still have time to address those gaps.
* Embed expert QA as early in the design and development process as possible. Upfront planning, optimal test strategies and the right suite of test cases will all end up saving you significant time and money. A study by the USC Center for Software Engineering found that it can cost as much as 100 times more to fix a bug in production vs. during the design/requirements phase.
* Staff up your internal QA team and/or bring in outside experts to handle the spike in workload, and don’t rely on producers, PMs or developers. Surprisingly, the 2016 SoDA survey found that 44% of agencies still rely on PMs and developers to do QA. Surprising, since most agency leaders understand the stress QA can create for their teams and the risk of failure introduced by not having dedicated QA expertise on digital projects.
If you’re in need of planning or testing execution for your next project, contact ULTRA Testing today.
Art Shectman is a serial entrepreneur and technologist, and leads technology and service delivery at ULTRA Testing. Having spent his career building new technologies and services, Art has extensive software engineering and systems integration experience with clients ranging from Fortune 100 companies to venture-backed start-ups. He is the founder of numerous startups, including Elephant Ventures, a full service digital agency focused on cutting edge web, social, mobile and back-end software development. Art has a BSc in Mechanical Engineering from M.I.T. with a concentration in Machine Learning.