Start Your Free Trial

Software is not quite ready to eat the world: State of dev/test infrastructure survey results

Lets take a closer look at what the state of dev/test infrastructure survey has revealed.

Software Is Not Quite Ready To Eat The World - Dev/Test Infrastructure Infographic
Survey shows enterprise dev/test infrastructure is holding back the pace of software driven innovation

As software development becomes more important than ever before, enterprises are forced to look for ways to shorten and streamline development processes. With virtualization empowering enterprises to treat their infrastructure very differently, they have made great progress towards becoming more agile over the last few years. In order to understand the current state of dev/test infrastructure across different enterprises we conducted a survey of cloud users, enterprise developers, IT administrators and operations professionals.

In August 2013 Ravello Systems collected the responses of 278 of these professionals. The results provide important insights to what is needed in the dev/test environment today.

Survey respondents:

Audience: Devops, development & test engineers, IT Admins, Automation Engineers, Virtualization Admins, Architects

State of dev/test environments today: Two burning issues emerge

Two thirds of respondents said that they urgently need to fix the fact that their infrastructure is holding them back from being as agile as their business demands. In order to address this issue 83% of respondents are adopting devops principles. However, the problem is that dev/test environments are not representative of productions environments, say 57% of respondents when asked how they perceived their dev/test setups.

Delving further into this problem, we found that there are two distinct but connected issues here:

1. It is difficult to provision complete application stacks due to lack of automation

As many as 75% of architects and IT admins noted that it’s too complex and time consuming to create dev/test environments and that they are not representative of production. When asked how long it takes them to get an isolated replica of their production environment for dev/test, 79% of respondents said it will take them a day or more to get an isolated replica of production and a third of the respondents said it will take them more than a week or that it can’t be done. Only 11% of enterprises that deploy dev/test environments on-premise report that their dev/test environments are replicas of the production environments.It is too complex a d time consuming to create development and test environments

2. Even if there was a magic wand to do #1, there is a lack of capacity in the internal data center

Irrespective of #1, capacity constraints are still preventing enterprises from being truly agile. In fact 80% of respondents say they face shortages of dev/test infrastructure sometimes, often or all the time. 92% of respondents need to buy more hardware for new dev/test projects, and for 70% it will take more than a week to do so (for half of them it will take more than a month!).

Lack of capacity for development and test infrastructure

Dealing with the problem: 3 workarounds

Our analysis indicates that enterprises have been dealing with these issues in three different ways:

1. Do nothing: developers face wait time, organizations deal with inefficiencies, and catastrophic issues are discovered in production. As mentioned, the majority of developers and QA engineers have to wait for more than week to get the required hardware for new dev/test projects, and over half of them wait for more than a month.

2. Build a massive lab: enterprises can always try to go the “huge capex route”. However, since its economically infeasible, it usually doesn’t happen. Only 7% of respondents say they rarely or never face dev/test infrastructure shortages, and still, among them over 70% say their dev/test environments are different than their production environments.

Development and testing environments are different from production

3. Try to use the cloud: more and more enterprises are realizing the cloud holds some key to a dev/test holy grail: agility, immediacy of resource allocation and cost-effectiveness in operations. In fact, 73% of respondents say their groups are running some dev/test workloads in a public IaaS cloud. This is clearly a step in the right direction, however, the result is unfortunately still a dev/test environment that is different than the production environment, since now developers and QA engineers are working with completely different virtual machines. Among the cloud-using enterprises, 78% report that their dev/test environments are different than their production environments – which eventually defeats the whole purpose

Some run their development and testing in the public cloud but still not replica of production

Even with these options…

The frustration from the current situation is clear: 28% of respondents say that over 50% of production problems could have been found and fixed with the right test environment. An additional 52% say that between 25% and 50% of production problems would have been fixed.

Production problems can be identified and fixed in development and test / staging time

Conclusion: Dev/Test Acceleration Needed

The conclusion is that  fast provisioning, dev/test environments that are representative of production, and enough capacity at hand are all necessary for the truly agile dev/test that enterprises aspire for.

Agile development and testing: Capacity on Demand, Instant Provisioning, Replicas of Production

What is required, is the ability for enterprises to use the infinite capacity of the public cloud for development & test – without making changes to their application. While there are a lot of tools for provisioning at the application layer, for true agility the automation needs to extend all the way down to the infrastructure (compute, network & storage).  Conceptually this “automatically automates” complex application provisioning end-to-end and enables continuous, representative and parallel testing throughout the application development lifecycle.


Thanks to @scott_lowe, @angeloluciani, @gurusimran @torontovmug and Arthur Schmunk.

About Ravello Systems

Ravello is the industry’s leading nested virtualization and software-defined networking SaaS. It enables enterprises to create cloud-based development, test, UAT, integration and staging environments by automatically cloning their VMware-based applications in AWS. Ravello is built by the same team that developed the KVM hypervisor in Linux.

Check our product demo video

Software is not quite ready to eat the world: State of dev/test infrastructure survey results