DevOps–Collaborating to Improve Application Performance

By: Mike Cuppett


As iron sharpens iron, developers and database administrators (DBAs) can challenge each other while building synergies resulting in improved application performance. The concept of DevOps requires IT development and IT operational teams working together to build and improve application performance and usefulness, using a cohesive, repeatable, continuously refined approach.  Two concerns must remain prevalent, exceptional customer experience and efficient use of computing resources.  Let’s first discuss the “big picture” encompassing the IT supply chain, followed by several examples specific to developer and DBA collaboration.

Scores of methods exist for measuring application performance, ranging from sophisticated Application Performance Management/Monitoring (APM) tools to a hodge-podge of tools and/or “home-grown” scripts. However, the truest measure of application performance and usefulness is the application user’s perspective, also known as the End-User Experience (EUE).

How IT Silos Impact Application Performance

Many in IT have misconceived that if each technology silo maintains a mandated uptime and measurable performance requirement, the resulting customer experience follows positively.Unfortunately, even minor losses of performance or uptime degrade the customer’s experience. For example, if each technology in the IT supply chain from the application user to the back-end database returning to the application user delivers 97% correctly, the application user’s experience is poor (see Figure 1).

Cuppett Devops1

Figure 1.

Underperforming systems ultimately result in premature investments to expand computing power, drawing scarce resources away from potentially better investments. While virtualization and other technologies have leveraged resources by minimizing unused capacity, they still don’t address end-to-end, data-moving performance.  Data is a product customers want to consume; IT infrastructure and applications are the tools of the data supply chain.  Logistically, the data supply chain must be able to move data from point-to-point securely and quickly, at the lowest possible cost. Meeting both end-user application performance expectations and cost-minimized data delivery challenges the status quo of existing IT organizational design and continuous improvement methods.

As shown in Figure 2, bringing the application and database up to five 9’s improves the customer experience, but not as significantly had each silo been improved by a single percentage point (see Figure 3).

Cuppett Dev Ops2

Figure 2.

Cuppett Devops3

Figure 3.

Knowing that a significant improvement in one or two areas does not improve the customer experience more than advancing all capabilities within the IT supply chain, it’s reasonable to affirm the need for a holistic approach to improved customer experience and enhanced operations. Developers and database administrators (all IT staff) need to extend their reach to include the full spectrum of application knowledge, which means working with all of the IT teams that can contribute to improving application delivery.

For these full-spectrum teams to cooperate in eradicating performance problems leading to healthier operations, communications and coordination become key success factors. Teams must talk frequently to synchronize the efforts supporting the end goal:  superior application performance for the end-user and efficient operational business data flows. The ability to deliver enhancements brought about from multiple angles – development and operations (DevOps) – will improve performance more dramatically than improvements completed by isolated teams. See Figure 4.

Cuppett De Ops4

Figure 4.

DevOps efforts must keep the big picture views of customer experience and operations in mind, even while focused on improving one specific segment of the application or infrastructure. Meanwhile, developers and database administrators must share information and labor together to deliver collaborative improvements to application and database performance.  Here are two example problems encountered at a large client recently.

Case Study: Pre-production Testing Doesn’t Match Production User Needs

The first problem involved the last minute realization that all of the pre-production testing had been done from the perspective of internal customers; however, most of the users would actually be external, which involved additional authentication. Immediately, a 30-second delay was noticed which would translate into disaster from the business perspective.  Fortunately, having many teams represented led to finding and resolving the problem quickly.  The teams focused on the problem by considering the data supply chain holistically, instead of the ineffective and all too typical approach where each team focusing in isolation on just the technology within their scopes of responsibility.

Case Study: New Application Functionality Appears to Negatively Impact Performance

The second problem required some digging. New functionality had recently been added to an existing system, putting in place the tracking of up-selling opportunities for customers.  However, the 30 to 40 second average wait was not acceptable, especially when customer service agents are working calls with customers. Here, understanding the customer experience not only validated the impact form the user’s perspective, it also led to the root cause.

Cuppett Dev Ops Timelin?

Figure 5.

The timeline shows over fifteen seconds from the user’s click to when the user can proceed with the next step.  Further testing revealed that the previous process (before the up-sell enhancement) also took 15-20 seconds to load, making the combined process duration 30-40 seconds.  Also, notice within the timeline that each individual step executes reasonably fast, so if considered in isolation a single step may not get flagged as poor performing.  Whereas, when the steps are viewed from the customer experience perspective it becomes obvious the slow process won’t support business operations. Mitigation discussions are still underway for this problem; however, having this data has proven unchallengeable when defining the predicament. Now is when developers and DBAs can specifically look to reduce the customer experience by focusing on the steps within the timeline where the application code or database can be made more responsive and by restructuring the process to include parallel thread execution. These enhancements, integrated with changes offered by other teams, will improve the customer experience and operations.

The Need for Continuous Integration

One of IT’s missteps has been the division of work by technical disciplines. For example, 20 years ago security was never the focus of a single dedicated team; rather everyone was responsible for and practiced security based on established policies. Security was entrenched deep within the delivery methodologies of every IT segment.

Today’s silo-ed IT teams must evolve into work cells that are not singularly focused on high-performance network, server, telephony or database point solutions, but rather on delivering high-performance data supply chains.

Conclusion

DevOps is the remediation plan needed to realign IT organizations, improve delivery, project and service capabilities needed to drive business operations. Only by starting from the perspectives of the customer experience and operation effectiveness, working in integrated teams of mixed-capabilities and experiences, will teams begin evolving IT’s ability to meet the business’ and customer’s demands for information where and when it’s needed. Additionally, progressing from DevOps to BusDevOps should further align business and IT capabilities, deliver expected performance levels and produce cost-effective processes needed to empower business growth, reduce waste and expand competitive advantage.

Leave a Reply