RESPONSIBILITIES

Context

Instrumentation is the process in which a user of AppDynamics configures the product in order to derive deeper value. Out of the box, AppDynamics is able to map out your application automatically, however instrumentation is necessary to make sure you are calibrating alert and error detection to your environment.

Problem

Instrumentation can be a convoluted process, even for technical users. With pop-up dialogs within pop-up dialogs and screens full of blank form fields, the initial experience is extremely overwhelming. Because of this, AppDynamics was spending large amounts of capital to employ savvy technical account managers and sales engineers to hand-hold customers through the process of getting the most value out of their system. As the company looked to scale, we needed to make the instrumentation process more intuitive for a wide array of customers, lessening our dependence on specialized high-touch services.

Challenges

The biggest, and most obvious challenge, was the sheer size of the decision tree. As an example, here is a sketch file containing the possible screens for configuring a single Java transaction.

It might have been more efficient to actually code these interactions...

Because of this, the simplification process would be a painstaking game of prioritization. What options are most pertinent? What options are one-offs that can be obscured?

Another challenge was striking a suitable balance for the tone of the wizard. What level of explanation would be too basic for the audience? What level would not be basic enough? Is there a way to accommodate a range of experiences? These were questions without easy answers.

Process

Before building the proper decision tree for the wizard, I needed to gain a better understanding of the common instrumentation paths that occurred in the field. To do this I sat down with our CTO and some of the PMs to map out common experiences and frustrations that users had, and then brainstormed possible solutions.

When instrumenting from scratch, users were often flying blind. This is a sketch concept of a "live" view to help users gain context of their environment.

After mapping out the common paths, I then went and did my own research on what interactions occurred along those paths, taking note of ones that were confusing or convoluted. Below are several screenshots of what the instrumentation process looked like.

Design

Because this was such a complex project, I decided to go with an "ideal state" design for the first pass. I wanted to produce a visionary concept that wouldn't necessarily be possible in the near term, but would set the tone for the direction we wanted to head.

A ruthlessly decluttered and simplified user flow. Though some components were missing, it helped build the tone for future designs.

After a first rough prototype, I tapped several members of our product team to do a quick user test. It was quickly noted that we missed an important consideration: current practitioners and power users. It's usually not a good idea to leave all your current users in disarray by implementing an entirely new design. Below are a few screens from the refinement iteration that retained some of the original configuration options from our current product.

The refinement iteration included elements of the current user flow to ease users into the new design.

A few of the original user flow concepts that had to stay were the initial page users started at, and the popup dialog that housed all the configuration screens. We also included a toggle button to flip between the newer "interactive" mode and the "advanced" mode where the old screens were hidden.

I also introduced a step-by-step guiding panel on the right would allow users to gauge where they were in the process as they went along their instrumentation journey. This allows users to orient themselves and understand what to expect if they haven't gone through the flow before.

In addition to these refinements, we realized from our testing that liveness was the most important factor that was missing. Thus, we added a screen for immediate feedback dubbed "live mode". The heart of the "live mode" concept was to allow users to preview the configuration options they were setting in real time. This would help users demystify if the options they were setting actually worked or not before putting it into production.

Taking the "blind" out of "flying blind".

It was important to give the main workspace on the left a different treatment to denote a "live" scenario. This was accomplished with the darker header and footer sandwiching the workspace and the iconic red dot to indicate "liveness".

Conclusion & Next Steps

Ambitious projects can be daunting, but shouldn't be avoided. They are great openings to kick-off a greater discussion around the usability philosophies of the product. In addition, they present opportunities for more visionary concepts that can help shift the course of the product design.

Continuing this project, I would have developed a broader test plan with customers and a higher fidelity prototype to validate some of the high level assumptions we had. Upon successful testing and validation, I propose a gradual roll out of features to a small subset of users, constantly comparing metrics to ensure a smooth transition.