Giving network administrators visibility into network applications performance

SD-WAN APPLICATION QUALITY OF EXPERIENCE - APPQOE
Overview
In 2018, Juniper Networks moved from its core hardware business to a Software as a Service (SAAS) model and introduced a new network cloud service (SD-WAN) that automatically chooses the best available network path (link) for application traffic.  Using SD-WAN, I helped create a management interface for network applications called Application Quality of Experience (AppQoE) that allows network admins to monitor and troubleshoot network application performance across tenants, sites, and links.
But first, some context - the business need

To stay competitive and relevant in the network business, the company needed to embrace new cloud network technologies

With an initiative started in response to the predictions that more enterprises will be moving to the cloud, the company focused on changing its core hardware business to a Software as a Service (SaaS) model.

Our approach? A Cloud environment of hardware and software that allows the creation and support of many network models to operate services like VPN, Security, and latest cloud software technology like SD-WAN.
BUSINESS STRATEGY
The company's business strategy shows the cloud environment building blocks  - platforms, services, and products - that support the SD-WAN network technologies.
SD-WAN ADVANTAGES
Traditional networks used to direct all the traffic to the Headquarters or Cloud Providers for security reasons. With SD-WAN, Enterprises or Branch Offices can connect directly to the internet since SD-WAN Controller provides security through policies.

Enter Software Defined Wide Area Networks (SD-WAN)

SD-WAN improved application performance by automatically choosing the best available network path - link - for application traffic.
The opportunity

Network administrators had no guarantee of optimal application performance due to a lack of visibility into the SD-WAN inspection process

Once the company created the cloud environment building blocks to use SD-WAN traffic inspection technology, SD-WAN started to measure the link parameters - throughput, Latency, Packet Loss, and Jitter - to use them as metrics to determine application performance.

However, as the back-end system collected SD-WAN metrics and automatically performed application traffic changes, network administrators did not have visibility to monitor that business-critical applications running in the network were getting what they needed to function optimally based on the changes made by the SD-WAN technology to application traffic.
Who are network administrators

We needed to support two types of network admins:
Enterprise and Services Providers

Network administrators current tasks

- Service provider admins help enterprises by recommending and implementing the network solutions they need.
- Enterprise admins handle the company's network infrastructure.

Working with internal domain experts and current Juniper customers, we mapped the current task flow for enterprise and service providers' network administrators to understand how they currently configure and manage their networks.
Validating Assumptions

Comprehend network admins' mental model of the jobs they need to get done with SD-WAN to accomplish their goals

Through marketing analysis, PM provided a list of requirements of what customers needed from the product. However, we first wanted to understand enterprise and service providers' goals and needs to know what they were trying to achieve and validate the areas of the SD-WAN product required to help users accomplish their tasks.
WHAT JOBS USERS WANT TO GET DONE
After interviewing network administrators who were thinking of using SD-WAN in the future, we identified critical tasks and listed a set of users' needs for the jobs to be successful.
ALIGNMENT OF NEW SD-WAN JOBS WITH CURRENT CSO PLATFORM
Based on the jobs found, I incorporated them into the existing product to visualize where these new jobs could live, assert which new workflows were necessary, and what areas of the product needed improvement to support them.
User Insights
On a daily basis, Network Administrators will monitor that their enterprise customers or networks are meeting service levels.
‍
In case of a problem, they need to be able to get to the affected sites, look at the connected links, and get information about the applications running traffic to those sites to solve the issue.
Working with our UX researcher, we talked to network administrators about their motivations on how the SD-WAN solutions will help them achieve their business goals.
Competitive Analysis

Focusing on broad impact and ease of execution - providing performance information at the SD-WAN service level.

Using network administrators' goals and needs, define with stakeholders what features we should concentrate our resources on to improve upon from other businesses
COMPETITIVE ANALYSIS SHEET
Discussing the Competitive Analysis sheet with stakeholders provided the basis to understand our competitors’ value propositions and the features our product needed (highlighted in yellow) to compete with similar products in the market.
DEEP DIVE FEATURE PRIORITIZATION SESSION
Working with stakeholders, we prioritized the features that would solve network administrators' goals and needs based on our feasibility and capabilities to develop those features.
Understand SD-WAN back-end architecture and specify elements missing

Get a clear idea of the information available that we could surface in the UI and which elements are missing from the initial designs

SD-WAN BACK-END ARCHITECTURE
I set meetings with stakeholders to understand the architectural components that support SD-WAN to have a clear idea of the information available from the back-end that we could surface in the interface.
INITIAL SYNTHESIS
After collecting information about our existing features in our CSO platform, what competitors were offering, and the current design proposals, I specified the design elements missing that we needed to provide the ideal experience for network admins to visualize application performance in their networks.
Specify the user flow

Define SD-WAN configuration, monitoring, and troubleshooting workflows based on user insights and stakeholders' discussions.

Using user insights and the outcomes from the competitive analysis, back-end architecture, and navigation models meetings, I worked with stakeholders to define the new SD-WAN workflows and how they fit inside the current Contrail Service Orchestrator - CSO - platform network configuration and management workflows.
Challenges - system performance

Significant processing time to identify which applications used most of the network

We wanted to provide network admins with the capability to identify which applications used most of their network, but that would require significant processing time from the system, compromising performance. Instead, we agreed to display application usage per site.
Our goal - what was our intent with the product?

Design for assurance - communicate the network's behavior and performance when network projections are not being met.

Provide users with tasks that communicate the network's behavior and performance, giving direction with relevant insights suitable to each network element and facilitating action when network projections are not being met.

FOR SERVICE PROVIDERS NETWORK ADMINS
Ensure that network service levels of their enterprise customers are being met, providing a clear picture of the site, links, and application performance.

FOR ENTERPRISE NETWORK ADMINS
Give guarantee of application health status across their network, communicating application traffic behavior during a specific time.
Initial designs - original hypothesis

Application performance for specific network parameters across all sites would help network admins to identify applications not meeting service level agreements  - SLAs

We believe that by providing visibility into application performance across all sites for particular parameters like throughput or latency during a specific time, network administrators would be able to know how many applications are not meeting Service Level Agreements - SLAs - and what are the sites that have application performance issues, allowing them to drill down and determine if the problem is at the application or site levels
Initial iterations tested with network administrators of the SD-WAN - AppQoE application pages focus on providing visibility into application performance across the network.
User Insights
Even though network administrators understood more or less the application chart view, instead of the applications being at the center of the workflow, service providers and enterprise network admins want first to observe if their enterprise(s') sites are having issues that may affect application performance.
Collecting additional user feedback and refining the design

Focus on the enterprise, then the sites, and finally on applications running on that site that are not meeting service levels.

We tested numerous times with users to refine the configuration, monitoring, and troubleshooting workflows for SD-WAN - AppQoE.

During the sessions, I kept updating the design proposal based on the issues or the suggested information common among participants. Then we tested with the following set of customers to verify that those updates solved the problems.
Based on user feedback, I updated the design flow, focusing first on monitoring the enterprise, then the sites, and finally on the applications.
User Insights
At the application level, “Knowing the specific time, the particular link, and the precise type of traffic for the application; this information will help me to get quickly to the events (system logs) that we need to see to fix any issue.” — Network Architect (information technology company)
The outcome

SD-WAN - Application Quality Of Experience (AppQoE)

AppQoE provides a comprehensive overview of applications running in the network, to determine that site app performance is within service standards and present information to take action when the network is unable to meet application requirements.

We are going to look at two journeys

Getting Started

SD-WAN - AppQoE configuration

After an enterprise has enabled SD-WAN, Traffic Type Profiles and SLA Profiles will be available with default options that Application Routing Policies can use. For first-time users, an onboarding panel will explain the main features of SD-WAN and how to get started.

Quick view of the enterprises / sites with low SLA performance

Determining if the site's low performance comes from the applications or the service parameters on the links

View site's applications performance across all network parameters - throughput, latency, packet loss, and jitter

Get application details for SLA performance, the number of sessions, and app behavior across the site links.

Indicators of success

Improving the customer experience

The company measured success in revenue, the number of new customers, and position on the market—the design team measured success by improving the customer experience.
‍
This project measured success by using the following metrics during usability studies:
  • 90% success rate of test task completion by review participants for each tested feature.
  • Participants should complete test tasks within a similar range of time. There should be no more than a 20% variance in completion rates.
  • Customers rated the ease of use of the tasks on a scale of 1 to 5, with 1 = very difficult to use and 5 = very easy to use.
Following the design reviews from the last usability study before implementation:
Participants completed 98% of the tasks, and there was a 12% variance between completion rates. The average ease-of-use rating was 3.8, a ‘Neutral’ ease-of-use rating.
To conclude

Adapting the design process to the business context

In companies like Juniper that are engineering-driven and design is not an essential part of the product development process, it is important to adjust and align the specific phases of the design method with the environment, processes, relationships, and priorities within the company for everyone to understand the value of design and accomplish results that translate user behaviors into products.

Pushing the stakeholders to be proactive

When defining what we needed to offer in the product for the first release, the team aimed to provide features according to the requirements gathered by our product team. However, when it came to market trends, the competition was way ahead. I found it necessary through a competitive analysis to show the team what we needed to improve in the product to be ahead of other businesses.

Finding out who the knowledge keepers are

Sometimes there are team members whose knowledge about a specific topic goes beyond their company title. These people’s opinions matter across stakeholders because they can advocate for the best user experience with you, which could be an advantage for your design proposal at every stage of the product development process.
LinkedinInstagramobejaran@gmail.com