Metrics Discovery Concept Testing
@ Sumo Logic
This is an exploratory research project I led at Sumo Logic. The high level objective of the study is to understand users’ mental model when they conduct certain tasks in our product, thus to inform the information hierarchy of design direction. Some information is removed or modified due to confidentiality.
Disclaimer: The visuals you see on this page are coming from my final UXR report. It is for showcasing my research reporting ability. The content here is for YouTube content demonstration purpose - some details might be tweaked.
Background and Goals
The product team needs evidence from research to determine key focuses of product development of the quarterly road map. Some key goals we have:
Generative Research Goals
-
Understand our target personas’ workflow when browsing and searching for metrics in Sumo Logic
-
Understand the layers of validation & feedback our persona needs in order to know they’ve completed their necessary tasks (e.g. count of 5XX errors).
Design Research Goals
-
Understand whether users need the specific query to be visible at all times.
-
Evaluate the effectiveness and discoverability of user interactions within key-value tables, the left navigation bar, the search bar, and filter functions.
-
Evaluating proposed histograms and time series chart design for visualizing metrics.
Method
Crucial Insights (Demo)
When creating a chart using Sumo Logic product, users typically need 2 levels of mental validation: visualization and textual query. Our concept failed to assist users in accomplishing basic tasks because it didn’t provide the necessary system feedback users need.
The primary level validation is the visualization (chart).Before starting querying, users usually has a general assumption of how many lines they’d see and how “squiggly” the lines would be. They’ll later make the judgement of whether the chart looks correct or not based on this assumption.
The secondary level is texts that indicate a trail of actions, like breadcrumbs. As the line chart starts coming together, the user will also look back and forth between the chart and query to gain confidence of completion of the charting experience. If the chart doesn’t look correct, since the query offers users a history of their action, they can easily tweak the query and make adjustments.
However, our design back then lacked the textual validation that help document the history of their actions. It’s hard for users to remember what they have done and thus make the feature not usable.
For additional findings and learnings, please contact aona.ux@gmail.com.
Research Impact
Strategic Impact
-
Product strategy pivot backed by research insights.
-
This track of work has left a strong legacy, and a comprehensive playbook for the org to leverage.
Stakeholder collaboration Impact
-
Designed new strategies and company-wide processes in XFN stakeholder collaboration.
-
UXR had became a strategic partner of product team.
-
UXR championed collaboration between research and other functions that are not traditionally impacted by research (Engineering and Data Science).
Product Impact
-
Through collaboration, designers were able to quickly relay feedback from research and incorporate it to the prototype - Agile!
-
Design and visual components are currently being reused in a new initiative this year.
My Learnings
-
Starting strategic projects with a pragmatic approach.
-
Don’t start fire! You need a strategy to deliver “bad news” - and it takes practice.
-
Individual 1:1s with Leads before the team-wide research readouts help getting buy-in.
-
The more you engage stakeholders in every stage of the research process, the more they are willing to act upon your findings.
Figure 1. Foundational insight demo
Figure 2. Foundational insight demo
Figure 3. Foundational insight demo
Figure 4. Design insight demo