Yifang Li
Mixed-method User Experience Researcher
my.roku.com
01. Overview
Company & Team | Roku Pay Team
My Role | Lead UX Researcher
Cross-functional Partners | Roku Pay Product Managers, UX Designers, UX Writer, Data Analyst
Timeframe | Oct 2022 - Feb 2023
Skills | Tree Testing, Interview, Usability Testing, Qualitative Analysis
Tool | Optimal Workshop, Dscout
Problems of my.roku.com
my.roku.com was designed to serve as a hub for users to handle various tasks, such as managing Roku subscriptions, setting up parental control, changing password, and changing payment methods. However, the structure and design of my.roku have led to significant confusion among users. This web account experience is under-utilized and does not meet minimal standards of usability.
Research Approach
This is a three-step research.
-
The first step involves conducting tree testing to discover issues in the existing information architecture (IA). Subsequently, the designer formulated an improved IA. (This study was initiated by the previous Roku Pay researcher.)
-
In the second step, another round of tree testing is performed to validate the updated IA and labels. Based on this refined IA, the designer proceeded to create the new design. (I led this study.)
-
The third and final step involves user interviews and usability tests to evaluate the effectiveness of the new design. (I led this study.)
My Role
I drove user research for the project, from tree testing to find just the right information architecture, to understanding user goals for the page, to validating user flows from my.roku to give the team confidence they were on the right track.
Impact of This Project
The redesign of my.roku has been launched to 100% of Roku customers in November 2023. My.roku is a critical customer landing page, receiving 35.9 million sessions in Q3 alone –more than half of all of the sessions on roku.com. Before this change, the page was a collection of poorly organized links that had been tacked on over time. It is now a much clearer starting point for customers to manage and enrich their relationship with Roku. Early data suggests the page is driving 30% more subscriptions and 3.8% fewer customer service contacts.
02. Tree Testing on the Updated IA
The goal of this step is to assess the new IA structure and measure improved success in common account page tasks.
I conducted an unmoderated tree testing using Optimal Workshop with 59 participants (28 Roku users, 31 non Roku users).
The results of the study are summarized in the tables below. I identified high risk failure tasks and medium risk failure tasks that needed to be fixed in the design.
03. User Interview and Usability Testing
Design To Test
Following the integration of insights gathered from the IA tree testing, the designer enhanced the information architecture and developed various design variations for testing.
5 different layouts that highlight different info:
We would also like to know, of these 11 info modules, what modules users expect to see on my.roku.
Research Goals
-
Understand users' impressions/perceptions on different versions of the new my.roku landing page design
-
Investigate which layout strategy is the most well-received and preferred, e.g., highlighting top user tasks, offer visibility, feature discoverability, device management, plain and simple layout with links (short-term)
-
Determine which modules (e.g., subscription summary, device summary, shop by category) are useful to users and where they should be placed and prioritized on the landing page
-
Get feedback on any pain points in the design flow
Method
-
60-min remote usability testing through Dscout
-
Partial counter balancing for the 4 long-term design variations and the short-term design. Each participant saw 2 long-term designs and the short-term design in randomized order.
-
Participants N = 12
-
Roku users who have subscriptions through Roku & Roku users who do NOT have subscriptions with Roku.
-
Key Findings
-
Users typically come to the account page with a specific task in mind, such as managing their account or viewing their active subscriptions. They expect to see relevant information presented at the top of the page for easy access.
-
Users want the account page to be tailored to their specific needs and preferences, with clear indications of the available actions they can take.
-
Option A, which highlights users' most important tasks upfront, is the most popular of the five design options tested.
-
My subscriptions, My account, My devices, and Subscription offers are most popular modules. While Users have various preferences for the order of My Subscriptions, My Account, and My Devices, they all agree that Subscription Offers should be placed below them. However, users have a negative perception of Campaigns, Ads, and Did you know modules.
-
The information that users prioritize may change over time. For example, "Maximize your Roku" may be helpful at the top of the page during onboarding, but should be de-prioritized once users are familiar with their Roku.
-
Option E (short-term) has a higher task completion rate than the long-term options as it surfaces all actions/links.
04. Recap on My Research Impact
My research significantly influenced the redesign of my.roku, enhancing its IA, identifying users' informational need on this page, and evaluating the design with users. The redesigned my.roku was launched to all Roku customers in November 2023. This page is crucial, receiving 35.9 million sessions in Q3, over half of all roku.com sessions. It has evolved from a cluttered collection of links into a more user-friendly platform for customers to manage their relationship with Roku. Early data shows a 30% increase in subscriptions and a 3.8% reduction in customer service contacts, highlighting the positive impact of my research on my.roku's usability and customer engagement.
Thanks for reading!
Due to the constraints of NDA, I'm unable to share further details on my website. I would be delighted to have a more in-depth conversation through yvonneli426@gmail.com.