Atlas: Degree Progress UXR Case Study
Aug 22, 2023
Company: Center for Academic Innovation
Project Duration: July 11th to August 21th 2023
Background
Atlas is a software tool developed by the Center for Academic Innovation. It offers comprehensive information, data visualizations, and networked views of courses, instructors, and majors at the University of Michigan. Students commonly use it to learn about different courses and create their schedules.
Problem Overview
The product team for Atlas reached out to UXR about conducting an evaluative study for a new feature being developed. This feature, Degree Progress, offers an overview of a student's academic progress by listing the courses they have completed toward their degree, as well as indicating remaining coursework required for their enrolled academic program. The product team was primarily interested in whether the current iteration of Degree Progress meets students’ academic planning needs and is easy to use.
Collaboration
I worked as the sole primary UX researcher for this project from end to end, but I actively engaged with the rest of the UXR team for collaborative brainstorming and advice during our bi-weekly standups. I also worked closely with the Atlas product team which consisted of product managers, designers, and devs.
Overall Research Goal
Evaluate the usability and usefulness of the Degree Progress experience for students’ academic planning
Guiding Research Questions
What are students’ overall impressions of the Degree Progress feature?
How easy or difficult is it for students to find and access Degree Progress within the Atlas platform?
How do students perceive the accuracy of Degree Progress’ reflection of academic progress and degree requirements?
How clear and understandable are the labels, icons, and text used within Degree Progress?
What are the pain points or difficulties that students experience while using Degree Progress, if any?
How does the Degree Progress feature perform for students with different majors, academic years, and backgrounds, and are there any groups that may require additional support or customization to benefit from the tool fully?
How confident do students feel in using Degree Progress to plan their academic progress and select courses?
Methodology
Usability testing with think-aloud protocol
Remote
Moderated,
Qualitative focus
Recruitment
Degree Progress has slight changes in layout and information depending on certain factors (dual-majors, graduate vs undergraduate, degree program, etc.) Therefore, it was important to make an effort to include these types of students in testing. In order to use the beta version of the tool, students have to be from the University of Michigan and currently enrolled. Students from the education and nursing program are not eligible due to technical incompatibilities between their audit software and Degree Progress. Doctoral-level students are also not eligible.
Inclusion Criteria:
Current students enrolled at the University of Michigan
Exclusion Criteria:
Students enrolled in education or nursing programs
Students enrolled in a Ph.D. or an equivalent program
Usability Test
Each session lasted around 45-60 minutes. Participants completed a series of 8 tasks and were asked to voice their thoughts aloud. Questions were asked both during and after tasks. A total of 10 sessions were conducted for this study. Initially, 5 sessions were scheduled and conducted. However, I made the decision to extend it to 10 for a few reasons:
I was still observing new insights emerge during the 5th session
It became clear from initial testing that undergraduates and graduate students might have significantly different impressions and experiences when using Degree Progress. Given graduate students were underrepresented in the first 5 sessions, I wanted to spend more time with that user group.
Expanding to 10 would still be within the allocated budget
Key Findings
Upon completion of all 10 sessions, I moved on to qualitative tagging and insight analysis. Some of the key findings included:
The design and information layout of Atlas Degree Progress was viewed as a huge upgrade over Wolverine Access
Students enrolled in multiple programs or majors dealt with issues regarding how the tool tracked their academic progress
Students expected the progress bar to use light blue to represent courses 'in progress' and dark blue for those 'completed.' This was the opposite of how the progress bars werecurrently implemented.
The existing process of adding a required course from Atlas Degree Progress to their Atlas Backpack was needlessly complicated and confused users
Graduate students found the tool to feel tailored to undergraduates, and many features were either not applicable to them or did not operate in a way that suited their needs
For undergraduate students, the academic requirements typically involve multiple classes, which makes the progress bars useful
On the other hand, graduate students have lots of specific courses they need to take. Resulting in the progress bars being less meaningful and contributing to a cluttered interface.
Recommendations and Solutions
The product team had one sprint between the completion of the study and the ideal release window for the tool. Therefore, discussions between the product team and myself led to a few decisions:
The product team's sprint included finding a way to enhance the accuracy of degree requirements for dual majors. Alternatively, if the improvements could not be completed in time, a warning for dual majors to check with their departments/advisors for accuracy would be added.
Addition of a button that allows students to add a required course to their backpack directly from this page
Rollout to graduate students was delayed until a reworked version could be tailored to their academic needs
Changes to progress bars and other features were to be made immediately based on feedback
Results
Degree progress was released to eligible undergraduates in Fall 2023 in time for the rush of schedule changes and academic planning meetings. Graduate students had their version made available to them during Winter 2024
Degree Progress has been the most used feature of Atlas and has resulted in more consistent use of Atlas than before (previously most students only used Atlas during course registration season).
Student feedback gathered via surveys over the last year reported high satisfaction with the tool. Many have highlighted its effectiveness in breaking down information into an easily understandable format and ease of use
Future Work
✔ Usability Testing before Winter 2024. Focusing on the changes for grad students. (Now Completed)
Unexpected insight into users' mental models of their scheduling process and how they keep track of their academic progress emerged during these series of usability sessions. After the sessions, I followed up with some of the participants, and they were willing to share documents/spreadsheets of how they currently track their academic progress. The creation of these documents appears to mostly happen during academic advising, and they reported using multiple tools, including Atlas.
If academic advisors are already using Atlas to help students, then Degree Progress will likely be a great addition for them too. Therefore, a future study involving academic advisors would be great for uncovering how they are utilizing Atlas and what their unmet needs are. I am recommending a contextual inquiry that would allow us to see how academic advisors interact with students within a real academic advising session and to gain a deeper insight into the academic planning process and Atlas's role within it.