Usability Case Study
HotChalk Ember Learning Management Tool
HotChalk Ember is a web and mobile application that allows students and teachers to interact outside of the classroom. HotChalk has won multiple awards for its education solutions, including a 2008 Codie Award for “Best Education Solution” and a 2007 Technology & Learning Award of Excellence for “Teacher Resources.”
To comply with the non-disclosure agreement, I have omitted and altered confidential information. The research and designs are a reinterpretation of the original work.
A Shifting Educational Environment
Nearly 50% of all higher ed students are "non-traditional," such as part-time and remote students.
As education shifts from the traditional classroom format, learning management tools become increasingly important.
Nontraditional graduate students in particular rely on learning management tools to stay organized, manage their time, and prioritize responsibilities.
HotChalk is designed specifically for small, private, nonprofit, on-ground universities that want to reach more students by developing online programs. The HotChalk app provides students an online tool to engage and stay connected with their online course.
"Don't Make Me Think"
Steve Krug's book, "Don't Make Me Think," outlines best practices so that users can accomplish tasks as quickly and easily as possible.
Initial feedback from users of HotChalk, however, indicated confusion and frustration with the existing app, and users felt the UI was out of date.
To resolve these issues, HotChalk Ember hired Miami University students studying Interactive Media Studies to conduct in-depth user testing over a 15-week period and design a solution.
Ultimately, the students using HotChalk need a tool that is easy to use and quickly answers the following:
1. What do I have to do, when is it due, how do I submit it, and where can I view feedback?
2. How can I contact professors or other students?
3. How am I doing in a particular course?
4. Where can I find resources and information about a course, assignment, or exam?
Don't make students think
(Ironic, isn't it?)
Visual Design Lead
PHASE 1: PROBLEM IDENTIFICATION
Evaluating the Original App
Jakob Nielsen's 10 Usability Heuristics for User Interface Design can be used to identify problems in UI design. We chose to evaluate the HotChalk app for each heuristic on a scale of 0-4, with 0 being no problem and 4 being catastrophic.
The following heuristics were rated as a 3 or 4:
Set a Benchmark
Testing the Original App
After the heuristics revealed areas for improvement, it was time to see what real students thought.
Our team conducted user testing of the existing HotChalk app by measuring the time it took to complete specific tasks. We recruited 5 undergraduate students at Miami University from a variety of disciplines to participate. The usability session was divided into 3 parts: pre-screen questions, 5 realistic & important tasks, and exit questions.
While there were mixed reviews, word choice and information architecture were the largest hurdles for users.
Information Architecture & Vocabulary
To organize the app, we recruited 37 students for card sorting. We asked them to sort an overwhelming 39 cards which could explain why only 59% completed the task. Simplifying the app and streamlining categories would be crucial to improve usability.
This exercise informed the information architecture of the application.
The majority of students organized the app by course first, with discussions, assignments, quizzes, and more nested within respective courses.
What Did We Learn Today?
The Importance of Language
Usability heuristics, user testing, and card sorting highlighted an important detail: the importance of language. There is not a standardized vocabulary for learning management apps, so we were challenged to develop a language that students understand universally.
Information Architecture is Key
The existing organization of the app was unintuitive and confusing. Students did not understand the difference between the "Home" and the "Dashboard," and they also preferred a course-first framework.
PHASE 2: UNDERSTAND & DEFINE USERS
Walk a Mile in Someone Else's Shoes
Understanding Core Users
We conducted in-depth interviews with 6 non-traditional graduate students, the app's primary users, to learn their needs and expectations for learning management programs, as well as to learn about their lifestyles.
Our interviews revealed the true needs of our target user: Between juggling careers, education, and personal life, they lead incredibly busy lives. They want to quickly find what they need in a well-organized, simple, and efficient app.
"I don't really care about notifications. As a part-time grad student, I want an app that I can quickly check to find due dates, resources, or whatever I need."
Know Your Audience
Defining Core Users
Utilizing our in-depth interviews, we created two personas to represent the typical HotChalk user.
Enrolled in 1-3 courses, own a smartphone, 4+ hours of coursework/day, use a combination of written & digital checklists, as well as calendar apps, interests include sports, traveling, home improvement, cooking, TV, and spending times with friends & family
A Day in the Life: Erin Anderson
Erin is an architect by day and Pratt student by night. When she comes home from work and school, she cooks dinner with her boyfriend Tim before wrapping up her day by working on studio projects.
Between work and school, Erin is incredibly busy. She manages most of her life with Google Calendar and Checklist in order to save time wherever possible, but she admits even calendar apps can barely keep her organized. She finds value in quickly gathering the most important information, whether she is reading the news, checking her schedule, or reviewing tasks for her job and school.
"One of the most frustrating things is when professors change due dates or assignments at the last minute. It doesn't make it any easier that I can't always get on my work computer to check updates! ! I think they forget how busy grad students are sometimes..."
A Day in the Life: Jay Washington
Jay is an audio engineer for a small tech startup, and he is working towards his Master of Science in Computer Engineering. He works with 4 other engineers and admits his days are typically hectic, but he hopes to be the Product Manager someday. In his free time, he enjoys cycling, going to concerts, and toying with his latest app.
Organization is key for Jay. He frequently checks up on coursework when he has a break during the day, and he appreciates being notified of new grades, messages from professors and fellow students, and assignment updates. However, his professors different sites for each course, and Jay would prefer everything be in one central location.
"I can't tell you how many times I've almost missed a deadline because I checked the wrong site for course information."
PHASE 3: DESIGN ITERATIONS
The Problem Child
Defining the Homepage
One of the main points of confusion while testing the app was the homepage. The original HotChalk app had a "Dashboard" page, a "Recent Updates" page, and a "Home" page. Users could not figure out which was actually the homepage which made it challenging to navigate the app. After multiple iterations, we created a sitemap to guide our low-fidelity wireframes.
The "Dashboard" became the new homepage, and it is organized by course. Per user feedback, the Calendar and Inbox remained on the Dashboard for easy accessibility.
Pen & Paper
Designing the UI
We identified the information architecture and vocabulary as two of the major hurdles for users, and we created personas to hone in on user behaviors and needs. Next, it was time to sketch and rapidly ideate to answer "how might we clearly define the homepage and organize the app?" and "how might we organize information into clear and understandable categories?"
You Say Tomato...
Choosing a Universal Vocabulary
Discussions | Assignments | Syllabus | Calendar | Gradebook | Roster | Resources | Tests & Quizzes | Modules | Announcements
PHASE 4: LOW-FIDELITY PROTOTYPE
Will It WOrk?
Testing the Prototype
We chose the most promising UI sketches and began designing low-fidelity wireframes with Axure to create a prototype for user testing.
We re-tested 5 main tasks with our low-fidelity prototype.
Check Assignment Due Date
Before - 0:45
After - 0:20
Check Assignment Grade
Before - 2:10
After - 0:30
Email a Classmate
Before - 0:45 (task abandoned)
After - 0:40
Find Office Hours
Before - 0:30
After - 0:20
Post in a Course Discussion
Before - 2:30 (task abandoned)
After - 0:20
PHASE 5: HIGH-FIDELITY PROTOTYPE
The Grand Finale
We designed and tested our low-fidelity, functional prototypes, and then applied user feedback to create high-fidelity mockups according to the HotChalk style guide.
The design was improved with consistent and familiar vocabulary, simple navigation, and iconography and other visual cues to help users quickly find what they need.
Our redesign reduced the time and effort to complete primary tasks, making the app more intuitive, usable, and efficient.