• Introduction• Solution Overview• Design Process• Research & Synthesis• Ideation• Design & Iteration• Reflection


Interactive Language Learning Tool for second language learners

Capstone project @ CMU
Houghton Mifflin Harcourt
8 Months | Jan. 2018 - Aug. 2019
Team Member
Shivang Gupta, Hanyu Tang, Wenjie Li

My Contribution

  • Led research process to define problem space
  • Created journey map and personas to consolidate research findings
  • Created wireframes
  • Conducted user testings and refined interfaces according to user testing results



Help Spanish learners from 6 to 12 grades improve their language proficiency in terms of the abilities in listening and speaking. 

Problem Space

Solution Overview

To tackle with the problems above, we designed Otter — a Spanish learning tool that creates an authentic language environment for Spanish learners.

It is featured with:
• Authentic language environment through interactive video
• 3-layer immediate and adaptive assistance
Personalized feedback for each skill practiced

Interactive Video

•  Háblame is a video platform that provides a list of Spanish videos from the beginning to advanced level.

•  There will be a character in the video talking to the user and asking him questions in Spanish, which the user can answer via voice input. As long as the voice is recognized, it will lead him to the next clip of the video, in which the character will keep talking to him and asking him questions.

3-Layer Assistance

•  While trying to answer the question in Spanish, the user can get 3-layer assistance according to their performance which can be indicated through automatic speech recognition.

Personalized Feedback

•  Users can see the score they get for each skill they practiced in the video, which gives them a better guidance on where they should put more effort.

Design Process

Research & Synthesis

With the broad design challenge, we came up with the driving questions as follow in order to guide the research process and narrow down the problem space:
• Who are the users? What are their goals, needs and challenges?
• How is their current learning experience and what are the major breakdowns?
• What are the best practices to build language proficiency in terms of listening and speaking?

With the driving questions in mind, we applied a variety of research methods including secondary and field research which helped us have a better understanding of the problem scope and how might we innovate to solve the problems.

After a series of research, we consolidated the research data through affinity diagramming and created different deliverables to convey the research findings.

Who are the users?

We created four student personas to help us consolidate our users, their goals, needs and challenges. The personas are divided by their motivation to learn and level of mastery because we found such a correlation according to the survey we sent to the students.

How is their current learning experience?

With all the research data in hand, we were able to draw a student experience map, demonstrating the learning experiences of the four student personas for different learning activities, which are shown on the x axis. The ones that are closer to the bottom are more traditional activities, the further ones are multimedia activities including video and games.

Key Insight

For the two personas with higher motivation, all activities are favorable. However, the Wallflower and the Box-Checker both lean towards more innovative and interactive activities. Since we hoped to reach as many users as possible and make something that is generalizable to a wide range of students, we found the design opportunity in the interactive learning activities.

What are the best practices to build language proficiency?

To answer this question, we consolidated our research findings and categorized the insights we got into 4 categories: practice, adaptivity, feedback, and culture.

Key Insight

One of the  keys to building communication proficiency is practicing impromptu conversation, but students lack such opportunities because of a lack of an immersive and authentic language environment.

Successful adaptivity involves customizing the task rather than the educational content itself, which means that we can customize the difficulty of the task according to each student’s performance.

An ideal way for students to learn a second language is to explore it on their own, as long as they can get in-time and personalized feedbacks from the teacher. 

Language is not merely an oral endeavor. Deciphering contextual cues such as body language and physical environment is essential in communication.


With the research findings and insights, we held a brainstorming session to come up with design concepts. After categorizing similar concepts together, we narrowed down to 9 storyboards and made the ideation matrix to summarize the key values we may give to our users through the design concepts.


In order to validate the design concepts, we conducted speed datings with potential users and pitched them to our client, asked them to give two scores for each storyboard, after which we made the graph below to evaluate each concept in two dimensions: feasibility and value.

This diagram not only helped us to choose and refine the final design concept, but also helped our client to make the decision, considering they are more familiar with the market. 

Selected Storyboard

We presented the storyboards to one of the stakeholders to validate the concepts. After the speed dating, we concluded the stakeholder’s feedback and analyzed the features of our storyboards in 3 dimensions: value,  feasibility, compatibility.

Design & Iteration

User Flow

Based on the storyboard, we started our design process with the user flow chart.

Parallel Design & Wireframe

With the user flow, we made the wireframe. In order to explore diverse ideas and ensure we made the best design decisions, we also did parallel design for several pages.

MVP & User Testing

We did 3 rounds of user testing with prototypes of different fidelity applying different user testing methods according to the goals of each round of user testing.

Design Decisions

We were constantly making iterations according to user testing results. Below are the highlighted iterations we made along the process.

Initially, I did parallel designs for the homepage, one of which was the layout on the left side, allowing users to filter the level, topic, and length of the videos they want to watch.

However, I finally made the decision to apply layout on the right side with the sidebar on the left and the default video on the right because: 1. Although this is a video platform, we don’t provide so many videos as Youtube does, so we don’t need many filters; 2. This is also an online learning platform providing videos with a structured sequence, so we need to guide users to watch the videos following a structured sequence.

According to user testing, many participants couldn't hear the questions clearly at the first time. They need a replay function. However, due to technical contexts and constraints, we couldn’t add a replay button on the initial interface.

To solve this problem, we decided to apply a CUI design pattern to incorporate the replay function into the dialogue, which not only solved the problem, but also maintained the sense of immersion in an authentic environment and enhanced the user experience.


The 8-month capstone project gave us an opportunity to work directly with a client in the industry, solve their problem, and experienced the whole feature release process from kickoff meeting, to research, design, development and testing. I learned a lot from the process.