Study Guides
Role
Lead product designer
Platforms
Web, iOS, Android
Timeline
2023-2024
For the biz
Grow the pie
Quizlet has long dominated the market in Flashcards, but now we want to go after a new area: creating and reviewing notes. It's a big opportunity area, especially with AI.
For the user
Save time
Help the user save time and study more effectively by using AI to create tailored, high-quality study material for them.
And so what?
The impact
In 1.5 years, the product reached 1.3M MAU, distinct from the existing flashcard audience, and 25M notes uploaded. It grew the Quizlet pie by offering a new and distinct value for users.
First, the research: What do students do with their notes?
Prioritize key concepts: What they need to know for the exam, cut the other fluff.
Organize source material: Combine multiple notes, slides, or worksheets into one source of truth.
Rework information to make it stick: Interacting with the notes, reworking it to help understand and remember it.
What can AI generate from a student's notes? What should it?
The Machine Learning engineers and I mapped possible artifacts and according to feasibility, quality of output, and value to the user. We were told to launch with a lot of outputs at first so we can learn quickly. So, we prioritized these five:
Summary
Outline
Flashcards
Sample essay questions
Practice test
Designing the flow
Key questions I considered: How do we encourage users to upload materials? Does the user have any say in what is created for them? What input methods are most useful on your phone vs. computer? After the generation process, how do we promote discovery through all the generated study material?
The initial feedback blew us away
But after the initial shock…
We dug deeper, we found that all five artifacts together created a confusing experience. The summary and outline were too similar, Practice Test quality was buggy, sample essay questions didn't make sense for STEM subjects. This user quote was personally very memorable for me:
“This experience feels like chaos mode. It’s too confusing when it gives me all these options at once.”
% engagement by artifact
Looking at the quantitative data, it's no surprise Flashcards comes first in engagement. We are Quizlet after all, and it's the most recognizable artifact on the page. The outline surprised expectations. The Summary was disappointing, especially given that it was at the top of the screen.
Flashcards
Outline
Practice test
Summary
Sample essay questions
68%
36%
29%
25%
17%
Though the experience felt magical, I compared the experience to a forced buffet with no main course.
The team was at a crossroads on what to do next. I felt strongly we should move away from the "buffet experience" and have our team invest in the Outline as the main course. I was bullish on the outline because:
It's the second-most engaged with artifact, and flashcards are already covered on Quizlet.
One of the top feature requests from user research was editing the outline, showing its potential.
The outline supports the original research on what users want to do with their note: prioritize, organize, transform
I needed to get stakeholders on board with this vision.
It was hard.
Here are some things I did to eventually get folks on board. Different cross-functional stakeholders cared about different things.
Product cared about the amount of content created to build up a content bank. I did explorations on what it would look like to "upsell" the user to create other the other artifacts.
ML cared about OpenAI costs. By generating a buffet every time, we were running through the content generation budget quickly. I tapped the PM to run analyses on whether the benefit of creating all the artifacts every time is really worth the cost.
Everyone cared about metrics. I suggested running an A/B test to remove Summary and expand the Outline. This resulted in ~5% increase in overall time spent and a +3% retention increase. Users were also +39% more likely to engage with the outline. This gave the team more conviction that creating more focus on the Outline is a good thing.
Outline as main course
When we finally got the team and stakeholders aligned with this vision, we built features on top of the Outline based what we heard from users. Images from their original notes helped visual learners, highlighting lets them interact with their notes, and a full markdown editor lets them have full control.
The impact
Overall AI creation (what the PMs cared about!):
+15%
in overall creation with AI, after splitting into distinct artifacts.
+47%
in users uploading a note once they reached the Upload screen, after giving users the choice.
On PMF of the new study guides product:
+81%
Growth in monthly engaged users despite decoupling it from Flashcards.
+36%
Increase in 1-30D retention
71%
Would be “very disappointed” if they could no longer use the product
