A demonstration of the half-heart experiment (no longer running)


Office Hours Duolingo

Duolingo co-founder Severin Hacker explains the service's use of experimental feature testing.

An experimental feature is a feature that is only made available for a subset of users in order to test its effects on their use of the service. Such tests are often referred to as A/B tests as they usually involve testing two versions of the service ("A" and "B"), one with the feature and one without it, and comparing user retention and completion in each group.

Motivation[edit | edit source]

One of the ways Duolingo improves itself is by testing possible new features on a subset of its users before rolling them out to everyone.[1] This is part of "Duolingo’s data-driven approach to education" which relies on analyzing activity data to improve language learning.[2] Using controlled experiments can help Duolingo decide whether a feature is actually an improvement or not before it commits to it.

Examples of experimental features[edit | edit source]

Feature description Experiment status Adopted
A Words section showing the user's learned vocabulary. Ended Yes
A new user interface for Immersion. Ended Discontinued feature
A "Start next lesson" button on the main page.[3] Ended [4][5] No
Half-heart: This feature allowed users to attempt correct their mistakes in exchange for losing a half-heart. Ended [6] No
Share credit: When editors makes a change to translation they had the opportunity to share credit, thereby sharing any upvotes with the original author of the sentence. Ended Yes; discontinued feature
Flashcards in words section: This involves displaying a word and checking if you get it right. This replaces the "Practice weakest words" button, whose functionality was the same as the Strengthen skills button on the home page.[7] Ended Yes; discontinued feature
A major redesign of the website. Ended Yes
Word description: This includes a descriptions of a word, its word strength, the part of speech, the possible translation, and example sentences, gender, as well as pronounciation, and conjugation tables. Ended Yes
Translate orally: Read a sentence and translate it orally Ended Yes
Lingots for translating[8] Ended No
Weakest Words shown within lesson Ended No
Duolingo coach for website Ended Yes
Multiple TTS engines (male and female)[9][10] Ended Yes
No lingots for continuing streak Ended No
Fluency meter: Track fluency throughout a language Ended Yes; discontinued feature
Conversation Exercises[11][12] Ended Yes?
Immersion removal Ended Yes
New Year's Resolution Unknown Unknown

References[edit | edit source]

Community content is available under CC-BY-SA unless otherwise noted.