Designing Adaptive AI Learning Conversations

2. Keyword Transparency, Users set the agenda, the product tracks progress

Users select learning keywords before the session starts, based on an AI-generated list for their topic. During the session, a progress bar tracks which keywords have been covered in real time.

• This does two things:
• it gives users ownership over what the session covers, and it makes the AI's evaluation criteria visible upfront.
• Users know exactly what they are being assessed on, and can see themselves making progress.

Most conversational AI follows a fixed script.
Question → answer → next question, regardless of what was actually said.
I worked with engineering to define prompt logic where the AI classifies every user response into one of three categories, then responds accordingly:

• Match, explanation is correct and covers the keyword
  → AI asks a deeper follow-up
• Wrong understanding, explanation has errors
  → AI surfaces the gap without saying "you're wrong," asks a clarifying question
• Unrelated, explanation drifts off topic
  → AI gently redirects back to the keyword

Each path required a distinct UI state. I mapped and designed all three before engineering built them, not after.

The key design principle:

designing for LLM conversation means designing for variability. The AI does not always respond predictably. Define the edge states first. Design the UI to hold in all of them.

After the core chat was working, internal review revealed a gap: there was no accumulation. No reason to come back.
I designed a session feedback dashboard showing four signals on a 1–10 scale, visualized as a chart:

1. Understanding, how accurately did the user explain the concept
2. Learning Achievement, how many keywords were successfully covered
3. Satisfaction, how well did the AI student respond throughout
4. Passion and Attitude, engagement and persistence through the session

Plus

per-keyword summary, mistake correction, and review materials. Feedback tone adapted to the persona, a warm elementary student gave encouraging feedback, an expert gave detailed critical analysis.

Three-phase product roadmap I defined:

1. Phase 1 (shipped): Per-session feedback, 4-factor score, chart visualization
2. Phase 2: Per-keyword scoring → diagnose weak areas → recommend next persona or topic → score cycle with next action
3. Phase 3: Gamification, real-time achievement tracking, hint cycle, competitive factors, per-keyword analytics (questions asked, answer quality average, correct/incorrect ratio)