Voxa: What It Takes to Build a Multimodal AI Tutor (Not Just Another Chatbot)

Most "AI tutor" projects are a chat window with a system prompt bolted on. Voxa started from a different question: what does a student actually want from an AI assistant during a real study session? Usually it's not just answers — it's someone (or something) that can listen, read what you upload, and help you turn a messy study session into structured notes.

Three input modes, one assistant

Voxa is built around three tightly integrated capabilities rather than one chat box with extra buttons:

• Conversational AI & Tutor — a context-aware chat that remembers the flow of the session, not just the last message, so follow-up questions actually make sense.
• Voice Sessions — real-time voice interaction for when typing isn't the point; you want to talk through a problem hands-free.
• Document Intelligence — upload a PDF or document and Voxa parses and analyzes it, extracting text and generating insights instead of just dumping raw content back at you.

On top of those three, Voxa can auto-generate structured notes and presentation materials from a session — so the output of "talking to an AI for 20 minutes" is something you can actually revise from later, not a chat log you'll never reopen.

Why this is harder than it sounds

The difficulty isn't any single feature — voice APIs, PDF parsing, and LLM chat are all well-documented individually. The difficulty is state: a voice session and a document upload and a chat message all need to feed into the same context so the assistant doesn't forget what you uploaded five minutes ago just because you switched from typing to talking. That's a backend and context-management problem as much as it's an AI problem.

Stack

Three input modes, one context

Capability	Input mode	Output
Conversational tutor	Text chat	Context-aware answers
Voice sessions	Real-time audio	Hands-free interaction
Document intelligence	PDF / document upload	Extracted text and insights
Notes & presentations	Any of the above	Auto-generated study material

The hard problem isn't any single input mode — it's making chat, voice, and documents all feed into the same shared context instead of three separate memories.

Next.js on the frontend, FastAPI on the backend, with the usual TypeScript/Python split doing the heavy lifting on each side. Source is on GitHub if you want to see how the session state is structured.