The Challenge
Create a truly conversational AI experience that goes beyond text — a virtual human that can listen, think, speak, and animate lip movements in real-time.
AI
Digital Human Chatbot
AI-driven virtual human chatbot integrating GPT-3, Whisper speech-to-text, Eleven Labs voice synthesis, and Rhubarb lip-sync. Multi-stage pipeline from speech recognition to animated response.
Impact Multi-modal AI
0 integrated AI Models
0s E2E Latency
Case Study
My Approach
Architected a multi-stage pipeline: Whisper for speech-to-text, GPT-3 via LangChain for contextual responses, Eleven Labs for natural voice synthesis, and Rhubarb for lip-sync animation. Built as a monorepo with Express backend.
The Results
Delivered a fully functional multi-modal AI chatbot with sub-2-second end-to-end latency. The project demonstrated the feasibility of real-time digital humans using off-the-shelf AI APIs.
Tech Stack
LangChainOpenAIEleven LabsExpressNode.jsMonorepo