Coding Challenge 188: Voice Chatbot

Browser-based speech processing with Whisper

Implements OpenAI's Whisper model via Transformers.js to convert speech to text locally using WebGPU acceleration, ensuring no audio data leaves the computer.

Open-source text-to-speech with Kokoro

Integrates the Kokoro TTS model to generate natural speech from text responses, loading the model directly from HuggingFace into the browser.

Zero-cloud audio privacy

All audio processing happens client-side, with microphone input never transmitting to external servers despite models downloading from the cloud.

Native Web Audio API workflow

Uses Navigator.mediaDevices and MediaRecorder to capture audio chunks into blobs for processing, avoiding the p5.sound library to demonstrate underlying web audio mechanics.

16kHz waveform preparation

Decodes audio data through AudioContext at 16,000Hz sample rate and extracts single-channel waveform data to match Whisper's specific input requirements.

Async/await pipeline setup

Leverages p5.js 2.0 features to asynchronously import Transformers.js and initialize machine learning pipelines within the sketch setup function.

Alternatives to large language models

Demonstrates that chatbot 'brains' need not be LLMs, suggesting pattern-matching systems like ELIZA, RiveScript, or context-free grammars for simpler responses.

Demystifying AI through creative coding

Advocates learning AI through hands-on building with open-source models on consumer hardware, using creative play to understand and critique emerging technologies.

Push-to-talk interaction design

Implements a mouse-press interface to control when the bot listens, managing audio state between recording start and stop events to capture discrete utterances.