AI VTuber

Stella

Built from scratch. Live2D avatar with spring physics & lip‑sync, a local LLM personality with mood dynamics, persistent memory, Twitch & Discord integration, a web dashboard, and an original EP on Spotify.

Explore the systems ↓
Stella — monochrome gothic anime VTuber with long white hair.
Live2D Rig Procedural Motion Mood-Aware
What Stella Does

Six systems,
one character.

01

Live2D Avatar

Critically damped spring physics drive idle motion, gestures, and head tracking. Lip‑sync maps ElevenLabs character timestamps to mouth shapes. Expressions and accessories toggle in real time from LLM tool calls.

PixiJSSpringsPhoneme SyncTool Calls
02

AI Personality

A local Ollama model generates Stella's responses. A four-axis mood system — happiness, energy, irritability, affection — shifts her tone dynamically. She muses out loud when chat goes quiet.

OllamaMood EngineAutonomySentiment
03

Tiered Memory

Conversations run through an LLM consolidation pipeline that extracts facts as ADD, MERGE, SUPERSEDE, or NOOP directives. Hybrid retrieval combines pgvector cosine similarity with keyword search.

pgvectorConsolidationHybrid SearchScoped
04

Platform Integration

Twitch and Discord bots run as background daemons. Viewer identities link across platforms with DM verification. Creator identity is verified by numeric ID to prevent impersonation.

TwitchDiscordLinked IDsPolls
05

Voice Pipeline

Browser Speech API for free local TTS, ElevenLabs for premium voice with character-level timestamps, and a priority message queue that deduplicates and applies backpressure under load.

ElevenLabsWeb SpeechPriority Queue
06

Web Dashboard

React frontend with a live WebGL avatar viewport, memory editor, mood meters, bot controls, and token-protected admin actions. Pop-out OBS overlay mode with transparent background.

ReactWebSocketOBS OverlayAuth
Tech Stack

Under the hood.

Frontend
React 18, Vite, PixiJS + Live2D Cubism SDK, Web Audio API, custom spring physics engine
Backend
Python, FastAPI, WebSocket streaming, httpx async client, priority message queue
AI / LLM
Ollama (Gemma 3 12B), semantic sentiment analysis, mood-driven prompt compilation
Voice
ElevenLabs with character timestamps, Browser SpeechSynthesis fallback, Whisper STT
Memory
PostgreSQL + pgvector, LLM consolidation pipeline, hybrid cosine + keyword retrieval, JSON fallback
Integrations
TwitchIO, discord.py (text + voice), cross-platform identity linking, hot-reload config
Infra
Token auth middleware, masked secrets, structured logging via Loguru, OBS browser source overlay
Spotify Release

Still Not
Your Doll

EP · 4 tracks · Stella-chan

Stella's first release gives the project a presence beyond the technical demo.

Listen on Spotify →
Built by BunnyBoyTamaki

Everything here
is from scratch.

The memory, the voice, the bots, the dashboard, the music. One person, one project, every layer.

Back to top ↑
Stella Ready