I'm Matt Wood, and this is For Your Information. A live list of riffs and links for you and your agent, drawn from what I'm reading, noticing, questioning, concluding, and revising.
Dessa is a conversational AI model trained exclusively on speech data, enabling natural dialogue interactions based on spoken language rather than text.
Dust: Pretraining Transformers Without BackpropagationBoth challenge conventional transformer training paradigms — Dust avoids backpropagation while Dessa avoids text data, both exploring alternative training methodologies for language models
Gemini AI ModelsBoth are conversational AI models, but Dessa's exclusive speech training represents a fundamentally different modality approach compared to Gemini's text-centric training pipeline
WorldTUI: Training a 360M Model as a Terminal EmulatorBoth explore unconventional training data sources for language models — WorldTUI uses terminal emulator interactions while Dessa uses speech audio, both pushing beyond standard text corpora
Language Models that Play Chess and Explain Their MovesBoth models are trained on domain-specific interaction data (chess moves vs. spoken dialogue) to produce models capable of explaining or generating natural language in specialized contexts
Supports
How Long Contexts FailDessa's speech-based training implicitly supports concerns about how long contexts fail — spoken language is inherently more concise and turn-based, suggesting speech data may produce more robust short-context conversational models
Challenges
AI Model GroupthinkDessa's speech-only training directly challenges AI model groupthink around text-based training data, demonstrating that natural dialogue capabilities can emerge from an entirely different modality