ElevenLabs Adds Memory, Routines and Model Fine-Tuning to ElevenAgents
The Development
ElevenLabs used its New York Summit to expand from voice models into an enterprise interaction stack. ElevenAgents, already deployed by Yahoo, Westgate and KPN, gains Memory, which builds structured customer records across conversations, and Routines, which run multi-step work before, during and after calls, including CRM writes and outbound callbacks. ElevenLabs will help businesses fine-tune certain open-source language models and its own voice models on live deployment data, optimised for metrics such as CSAT. An Interaction Engine previews turn-taking and per-step LLM routing. ElevenAI, an employee-facing meeting and dictation app built on Scribe v3, opens in free early access. The ElevenLabs Partner Program covers 550+ partners across 44 countries, plus a McKinsey and QuantumBlack alliance.
Our Take
The strategic move is Memory plus fine-tuning. Staniszewski framed it as every company getting its own reinforcement-learning stack, and that is the point: each conversation improves a model competitors cannot access. Memory also captures what CRMs never held, such as constraints, occasions and language preference. That is first-party data generated at the point of service, a direct hedge against signal loss elsewhere. Routines pull agents into the territory of lifecycle marketing, covering callbacks, reminders and waitlist offers. The McKinsey alliance and 550-partner program signal that ElevenLabs is selling transformation programs, not API calls. Treat this as a customer-experience data strategy decision, not a contact-centre tooling decision.
What Changed
Voice and chat agents can now retain conversational context across sessions, execute follow-up workflows autonomously, and be tuned against a company's own outcome metrics. Customer conversations become a continuously trained, company-specific system rather than a stateless script running on a generic model.
Marketing Impact
CRM and lifecycle teams gain a conversational channel that writes structured preference data back into customer records. Customer experience and support leaders can tie model tuning directly to CSAT. Marketing operations inherits new governance work: consent, retention and memory-record policy for voice interactions, owned jointly with legal and data teams.
Competitive Implication
Brands with high-volume inbound and outbound voice traffic gain a compounding data advantage, since every call trains a model rivals cannot reach. Legacy IVR vendors and scripted-bot providers become vulnerable because they cannot learn per customer. Generic agent platforms without fine-tuning loops face pressure from vendors owning both voice quality and optimisation.
Strategic Outlook
Expect Memory and fine-tuning to roll out through the coming months into Q1 2027, with Google Cloud and Genesys-style ecosystems copying the pattern. By mid-2027, voice agent differentiation shifts from voice quality to proprietary interaction data and tuning discipline. Regulators will turn to persistent conversational memory and consent next.
The Exploit
Action Item
Heads of CX and CRM at brands with 100,000+ annual customer calls should scope a Q1 2027 ElevenAgents pilot on one use case, with CSAT as the tuning metric and a written Memory consent policy agreed with legal before launch.