Understanding the Moshiko-0.1-8b model

Model overviewLink to this anchor

Model namesLink to this anchor

kyutai/moshiko-0.1-8b:bf16
kyutai/moshiko-0.1-8b:fp8

Compatible InstancesLink to this anchor

Model introductionLink to this anchor

Kyutai’s Moshi is a speech-text foundation model for real-time dialogue. Moshi is an experimental next-generation conversational model, designed to understand and respond fluidly and naturally to complex conversations, while providing unprecedented expressiveness and spontaneity. While current systems for spoken dialogue rely on a pipeline of separate components, Moshi is the first real-time full-duplex spoken large language model. Moshiko is the variant of Moshi with a male voice in English.

Why is it useful?Link to this anchor

Moshi offers seamless real-time dialogue capabilities, enabling users to engage in natural conversations with the model. It allows the modeling of arbitrary conversational dynamics, including overlapping speech, interruptions, interjections, and more. In particular, this model:

• Processes 24 kHz audio down to a 12.5 Hz representation with a bandwith of 1.1 kbps, performing better than existing non-streaming models.
• Achieves a theoretical latency of 160 ms, with a practical latency of 200 ms, making it suitable for real-time applications.

How to use itLink to this anchor

To perform inference tasks with your Moshi deployed at Scaleway, a WebSocket API is exposed for real-time dialogue and is accessible at the following endpoint:

wss://<Deployment UUID>.ifr.fr-par.scaleway.com/api/chat

Testing the WebSocket endpointLink to this anchor

To test the endpoint, use the following command:

curl -i --http1.1 \
-H "Authorization: Bearer <IAM API key>" \
-H "Connection: Upgrade" \
-H "Upgrade: websocket" \
-H "Sec-WebSocket-Key: SGVsbG8sIHdvcmxkIQ==" \
-H "Sec-WebSocket-Version: 13" \
--url "https://<Deployment UUID>.ifr.fr-par.scaleway.com/api/chat"

Make sure to replace <IAM API key> and <Deployment UUID> with your actual IAM API key and the Deployment UUID you are targeting.

The server should respond with a 101 Switching Protocols status code, indicating that the connection has been successfully upgraded to a WebSocket connection.

Interacting with the modelLink to this anchor

We provide code samples in various programming languages (Python, Rust, typescript) to interact with the model using the WebSocket API as well as a simple web interface. Those code samples can be found in our GitHub repository. This repository contains instructions on how to run the code samples and interact with the model.