Swift Voice Agent starter app

This starter app template for LiveKit Agents provides a simple voice interface using the LiveKit Swift SDK. It supports voice, transcriptions, live video input, and virtual avatars.

This template is compatible with iOS, iPadOS, macOS, and visionOS and is free for you to use or modify as you see fit.

Getting started

Run the following command to clone this template using the LiveKit CLI. This will create a new Xcode project in the current directory.

lk app create --template agent-starter-swift

Then, build and run the app from Xcode by opening VoiceAgent.xcodeproj. You may need to adjust your app signing settings to run the app on your device.

The app is configured to connect to the LiveKit homepage agent by default, which you can also try at livekit.com. That agent takes voice and text input only, so video and screen sharing are hidden until you point the app at your own agent (see Connect to your agent).

Note

To set up without the LiveKit CLI, clone the repository via git.

Connect to your agent

To switch from the default agent to your own, you first need a LiveKit agent to speak with. For a no-code setup, use the Agent Builder. For more customization, try our starter agent for Python, Node.js, or create your own from scratch.

Second, you need a token server. For development, the easiest option is the sandbox token server: enable it from your project's Options on the Settings page in LiveKit Cloud and copy the sandboxId.

Then edit AgentToConnect.current in VoiceAgent/VoiceAgentApp.swift:

static let current: Self = .sandbox(id: "your-sandbox-id")

That single value picks the token source and determines whether video and screen share input are enabled. For any other setup, add a case with your own token source.

Feature overview

This starter app supports several features of the agents framework and is easily configurable to enable or disable them in code based on your needs as you adapt this template to your own use case.

Text, video, and voice input

This app supports text, video, and/or voice input according to the needs of your agent. To update the features enabled in the app, edit VoiceAgent/VoiceAgentApp.swift and modify the .environment() modifiers to enable or disable features:

.environment(\.voiceEnabled, true)                                 // Enable voice input
.environment(\.videoEnabled, AgentToConnect.current.videoEnabled)  // Video and screen share input
.environment(\.textEnabled, true)                                  // Enable text input

Voice and text are enabled by default; video follows AgentToConnect.current, since the default homepage agent does not accept it. To override a feature, replace its value with true or false.

Available input types:

• .voice: Allows the user to speak to the agent using their microphone. Requires microphone permissions.
• .text: Allows the user to type to the agent. See the docs for more details.
• .video: Allows the user to share their camera or screen to the agent. This requires a supported model like the Gemini Live API. See the docs for more details.

If you have trouble with screen sharing, refer to the docs for more setup instructions.

Session

The app is built on top of two main observable components from the LiveKit Swift SDK:

• Session object to connect to the LiveKit infrastructure, interact with the Agent and its local state, and send/receive text messages.
• LocalMedia object to manage the local media tracks (audio, video, screen sharing) and their lifecycle.

Preconnect audio buffer

This app enables preConnectAudio by default to capture and buffer audio before the room connection completes. This allows the connection to appear "instant" from the user's perspective and makes your app more responsive. To disable this feature, set preConnectAudio to false in SessionOptions when creating the Session.

Virtual avatar support

If your agent publishes a virtual avatar, this app will automatically render the avatar's camera feed in AgentView when available.

Token generation in production

In production, you'll need to develop a solution to generate tokens for your users that integrates with your authentication system. Replace the SandboxTokenSource in AgentToConnect.tokenSource with an EndpointTokenSource (as the homepage agent case does) or your own TokenSourceFixed or TokenSourceConfigurable implementation.

Running on Simulator

To use this template with video (or screen sharing) input, you need to run the app on a physical device. Testing on the Simulator will still support voice and text modes, as well as virtual avatars.

Submitting to the App Store

LiveKitWebRTC.xcframework, which is part of the LiveKit Swift SDK, does not contain dSYMs. Submitting the app to the App Store will result in the following warning:

The archive did not include a dSYM for the LiveKitWebRTC.framework with the UUIDs [...]. Ensure that the archive's dSYM folder includes a DWARF file for LiveKitWebRTC.framework with the expected UUIDs.

It will not prevent the app from being submitted to the App Store or passing the review process.

Contributing

This template is open source and we welcome contributions! Please open a PR or issue through GitHub, and don't forget to join us in the LiveKit Community Slack!