Skip to content

Commit a062cdd

Browse files
h4x3rotabclaude
andcommitted
feat: add confidential AI examples
Add confidential-ai/ directory with three examples: - inference: Private LLM inference with vLLM on Confidential GPU - training: Confidential fine-tuning using Unsloth - agents: Secure AI agent with TEE-derived wallet keys Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 43da63b commit a062cdd

File tree

16 files changed

+982
-1
lines changed

16 files changed

+982
-1
lines changed

README.md

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99

1010
**Example applications for [dstack](https://github.com/Dstack-TEE/dstack) - Deploy containerized apps to TEEs with end-to-end security in minutes**
1111

12-
[Getting Started](#getting-started)[Use Cases](#use-cases)[Core Patterns](#core-patterns)[Infrastructure](#infrastructure)[Dev Tools](#dev-scaffolding)[Starter Packs](#starter-packs)[Other Use Cases](#other-use-cases)
12+
[Getting Started](#getting-started)[Confidential AI](#confidential-ai)[Use Cases](#use-cases)[Core Patterns](#core-patterns)[Infrastructure](#infrastructure)[Dev Tools](#dev-scaffolding)[Starter Packs](#starter-packs)[Other Use Cases](#other-use-cases)
1313

1414
</div>
1515

@@ -61,6 +61,22 @@ See [Phala Cloud](https://cloud.phala.network) for production TEE deployment.
6161

6262
---
6363

64+
## Confidential AI
65+
66+
Run AI workloads where prompts, model weights, and inference stay encrypted in hardware.
67+
68+
| Example | Description |
69+
|---------|-------------|
70+
| [confidential-ai/inference](./confidential-ai/inference) | Private LLM inference with vLLM on Confidential GPU |
71+
| [confidential-ai/training](./confidential-ai/training) | Confidential fine-tuning on sensitive data using Unsloth |
72+
| [confidential-ai/agents](./confidential-ai/agents) | Secure AI agent with TEE-derived wallet keys using LangChain and Confidential AI models |
73+
74+
GPU deployments require: `--instance-type h200.small --region US-EAST-1 --image dstack-nvidia-dev-0.5.4.1`
75+
76+
See [Confidential AI Guide](https://github.com/Dstack-TEE/dstack/blob/master/docs/confidential-ai.md) for concepts and security model.
77+
78+
---
79+
6480
## Tutorials
6581

6682
Step-by-step guides covering core dstack concepts.

confidential-ai/README.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Confidential AI Examples
2+
3+
Run AI workloads with hardware-enforced privacy. Your prompts, model weights, and computations stay encrypted in memory.
4+
5+
| Example | Description | Status |
6+
|---------|-------------|--------|
7+
| [inference](./inference) | Private LLM with response signing | Ready to deploy |
8+
| [training](./training) | Fine-tuning on sensitive data | Requires local build |
9+
| [agents](./agents) | AI agent with TEE-derived keys | Requires local build |
10+
11+
Start with inference—it deploys in one command and shows the full attestation flow.
12+
13+
```bash
14+
cd inference
15+
phala auth login
16+
phala deploy -n my-llm -c docker-compose.yaml \
17+
--instance-type h200.small \
18+
-e TOKEN=your-secret-token
19+
```
20+
21+
First deployment takes 10-15 minutes (large images + model loading). Check progress with `phala cvms serial-logs <app_id> --tail 100`.
22+
23+
See the [Confidential AI Guide](https://github.com/Dstack-TEE/dstack/blob/master/docs/confidential-ai.md) for how the security model works.

confidential-ai/agents/Dockerfile

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
FROM python:3.11-slim
2+
3+
WORKDIR /app
4+
5+
COPY requirements.txt .
6+
RUN pip install --no-cache-dir -r requirements.txt
7+
8+
COPY agent.py .
9+
10+
EXPOSE 8080
11+
12+
CMD ["python", "agent.py"]

confidential-ai/agents/README.md

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
# Secure AI Agent
2+
3+
Run AI agents with TEE-derived wallet keys. The agent calls a confidential LLM (redpill.ai), so prompts never leave encrypted memory.
4+
5+
## Quick Start
6+
7+
```bash
8+
phala auth login
9+
phala deploy -n my-agent -c docker-compose.yaml \
10+
-e LLM_API_KEY=your-redpill-key
11+
```
12+
13+
Your API key is encrypted client-side and only decrypted inside the TEE.
14+
15+
Test it:
16+
17+
```bash
18+
# Get agent info and wallet address
19+
curl https://<endpoint>/
20+
21+
# Chat with the agent
22+
curl -X POST https://<endpoint>/chat \
23+
-H "Content-Type: application/json" \
24+
-d '{"message": "What is your wallet address?"}'
25+
26+
# Sign a message
27+
curl -X POST https://<endpoint>/sign \
28+
-H "Content-Type: application/json" \
29+
-d '{"message": "Hello from TEE"}'
30+
```
31+
32+
## How It Works
33+
34+
```mermaid
35+
graph TB
36+
User -->|TLS| Agent
37+
subgraph TEE1[Agent CVM]
38+
Agent[Agent Code]
39+
Agent --> Wallet[TEE-derived wallet]
40+
end
41+
Agent -->|TLS| LLM
42+
subgraph TEE2[LLM CVM]
43+
LLM[redpill.ai]
44+
end
45+
```
46+
47+
The agent derives an Ethereum wallet from TEE keys:
48+
49+
```python
50+
from dstack_sdk import DstackClient
51+
from dstack_sdk.ethereum import to_account
52+
53+
client = DstackClient()
54+
eth_key = client.get_key("agent/wallet", "mainnet")
55+
account = to_account(eth_key)
56+
# Same path = same key, even across restarts
57+
```
58+
59+
Both the agent and the LLM run in separate TEEs. User queries stay encrypted from browser to agent to LLM and back.
60+
61+
## API
62+
63+
| Endpoint | Method | Description |
64+
|----------|--------|-------------|
65+
| `/` | GET | Agent info, wallet address, TCB info |
66+
| `/attestation` | GET | TEE attestation quote |
67+
| `/chat` | POST | Chat with the agent |
68+
| `/sign` | POST | Sign a message with agent's wallet |
69+
70+
## Using a Different LLM
71+
72+
The agent uses redpill.ai by default for end-to-end confidentiality. To use a different OpenAI-compatible endpoint:
73+
74+
```bash
75+
phala deploy -n my-agent -c docker-compose.yaml \
76+
-e LLM_BASE_URL=https://api.openai.com/v1 \
77+
-e LLM_API_KEY=sk-xxxxx
78+
```
79+
80+
Note: Using a non-confidential LLM means prompts leave the encrypted environment.
81+
82+
## Cleanup
83+
84+
```bash
85+
phala cvms delete my-agent --force
86+
```
87+
88+
## Further Reading
89+
90+
- [Confidential AI Guide](https://github.com/Dstack-TEE/dstack/blob/master/docs/confidential-ai.md)
91+
- [dstack Python SDK](https://github.com/Dstack-TEE/dstack/tree/master/sdk/python)

confidential-ai/agents/agent.py

Lines changed: 198 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,198 @@
1+
#!/usr/bin/env python3
2+
"""
3+
Secure AI Agent with TEE Key Derivation
4+
5+
This agent demonstrates:
6+
- TEE-derived Ethereum wallet (deterministic, persistent)
7+
- Protected API credentials (encrypted at deploy)
8+
- Confidential LLM calls via redpill.ai
9+
- Attestation proof for execution verification
10+
"""
11+
12+
import os
13+
14+
from dstack_sdk import DstackClient
15+
from dstack_sdk.ethereum import to_account
16+
from eth_account.messages import encode_defunct
17+
from flask import Flask, jsonify, request
18+
from langchain_classic.agents import AgentExecutor, create_react_agent
19+
from langchain_classic.tools import Tool
20+
from langchain_openai import ChatOpenAI
21+
from langchain_core.prompts import PromptTemplate
22+
23+
app = Flask(__name__)
24+
25+
# Lazy initialization - only connect when needed
26+
_client = None
27+
_account = None
28+
29+
30+
def get_client():
31+
"""Get dstack client (lazy initialization)."""
32+
global _client
33+
if _client is None:
34+
_client = DstackClient()
35+
return _client
36+
37+
38+
def get_account():
39+
"""Get Ethereum account (lazy initialization)."""
40+
global _account
41+
if _account is None:
42+
client = get_client()
43+
eth_key = client.get_key("agent/wallet", "mainnet")
44+
_account = to_account(eth_key)
45+
print(f"Agent wallet address: {_account.address}")
46+
return _account
47+
48+
49+
def get_wallet_address(_: str = "") -> str:
50+
"""Get the agent's wallet address."""
51+
return f"Agent wallet: {get_account().address}"
52+
53+
54+
def get_attestation(nonce: str = "default") -> str:
55+
"""Get TEE attestation quote."""
56+
quote = get_client().get_quote(nonce.encode()[:64])
57+
return f"TEE Quote (first 100 chars): {quote.quote[:100]}..."
58+
59+
60+
def sign_message(message: str) -> str:
61+
"""Sign a message with the agent's wallet."""
62+
signable = encode_defunct(text=message)
63+
signed = get_account().sign_message(signable)
64+
return f"Signature: {signed.signature.hex()}"
65+
66+
67+
# Define agent tools
68+
tools = [
69+
Tool(
70+
name="GetWallet",
71+
func=get_wallet_address,
72+
description="Get the agent's Ethereum wallet address",
73+
),
74+
Tool(
75+
name="GetAttestation",
76+
func=get_attestation,
77+
description="Get TEE attestation quote to prove secure execution",
78+
),
79+
Tool(
80+
name="SignMessage",
81+
func=sign_message,
82+
description="Sign a message with the agent's wallet. Input: the message to sign",
83+
),
84+
]
85+
86+
# LangChain agent (lazy initialization)
87+
_agent_executor = None
88+
89+
90+
def get_agent_executor():
91+
"""Get LangChain agent executor (lazy initialization)."""
92+
global _agent_executor
93+
if _agent_executor is None:
94+
template = """You are a secure AI agent running in a Trusted Execution Environment (TEE).
95+
You have access to a deterministic Ethereum wallet derived from TEE keys.
96+
Your wallet address and signing capabilities are protected by hardware.
97+
98+
You have access to the following tools:
99+
{tools}
100+
101+
Use the following format:
102+
Question: the input question
103+
Thought: think about what to do
104+
Action: the action to take, should be one of [{tool_names}]
105+
Action Input: the input to the action
106+
Observation: the result of the action
107+
... (repeat Thought/Action/Action Input/Observation as needed)
108+
Thought: I now know the final answer
109+
Final Answer: the final answer
110+
111+
Question: {input}
112+
{agent_scratchpad}"""
113+
114+
prompt = PromptTemplate.from_template(template)
115+
116+
# Use redpill.ai for confidential LLM calls (OpenAI-compatible API)
117+
llm = ChatOpenAI(
118+
model=os.environ.get("LLM_MODEL", "openai/gpt-4o-mini"),
119+
base_url=os.environ.get("LLM_BASE_URL", "https://api.redpill.ai/v1"),
120+
api_key=os.environ.get("LLM_API_KEY", ""),
121+
temperature=0,
122+
)
123+
124+
agent = create_react_agent(llm, tools, prompt)
125+
_agent_executor = AgentExecutor(
126+
agent=agent, tools=tools, verbose=True, handle_parsing_errors=True
127+
)
128+
return _agent_executor
129+
130+
131+
@app.route("/")
132+
def index():
133+
"""Agent info endpoint."""
134+
try:
135+
info = get_client().info()
136+
return jsonify(
137+
{
138+
"status": "running",
139+
"wallet": get_account().address,
140+
"app_id": info.app_id,
141+
}
142+
)
143+
except Exception:
144+
return jsonify({"status": "running", "error": "Failed to retrieve agent info"})
145+
146+
147+
@app.route("/attestation")
148+
def attestation():
149+
"""Get TEE attestation."""
150+
nonce = request.args.get("nonce", "default")
151+
quote = get_client().get_quote(nonce.encode()[:64])
152+
return jsonify({"quote": quote.quote, "nonce": nonce})
153+
154+
155+
@app.route("/chat", methods=["POST"])
156+
def chat():
157+
"""Chat with the agent."""
158+
data = request.get_json()
159+
message = data.get("message", "")
160+
161+
if not message:
162+
return jsonify({"error": "No message provided"}), 400
163+
164+
try:
165+
result = get_agent_executor().invoke({"input": message})
166+
return jsonify(
167+
{
168+
"response": result["output"],
169+
"wallet": get_account().address,
170+
}
171+
)
172+
except Exception:
173+
return jsonify({"error": "Failed to process chat request"}), 500
174+
175+
176+
@app.route("/sign", methods=["POST"])
177+
def sign():
178+
"""Sign a message with the agent's wallet."""
179+
data = request.get_json()
180+
message = data.get("message", "")
181+
182+
if not message:
183+
return jsonify({"error": "No message provided"}), 400
184+
185+
signable = encode_defunct(text=message)
186+
signed = get_account().sign_message(signable)
187+
return jsonify(
188+
{
189+
"message": message,
190+
"signature": signed.signature.hex(),
191+
"signer": get_account().address,
192+
}
193+
)
194+
195+
196+
if __name__ == "__main__":
197+
print("Starting agent server...")
198+
app.run(host="0.0.0.0", port=8080)
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Secure AI Agent with TEE Key Derivation
2+
#
3+
# Runs an AI agent with:
4+
# - TEE-derived wallet keys (deterministic, never leave TEE)
5+
# - Encrypted API credentials
6+
# - Confidential LLM calls via redpill.ai
7+
#
8+
# Deploy: phala deploy -n my-agent -c docker-compose.yaml -e LLM_API_KEY=your-key
9+
10+
services:
11+
agent:
12+
image: h4x3rotab/tee-agent:v0.4
13+
volumes:
14+
- /var/run/dstack.sock:/var/run/dstack.sock
15+
environment:
16+
- LLM_BASE_URL=https://api.redpill.ai/v1
17+
- LLM_API_KEY # Encrypted at deploy time
18+
ports:
19+
- "8080:8080"
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
dstack-sdk>=0.1.0
2+
langchain-classic>=1.0.0
3+
langchain-openai>=0.0.5
4+
langchain-community>=0.4.0
5+
web3>=6.0.0
6+
flask>=3.0.0

0 commit comments

Comments
 (0)