Skip to content

Commit 3f6cc91

Browse files
committed
FEATURE: Add SQL plugin support
Signed-off-by: Pascal Zimmermann <pascal.zimmermann@theiotstudio.com>
1 parent 5662acd commit 3f6cc91

46 files changed

Lines changed: 13233 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
"prometheus",
2626
"pyroscope",
2727
"scatterchart",
28+
"sql",
2829
"statchart",
2930
"staticlistvariable",
3031
"statushistorychart",

sql/CONTRIBUTING.md

Lines changed: 287 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,287 @@
1+
# Contributing to SQL Plugin
2+
Thank you for your interest in contributing to the Perses SQL Plugin!
3+
## Development Setup
4+
1. **Prerequisites**
5+
- Node.js >= 22
6+
- npm >= 10
7+
- Docker (for test databases)
8+
2. **Install Dependencies**
9+
```bash
10+
cd perses/plugins/sql
11+
npm install
12+
```
13+
3. **Start Development Server**
14+
```bash
15+
# In perses main repository, enable dev mode
16+
# Update config.yaml:
17+
plugin:
18+
enable_dev: true
19+
# Start backend
20+
./bin/perses --config ./dev/config.yaml --log.level=debug
21+
# In a new terminal, start SQL plugin
22+
cd plugins/sql
23+
percli plugin start .
24+
```
25+
## Architecture
26+
The SQL plugin consists of several components:
27+
### TypeScript/React (Frontend)
28+
- **Datasource Plugin** (`src/datasources/sql-datasource/`): Configuration UI for SQL datasources
29+
- `SQLDatasource.tsx` - Plugin implementation
30+
- `SQLDatasourceEditor.tsx` - UI editor component
31+
- `sql-datasource-types.ts` - TypeScript types
32+
- **Query Plugin** (`src/queries/sql-time-series-query/`): SQL Time Series Query editor
33+
- `SQLTimeSeriesQuery.tsx` - Query plugin implementation
34+
- `SQLTimeSeriesQueryEditor.tsx` - Query editor UI
35+
- `sql-time-series-query-types.ts` - TypeScript types
36+
- **Explore Plugin** (`src/explore/`): SQL Explorer for interactive querying
37+
- `SQLExplorer.tsx` - Prometheus-style explorer with Table & Graph tabs
38+
- Uses `MultiQueryEditor` and `DataQueriesProvider` from Perses plugin system
39+
- Supports TimeSeriesChart and TimeSeriesTable visualization
40+
- **Model** (`src/model/`): Utility functions
41+
- `replace-sql-builtin-variables.ts` - SQL macro replacement ($__timeFilter, etc.)
42+
- `sql-client.ts` - HTTP client for query execution
43+
### Backend (Perses Core)
44+
- **SQL Proxy** (`internal/api/impl/proxy/proxy.go` in Perses main repository)
45+
- Handles all SQL queries via backend proxy
46+
- Connects to databases (PostgreSQL, MySQL, MariaDB)
47+
- Executes queries and returns JSON
48+
- Handles authentication and TLS
49+
### Schemas
50+
- **CUE Schemas** (`schemas/`)
51+
- `datasource/sql.cue` - SQL datasource configuration validation
52+
- `sql-time-series-query/sql-time-series-query.cue` - Query configuration validation
53+
### Test Data
54+
- **Test Databases** (`test-data/`)
55+
- PostgreSQL sample data (`test-data/postgres/`)
56+
- MySQL sample data (`test-data/mysql/`)
57+
- MariaDB sample data (`test-data/mariadb/`)
58+
## Adding New Features
59+
### Adding a New SQL Driver
60+
1. **Update Backend** (`internal/api/impl/proxy/proxy.go` in Perses core)
61+
- Add driver support in `sqlOpen()` function
62+
- Add driver-specific connection string handling
63+
2. **Update Frontend Types** (`src/datasources/sql-datasource/sql-datasource-types.ts`)
64+
```typescript
65+
export type SQLDriver = 'postgres' | 'mysql' | 'mariadb' | 'your-driver';
66+
```
67+
3. **Update UI** (`src/datasources/sql-datasource/SQLDatasourceEditor.tsx`)
68+
- Add driver option to the Select component
69+
- Add driver-specific configuration fields if needed
70+
4. **Update CUE Schema** (`schemas/datasource/sql.cue`)
71+
- Add driver to allowed values
72+
5. **Add Test Data** (`test-data/your-driver/`)
73+
- Create `init.sql` and `sample-data.sql`
74+
- Update `docker-compose.test.yaml`
75+
### Adding a New Query Type
76+
1. **Create Query Directory**
77+
```bash
78+
mkdir -p src/queries/your-query-type
79+
```
80+
2. **Implement Query Plugin**
81+
- Create `YourQuery.tsx` following `SQLTimeSeriesQuery.tsx` pattern
82+
- Implement `getTimeSeriesData()` or appropriate data fetching method
83+
- Create `YourQueryEditor.tsx` for UI
84+
3. **Add Types**
85+
- Create `your-query-types.ts` with spec interfaces
86+
4. **Register Plugin**
87+
- Export from `src/queries/index.ts`
88+
- Add to `package.json` perses.plugins array
89+
- Add to `rsbuild.config.ts` Module Federation exposes
90+
5. **Add CUE Schema**
91+
- Create `schemas/your-query-type/your-query-type.cue`
92+
### Adding SQL Macros
93+
SQL macros are processed in `src/model/replace-sql-builtin-variables.ts`.
94+
**Existing Macros:**
95+
- `$__timeFilter(column)` - Generates WHERE clause for time range
96+
- `$__timeFrom` - Start of time range
97+
- `$__timeTo` - End of time range
98+
- `$__interval` - Interval in seconds
99+
- `$__interval_ms` - Interval in milliseconds
100+
**To Add New Macro:**
101+
1. **Add macro processing logic** in `replace-sql-builtin-variables.ts`
102+
```typescript
103+
export function replaceSQLBuiltinVariables(...) {
104+
// ...existing code...
105+
// Add your macro replacement
106+
query = query.replace(/\$__yourMacro/g, 'replacement');
107+
return query;
108+
}
109+
```
110+
2. **Add tests** in `replace-sql-builtin-variables.test.ts`
111+
```typescript
112+
it('should replace $__yourMacro', () => {
113+
const result = replaceSQLBuiltinVariables('SELECT * WHERE $__yourMacro', ...);
114+
expect(result).toBe('SELECT * WHERE replacement');
115+
});
116+
```
117+
3. **Document in README.md**
118+
## Testing
119+
### Run Unit Tests
120+
```bash
121+
# All tests
122+
npm test
123+
# Watch mode
124+
npm test -- --watch
125+
# With coverage
126+
npm test -- --coverage
127+
```
128+
### Type Checking
129+
```bash
130+
npm run type-check
131+
```
132+
### Start Test Databases
133+
```bash
134+
# Start all databases (PostgreSQL, MySQL, MariaDB)
135+
make db-up
136+
# Verify test data
137+
make verify-all
138+
# Stop databases
139+
make db-down
140+
# Clean all (including volumes)
141+
make db-clean
142+
```
143+
### Test Individual Database
144+
```bash
145+
# PostgreSQL
146+
make verify-postgres
147+
# MySQL
148+
make verify-mysql
149+
# MariaDB
150+
make verify-mariadb
151+
```
152+
## Building
153+
```bash
154+
# Build the plugin
155+
npm run build
156+
# Build for Module Federation
157+
npm run build-mf
158+
# Type checking
159+
npm run type-check
160+
```
161+
## Development Workflow
162+
### 1. Make Changes
163+
Edit files in `src/` directory
164+
### 2. Test Locally
165+
```bash
166+
# Start backend (in perses main repo)
167+
./bin/perses --config ./dev/config.yaml --log.level=debug
168+
# Start plugin (in plugins/sql)
169+
percli plugin start .
170+
# Plugin will rebuild automatically on file changes
171+
```
172+
### 3. Test Changes
173+
- Open browser to Perses UI
174+
- Create/edit SQL datasource
175+
- Create dashboard with SQL query
176+
- Test Explore mode
177+
### 4. Run Tests
178+
```bash
179+
npm test
180+
npm run type-check
181+
```
182+
## Code Style
183+
- **TypeScript** for all new code
184+
- **Follow existing patterns** in the codebase
185+
- **JSDoc comments** for public APIs
186+
- **Functional components** with React Hooks
187+
- **Named exports** over default exports
188+
### Component Structure
189+
```typescript
190+
// Copyright header
191+
// ...
192+
import { ... } from '...';
193+
interface YourComponentProps {
194+
// Props definition
195+
}
196+
export function YourComponent({ prop1, prop2 }: YourComponentProps): ReactElement {
197+
// Component implementation
198+
}
199+
```
200+
### File Naming
201+
- Components: `PascalCase.tsx`
202+
- Types: `kebab-case-types.ts`
203+
- Utilities: `kebab-case.ts`
204+
- Tests: `*.test.ts` or `*.test.tsx`
205+
## Pull Request Process
206+
1. **Fork** the Perses repository
207+
2. **Create feature branch** from `main`
208+
```bash
209+
git checkout -b feature/your-feature-name
210+
```
211+
3. **Make your changes**
212+
- Write tests for new functionality
213+
- Update documentation
214+
- Follow code style guidelines
215+
4. **Test your changes**
216+
```bash
217+
npm test
218+
npm run type-check
219+
```
220+
5. **Commit with clear messages**
221+
```bash
222+
git commit -m "feat: add support for new SQL driver"
223+
```
224+
6. **Push to your fork**
225+
```bash
226+
git push origin feature/your-feature-name
227+
```
228+
7. **Create Pull Request** to `perses/perses` main branch
229+
### Commit Message Format
230+
Follow [Conventional Commits](https://www.conventionalcommits.org/):
231+
- `feat:` - New feature
232+
- `fix:` - Bug fix
233+
- `docs:` - Documentation changes
234+
- `test:` - Test changes
235+
- `refactor:` - Code refactoring
236+
- `chore:` - Maintenance tasks
237+
## Project Structure
238+
```
239+
sql/
240+
├── src/
241+
│ ├── datasources/sql-datasource/ # Datasource plugin
242+
│ ├── queries/sql-time-series-query/ # Query plugin
243+
│ ├── explore/ # Explore plugin
244+
│ └── model/ # Utilities
245+
├── schemas/ # CUE validation schemas
246+
├── test-data/ # Sample data for testing
247+
├── __mocks__/ # Jest mocks
248+
├── package.json # Plugin configuration
249+
├── rsbuild.config.ts # Module Federation config
250+
├── jest.config.ts # Test configuration
251+
└── Makefile # Database management
252+
```
253+
## Debugging
254+
### Backend Logs
255+
```bash
256+
# Start backend with debug logging
257+
./bin/perses --config ./dev/config.yaml --log.level=debug
258+
```
259+
### Plugin Logs
260+
```bash
261+
# Plugin automatically shows build output
262+
percli plugin start .
263+
```
264+
### Browser DevTools
265+
- F12 to open DevTools
266+
- Check Console for errors
267+
- Check Network tab for API calls
268+
## Common Issues
269+
### Plugin Not Loading
270+
- Check backend is running
271+
- Check plugin server is running (`percli plugin start .`)
272+
- Check browser console for errors
273+
- Verify Module Federation configuration
274+
### Database Connection Errors
275+
- Check database is running (`docker ps`)
276+
- Verify connection settings in datasource config
277+
- Check backend logs for detailed error messages
278+
### TypeScript Errors
279+
- Run `npm run type-check`
280+
- Check imports are correct
281+
- Ensure types are defined
282+
## Questions?
283+
- **Issues**: Open an issue in the Perses repository
284+
- **Discussions**: Join Perses community discussions
285+
- **Documentation**: Check Perses documentation
286+
## License
287+
By contributing, you agree that your contributions will be licensed under the Apache License 2.0.

0 commit comments

Comments
 (0)