Skip to content

Commit 8978fd7

Browse files
sathishksathishk
authored
Merge pull request #6 from Hari-Nikesh-R/main
Created the documentation for the core, compiler and project structure. Also added contribution md file.
2 parents 6d55878 + 0411ee0 commit 8978fd7

7 files changed

Lines changed: 470 additions & 0 deletions

File tree

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,5 @@ public
33
resources
44
dist
55
hugo_stats.json
6+
7+
.idea/

‎CONTRIBUTING.md‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# Contributing to SQL Components
2+
3+
Thank you for helping improve this project. This document explains how to work with the repository in a way that keeps changes reviewable and CI-friendly.
4+
5+
## Code of conduct
6+
7+
Be respectful and assume good intent in issues and pull requests. Follow the tone set in existing discussions and GitHub’s community guidelines.
8+
9+
## Before you start
10+
11+
1. **Read the docs** — [docs/README.md](docs/README.md) and the module pages under [docs/](docs/) explain architecture, Maven layout, and the datastore workflow.
12+
2. **Check the license** — [LICENSE](LICENSE) (Apache 2.0). Contributions are expected to be licensed under the same terms unless explicitly stated otherwise.
13+
14+
## Development setup
15+
16+
- **JDK**: The build uses **Java 17** in the Maven POMs. A JDK that matches `maven.compiler.source` / `target` is required.
17+
- **Maven**: Use the Maven Wrapper if present, or a recent Maven 3.x aligned with the project.
18+
- **Database (optional)**: For integration-style tests, PostgreSQL can be started with root `docker-compose.yml` (port **5432**, database `moviedb`). Align [database.properties](database.properties) with your environment.
19+
20+
## Building and testing
21+
22+
From the repository root (builds **core** and **compiler** only):
23+
24+
```bash
25+
mvn clean verify
26+
```
27+
28+
To skip tests when you only need a compile:
29+
30+
```bash
31+
mvn clean package -Dmaven.test.skip=true
32+
```
33+
34+
To build and test the **datastore** example module (not part of the default reactor):
35+
36+
```bash
37+
mvn -f datastore/pom.xml clean test
38+
```
39+
40+
Generate sources into `datastore/src/main/java` before datastore tests if that directory is empty (for example by running the compiler module’s generation tests or your own small driver that calls `Application.compile(new org.sqlcomponents.compiler.java.JavaCompiler())`).
41+
42+
## Style and quality
43+
44+
- **Checkstyle** runs in the `validate` phase. Fix violations rather than disabling rules unless there is a strong reason and a suppression entry is agreed with maintainers.
45+
- Match **existing code style**: naming, formatting, and Javadoc level in the files you touch.
46+
- Prefer **small, focused PRs** with a clear description of behavior change.
47+
48+
## How to contribute changes
49+
50+
1. **Fork** the repository (if contributing from outside the main org) and create a **feature branch** from the appropriate default branch.
51+
2. **Implement** your change with tests where behavior is non-trivial or regression-prone.
52+
3. **Run** `mvn clean verify` (and datastore tests if you changed generation or example code paths).
53+
4. **Open a pull request** that:
54+
- Summarizes the problem and the solution in complete sentences.
55+
- Links related issues (e.g. `Fixes #123`).
56+
- Calls out any breaking changes, migration steps, or follow-up work.
57+
58+
## Issues
59+
60+
Use the GitHub issue templates under `.github/ISSUE_TEMPLATE/` when filing bugs, features, or CI reports. Include:
61+
62+
- Expected vs actual behavior
63+
- JDK and Maven versions
64+
- Relevant logs or minimal reproduction steps
65+
66+
## Security
67+
68+
Do not open public issues for **undisclosed security vulnerabilities**. Contact maintainers through a private channel if one is documented on the repository; otherwise use GitHub’s security advisory feature if enabled.
69+
70+
## Questions
71+
72+
If something in [docs/](docs/) is unclear or wrong, opening an issue or a documentation-only PR is welcome.

‎docs/README.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# SQL Components documentation
2+
3+
This folder describes the **sqlcomponents** multi-module Maven project: how it reads database metadata, maps it to a Java-friendly model, generates persistence-oriented Java sources, and how the **datastore** example module validates that story end-to-end.
4+
5+
The root [README.md](../README.md) (sqlbridge notes, Docker, Maven commands) stays the quick operational entry; use this `docs/` tree for **architecture and module behavior**.
6+
7+
---
8+
9+
## High-level picture
10+
11+
SQL Components is a **code generator** built around JDBC:
12+
13+
1. **Connect** to a live database using URL, credentials, and driver settings held in `Application`.
14+
2. **Crawl** catalog metadata (tables, columns, keys, indexes, procedures, types, …) into relational POJOs in the **core** module.
15+
3. **Map** columns and entities to Java types and ORM structures (today: **`JavaMapper`** in the **compiler** module).
16+
4. **Generate** Java source files with **FreeMarker** templates—notably a façade **`DataManager`**, table-aligned **`…Store`** classes, and **`Record`** / enum-style types.
17+
5. **Consume** the generated code from application or test code (the **datastore** module demonstrates this against PostgreSQL).
18+
19+
```mermaid
20+
flowchart LR
21+
subgraph ingest [Ingestion]
22+
JDBC[JDBC metadata]
23+
Crawler[Crawler in core]
24+
end
25+
subgraph model [Model]
26+
App[Application ORM Entity]
27+
end
28+
subgraph gen [Generation]
29+
JC[JavaCompiler]
30+
FTL[FreeMarker templates]
31+
end
32+
subgraph use [Usage]
33+
Stores[DataManager and Stores]
34+
end
35+
JDBC --> Crawler
36+
Crawler --> App
37+
App --> JC
38+
FTL --> JC
39+
JC --> Stores
40+
```
41+
42+
**Design intent** (aligned with the root README themes): catch schema issues early, support multiple dialects, and emit readable Java that wraps SQL construction and execution—without replacing the database at runtime.
43+
44+
---
45+
46+
## Core-level reading guide
47+
48+
Read in this order if you are onboarding to the codebase:
49+
50+
| Order | Document | What you learn |
51+
|-------|----------|----------------|
52+
| 1 | [Project structure](project-structure.md) | Directories, which Maven modules build from the root reactor, dependency direction, where templates and generated files live. |
53+
| 2 | [Core module](core.md) | `Crawler`, relational model types, `Application`, `Compiler` SPI, YAML loading via `CoreConsts`. |
54+
| 3 | [Compiler module](compiler.md) | `JavaCompiler` pipeline, `JavaMapper`, FreeMarker template layout, how tests configure output paths. |
55+
| 4 | [Datastore module](datastore.md) | Example/integration tests, dependency on generated `org.example` code, building with `mvn -f datastore/pom.xml`. |
56+
57+
---
58+
59+
## Contributing and license
60+
61+
- **[CONTRIBUTING.md](../CONTRIBUTING.md)** — How to propose changes, run checks, and interact with maintainers.
62+
- **[LICENSE](../LICENSE)** — Legal terms for using and redistributing the project.
63+
64+
---
65+
66+
## Related links
67+
68+
- Upstream context in POM: [https://github.com/sqlcomponents/sqlcomponents](https://github.com/sqlcomponents/sqlcomponents)
69+
- Root [README.md](../README.md) — Docker Compose, `mvn` recipes, JDK notes.

‎docs/compiler.md‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Compiler module
2+
3+
The **compiler** module (`org.sqlcomponents:compiler`) turns a populated `Application` (from **core**) into Java source files: a **DataManager**, per-entity **Store** classes, **Record** (or similar) model types, and **enum**-style artifacts for certain database types. It implements `org.sqlcomponents.core.compiler.Compiler` via [`JavaCompiler`](../compiler/src/main/java/org/sqlcomponents/compiler/java/JavaCompiler.java).
4+
5+
For how metadata reaches `Application`, see [Core module](core.md). For where output is written in this repo’s workflow, see [Datastore module](datastore.md).
6+
7+
## End-to-end pipeline (`JavaCompiler.compile`)
8+
9+
At a high level, `JavaCompiler`:
10+
11+
1. **Optional Flyway migrations** — If Flyway is on the classpath and `src/main/resources/db/migration` exists relative to the process working directory, runs Flyway against a datasource built from `Application`, then closes the datasource. This is best-effort and environment-specific; see source for details.
12+
2. **Map relational model to ORM view** — Constructs [`JavaMapper`](../compiler/src/main/java/org/sqlcomponents/compiler/mapper/JavaMapper.java), which extends core’s [`Mapper`](../core/src/main/java/org/sqlcomponents/core/mapper/Mapper.java), and assigns the resulting `ORM` on `Application`.
13+
3. **Emit Java** — Uses FreeMarker via [`FTLTemplate`](../compiler/src/main/java/org/sqlcomponents/compiler/template/FTLTemplate.java) to render:
14+
- **Manager** — [`Manager.ftl`](../compiler/src/main/resources/template/java/Manager.ftl) → `DataManager.java` under the root package folder.
15+
- **Store** — [`Store.ftl`](../compiler/src/main/resources/template/java/Store.ftl) → `{Entity}Store.java` under each entity’s DAO package.
16+
- **Model / record** — [`Record.ftl`](../compiler/src/main/resources/template/java/Record.ftl) → `{Entity}.java` under the bean package.
17+
- **Enum / type** — For entities whose type is an enumerated DB type, [`Enum.ftl`](../compiler/src/main/resources/template/java/Enum.ftl) → `{Entity}Type.java`.
18+
19+
Entity processing uses a **parallel stream** over `orm.getEntities()` for throughput.
20+
21+
`Application.compile(Compiler)` (in core) **deletes the entire `srcFolder` tree** before calling `Compiler.compile`, so every run is a clean generation.
22+
23+
## `JavaMapper`
24+
25+
[`JavaMapper`](../compiler/src/main/java/org/sqlcomponents/compiler/mapper/JavaMapper.java) is responsible for mapping SQL column semantics (via core’s `Column` / `ColumnType`) to Java types—for example primitives, `String`, `UUID`, `BigDecimal`, JTS geometry types, JSON node types, and time API classes. The mapper feeds the templates so generated fields and method signatures match the database.
26+
27+
## FreeMarker templates
28+
29+
Templates live under [`compiler/src/main/resources/template/java/`](../compiler/src/main/resources/template/java/). `FTLTemplate` loads them from the classpath relative to `JavaCompiler`.
30+
31+
Notable groups:
32+
33+
| Subdirectory / file | Purpose |
34+
|---------------------|-----------|
35+
| `Manager.ftl`, `Store.ftl`, `Record.ftl`, `Enum.ftl` | Top-level artifacts per entity or application. |
36+
| `column/*.ftl` | Fragments for SQL/expression handling per Java column kind (e.g. `BooleanColumn.ftl`, `UUIDColumn.ftl`, geometry columns). |
37+
| `method/*.ftl` | Statement builders: `SelectStatement`, `InsertStatement`, `UpdateStatement`, `DeleteStatement`, `MViewRefresh`. |
38+
| `query/*.ftl`, `clause/*.ftl` | Query and WHERE composition. |
39+
| `base.ftl`, `jdbcbase.ftl`, `SqlBuilder.ftl`, … | Shared includes. |
40+
| `template/directive/` | Custom FreeMarker directives (e.g. column selection). |
41+
42+
Generated `package-info.java` files are also written with a minimal `package …;` declaration for each output package.
43+
44+
## Dependencies (`compiler/pom.xml`)
45+
46+
- **FreeMarker** — Template engine.
47+
- **JTS Core** — Geometry types referenced by `JavaMapper` / templates for spatial columns.
48+
- **Flyway** (`provided` / optional usage) — Migration hook described above.
49+
- **core** — Same-version `org.sqlcomponents:core`.
50+
51+
Drivers and pooling come from the **parent** POM when running in tests or integrated apps.
52+
53+
## How to run code generation in practice
54+
55+
### Properties-based setup (tests / local dev)
56+
57+
[`CompilerTestUtil.getApplication()`](../compiler/src/test/java/org/sqlcomponents/compiler/java/util/CompilerTestUtil.java) builds an `Application` when `SQLCOMPONENTS_CONFIG` is **not** set:
58+
59+
- Reads [`database.properties`](../database.properties) from the current working directory or parent (`../database.properties`).
60+
- Uses `DATABASE_TYPE` env var or defaults to `postgres` for property key prefix (`postgres.datasource.url`, …).
61+
- Sets `rootPackage`, encryption lists, insert/update maps, and **`srcFolder`** to `datastore/src/main/java` (or `../datastore/src/main/java` depending on cwd).
62+
63+
Then:
64+
65+
```java
66+
Application application = CompilerTestUtil.getApplication();
67+
application.compile(new JavaCompiler());
68+
```
69+
70+
Run from Maven with the working directory that matches your paths (often `compiler/` for tests).
71+
72+
### YAML-based setup (CI / explicit config)
73+
74+
When **`SQLCOMPONENTS_CONFIG`** points to a YAML file:
75+
76+
- `Application` is built with [`CoreConsts.buildApplication(File)`](../core/src/main/java/org/sqlcomponents/core/utils/CoreConsts.java).
77+
- **`SOURCE_FOLDER`** must be set to the output directory for generated sources (throws if missing).
78+
79+
## SPI: `Compiler` interface
80+
81+
The compiler module is the reference implementation of:
82+
83+
```java
84+
void compile(Application application) throws SQLException;
85+
```
86+
87+
Other implementations could target different languages or layouts as long as they accept `Application`.
88+
89+
## Related documentation
90+
91+
- [Core module](core.md)
92+
- [Datastore module](datastore.md)
93+
- [Project structure](project-structure.md)
94+
- [Documentation index](README.md)

‎docs/core.md‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# Core module
2+
3+
The **core** module (`org.sqlcomponents:core`) is the foundation of SQL Components. It connects to a database through JDBC, reads catalog metadata, and represents that metadata as Java objects. It also defines the configuration surface (`Application`) and the small **Compiler** SPI that the **compiler** module implements.
4+
5+
For repository layout and Maven coordinates, see [Project structure](project-structure.md).
6+
7+
## Responsibilities at a glance
8+
9+
1. **Introspection**: Walk JDBC `DatabaseMetaData` (and related queries) to build a graph of schemas, tables, columns, keys, indexes, procedures, and user-defined types.
10+
2. **Domain model**: Express that graph as immutable-friendly POJOs under `org.sqlcomponents.core.model` and `org.sqlcomponents.core.model.relational`.
11+
3. **Configuration**: Hold everything needed to open a datasource, filter objects, name Java packages, and tune ORM behavior—primarily via `Application` and nested `ORM`.
12+
4. **Compilation contract**: Declare `org.sqlcomponents.core.compiler.Compiler` so other modules can generate artifacts without core knowing about FreeMarker or Java source layout.
13+
14+
## Package map
15+
16+
| Package | Role |
17+
|---------|------|
18+
| `org.sqlcomponents.core.crawler` | `Crawler` base logic; database-specific subclasses (e.g. PostgreSQL, MySQL) refine behavior. |
19+
| `org.sqlcomponents.core.crawler.util` | Helpers such as `DataSourceUtil` for constructing datasources. |
20+
| `org.sqlcomponents.core.model` | Top-level app concepts: `Application`, `Entity`, `ORM`, `Method`, `Service`, … |
21+
| `org.sqlcomponents.core.model.relational` | Relational catalog types: `Database`, `Table`, `Column`, `Key`, `Index`, `Procedure`, `Type`, constraints, `Package`, enums (`ColumnType`, `DBType`, `TableType`, …). |
22+
| `org.sqlcomponents.core.compiler` | `Compiler` interface: `void compile(Application application) throws SQLException`. |
23+
| `org.sqlcomponents.core.mapper` | Abstract `Mapper` used by language-specific mappers (e.g. `JavaMapper` in the compiler module). |
24+
| `org.sqlcomponents.core.exception` | Shared error types. |
25+
| `org.sqlcomponents.core.utils` | `CoreConsts` and other utilities. |
26+
27+
## Crawler vs dialect crawlers
28+
29+
- **`Crawler`** ([`Crawler.java`](../core/src/main/java/org/sqlcomponents/core/crawler/Crawler.java)) centralizes JDBC metadata access: tables, columns, imported/exported keys, indexes, procedures, and more. It recognizes product names such as PostgreSQL, MySQL/MariaDB, H2, Oracle, SQL Server.
30+
- **Dialect-specific classes** (e.g. [`PostgresCrawler`](../core/src/main/java/org/sqlcomponents/core/crawler/PostgresCrawler.java), [`MysqlCrawler`](../core/src/main/java/org/sqlcomponents/core/crawler/MysqlCrawler.java)) extend or specialize behavior where generic metadata is insufficient or dialect-specific SQL is required.
31+
32+
The crawler layer uses **HikariCP** (provided at parent POM level) and standard JDBC types.
33+
34+
## Key types
35+
36+
### `Application`
37+
38+
[`Application`](../core/src/main/java/org/sqlcomponents/core/model/Application.java) is the root configuration object: human-readable name, JDBC URL, credentials, driver, schema, table/sequence include patterns, target `srcFolder`, `rootPackage`, naming maps (`wordsMap`, `modulesMap`, `pluralMap`), insert/update default maps, encryption column lists, and references to `ORM`.
39+
40+
Important behavior:
41+
42+
- **`compile(Compiler)`**: If `srcFolder` exists, it is **deleted recursively** (newest paths first), then `compiler.compile(this)` is invoked. Any implementation must repopulate that directory.
43+
44+
### `ORM`
45+
46+
Holds schema-level ORM settings and the entity graph produced after mapping (see compiler module’s mapper). `Application` delegates many getters/setters to `ORM`.
47+
48+
### `Entity`
49+
50+
Represents one generated unit (typically aligned with a table, view, or enum type), carrying relational metadata and naming used by templates.
51+
52+
### Relational types
53+
54+
Types under `org.sqlcomponents.core.model.relational` mirror JDBC catalog concepts so the compiler can emit Java without re-querying the database during generation.
55+
56+
## Configuration loading (YAML)
57+
58+
[`CoreConsts.buildApplication(File)`](../core/src/main/java/org/sqlcomponents/core/utils/CoreConsts.java) loads an `Application` from a YAML file using SnakeYAML’s `Constructor(Application.class)`, then sets `methodSpecification` to `Application.METHOD_SPECIFICATION` by default.
59+
60+
Compiler tests can instead use properties-based setup via [`CompilerTestUtil`](../compiler/src/test/java/org/sqlcomponents/compiler/java/util/CompilerTestUtil.java) when `SQLCOMPONENTS_CONFIG` is not set.
61+
62+
## Dependencies (`core/pom.xml`)
63+
64+
- **SnakeYAML**: YAML configuration for `Application`.
65+
- **JetBrains annotations**: Nullable/not-null hints for static analysis.
66+
67+
Other JDBC drivers and connection pooling are inherited from the **parent** POM for the whole project; core code expects a JDBC `DataSource` or equivalent when crawling.
68+
69+
## Relationship to the compiler module
70+
71+
The compiler module depends on **core** only. Core does **not** reference FreeMarker or `JavaCompiler`. Generation is triggered by application code calling `application.compile(new JavaCompiler())` (or another `Compiler` implementation).
72+
73+
## Related documentation
74+
75+
- [Compiler module](compiler.md)
76+
- [Datastore module](datastore.md)
77+
- [Project structure](project-structure.md)
78+
- [Documentation index](README.md)

0 commit comments

Comments
 (0)