Add ability to define config via environment variables + add local development tooling

README.md

@@ -1,64 +1,150 @@
 # Getting Started
-## Frontend
-Download or clone the repository on github. Then, make sure to install 
-all dependencies by running the following command in your terminal in the /frontend directory:
+
+SynBioHub3 runs as two separate processes:
+
+- **`frontend/`** — a Next.js app served on port **3333**
+- **`backend/`** — a Java (Spring Boot) app served on port **6789**
+
+The backend also needs a **Virtuoso** triplestore (a SPARQL endpoint) to be useful.
+
+For full, component-specific instructions see [`frontend/README.md`](./frontend/README.md)
+and [`backend/README.md`](./backend/README.md).
+
+## Quick start (one command)
+
+From the repository root you can start Virtuoso, the backend, and the frontend
+together:
+
 ```bash
-yarn install
-# or
-npm install
+npm install        # one time — installs the root dev tooling (concurrently)
+npm run dev        # or: yarn dev
 ```
 
-In order to connect the frontend to your backend, go to the next.config.js file.
-Then, change the env backendUrl variable to your backend's url.
+Requirements: **Docker** (for Virtuoso), **Java 17**, **Node 17+**, and **Yarn**.
+
+`npm run dev` will:
+
+1. Start a Virtuoso triplestore in Docker — creating the `virtuoso` container the
+   first time and reusing it afterwards (see `db:up` / `db:down` below).
+2. Run the backend (`./mvnw spring-boot:run`) on **http://localhost:6789**.
+3. Run the frontend (`yarn devNextGen`, the OpenSSL-legacy mode required by Node
+   17+) on **http://localhost:3333**.
 
-Finally, run the development server:
+Output from both services is shown together, prefixed with `backend`/`frontend`.
+Press `Ctrl+C` to stop the backend and frontend. The Virtuoso container is left
+running so the next start is fast; stop it explicitly with:
 
 ```bash
-npm run dev
-# or
-yarn dev
+npm run db:down    # docker stop virtuoso
 ```
 
-NOTE, we use the legacy openssl with security vulnerabilities. This makes the dev command only work on mac and linux and introduces security vulnerablilites, to read more see [here](https://stackoverflow.com/questions/69692842/error-message-error0308010cdigital-envelope-routinesunsupported).
+> The root `dev` script runs the frontend via `yarn devNextGen`, which requires
+> Node 17+. On Node 14–16 (matching the Docker image) that flag is rejected — run
+> the services separately as described below and use `yarn dev` in `frontend/`.
 
-Open [http://localhost:3333](http://localhost:3333) with your browser to see the result.
+To run each piece by hand (or on Node 14–16), follow the component sections below.
+Start the backend first.
 
-### Developer Notes
+## Backend (port 6789)
 
-Each component/page in SynBioHub should have a header which dicatates its purpose.
+Requirements: **Java 17** (the project targets 17 — newer JDKs are not guaranteed to
+work with Spring Boot 3.0.1) and **Docker** (for Virtuoso).
 
-This app utilizes Redux to handle global application state and simplify the passing of
-deeply nested props. To view how this application uses Redux, see the /redux directory.
+1. Start a Virtuoso triplestore on port 8890:
 
-This app uses eslint that is set up for React/Next.js code. To run the linter, navigate to
-the frontend directory in your terminal (the directory this README is in) and run the command:
-```
-npm run lint
-```
-If you'd like to format all code and fix minor styling errors (recommened before pushing anything
-to the directory), run the command:
-```
-npm run lint.fix
-```
+   ```bash
+   docker run --name virtuoso -d \
+     -p 8890:8890 -p 1111:1111 \
+     -e DBA_PASSWORD=dba -e SPARQL_UPDATE=true \
+     tenforce/virtuoso:virtuoso7.2.5
+   ```
 
-### Next.js Resources
+2. Point the backend at that Virtuoso. The bundled
+   `backend/src/main/resources/config.json` defaults to the `virtuoso3` Docker host
+   (used when the backend itself runs as a container). Running on the host, reach the
+   container above at `localhost` by exporting two environment variables — they take
+   precedence over `config.json` without editing it:
 
-To learn more about Next.js, take a look at the following resources:
+   ```bash
+   export SBH_SPARQL_ENDPOINT="http://localhost:8890/sparql"
+   export SBH_GRAPH_STORE_ENDPOINT="http://localhost:8890/sparql-graph-crud-auth/"
+   ```
 
-- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
-- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.
+   (`npm run dev` already sets these for you — see above. To point at a Virtuoso
+   running elsewhere, change the host in the variables, or create
+   `backend/data/config.local.json`, which is gitignored and overrides `config.json`
+   per-key but is itself overridden by the environment variables above.)
+
+3. Run the backend from the `backend/` directory (config is read relative to the
+   working directory). This uses the bundled Maven wrapper and the embedded H2
+   database — no external database required:
+
+   ```bash
+   cd backend
+   ./mvnw spring-boot:run
+   ```
+
+   The backend is now serving on http://localhost:6789.
+
+## Frontend (port 3333)
+
+Requirements: **Node 14–16** and **Yarn**. (The Docker image uses Node 14; newer
+Node versions need the OpenSSL workaround in step 3.)
 
+1. Install dependencies:
 
-## Backend
-### Enviroment Setup
-[IntelliJ IDEA](https://www.jetbrains.com/idea/download/) is highly recommended for backend development. The community edition is free to download.
-Make sure that a JDK is available on your machine. Java version 11 or greater is recommended. If you do not have a JDK, go [here](https://www.azul.com/downloads/?package=jdk#download-openjdk) and download one for your system. Java 17 LTS is recommended.
+   ```bash
+   cd frontend
+   yarn install
+   ```
 
-## Project Setup
-Once you have finished installing the developer tools, clone the repository and open it as a project in IntelliJ. To setup your JDK, go to File->Project Structure->Project->ProjectSDK and click on your downloaded Java JDK.
+2. Tell the frontend where the backend is. It reads the backend URL from the
+   `backend` environment variable, loaded in dev from `frontend/.env.development`,
+   which ships pointing at this repo's backend (`http://localhost:6789`). Edit that
+   file if your backend runs elsewhere. (`next.config.js` exposes it via
+   `publicRuntimeConfig` — there is no `backendUrl` variable to edit.)
 
-## Running
-You can run the application by right clicking the `Synbiohub3Application` class and clicking run. On later runs, simply clikc the run or debut icons at the top right of your screen.
+3. Start the dev server:
 
-## Troubleshooting
-If IntelliJ is warning about packages not being found, right click on the `pom.xml` file and click Maven->Reload Project. This should download all required dependencies to run the application.
+   ```bash
+   yarn dev
+   ```
+
+   On Node 17+ this fails with an OpenSSL error (`ERR_OSSL_EVP_UNSUPPORTED`) because
+   Next.js 10 / webpack 4 use a legacy algorithm. Use the legacy-OpenSSL script
+   instead:
+
+   ```bash
+   yarn devNextGen
+   ```
+
+   Open [http://localhost:3333](http://localhost:3333) in your browser.
+
+## Run everything with Docker (prebuilt images)
+
+`tests/docker-compose.yml` starts Virtuoso, the backend, and the frontend from
+published images (not your local code) — handy for a quick instance, not for
+development:
+
+```bash
+cd tests && docker compose up
+```
+
+## Developer Notes
+
+Each component/page in SynBioHub should have a header which dictates its purpose.
+
+The frontend uses Redux for global application state and to simplify passing deeply
+nested props; see the `frontend/redux` directory.
+
+The frontend uses ESLint configured for React/Next.js. From the `frontend` directory:
+
+```bash
+npm run lint        # check for problems
+npm run lint.fix    # auto-fix styling errors (recommended before pushing)
+```
+
+### Next.js Resources
+
+- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
+- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.

backend/README.md

@@ -41,14 +41,27 @@ Open ``backend`` as a project in IntelliJ.
 To setup your JDK, go to ``File``->``Project Structure``->``Project``->``ProjectSDK``
 and click on your downloaded Java JDK.
 
-If you're not using IntelliJ:
-To build a JAR file using maven, navigate to the ``backend`` directory and run:
+If you're not using IntelliJ, use the bundled Maven wrapper (`./mvnw`) — a
+separate Maven install is not required. From the ``backend`` directory:
 
-``mvn package``
+Run directly (recommended for development):
 
-and run the JAR file with
+``./mvnw spring-boot:run``
 
-``java -jar target/{name of JAR}``
+Or build a JAR and run it:
+
+``./mvnw package``
+
+``java -jar target/sbh3-0.0.1-SNAPSHOT-exec.jar``
+
+The backend serves on **http://localhost:6789** and uses an embedded H2 database
+by default, so no external database is needed. (The `local` Spring profile in
+`application-local.yml` switches to PostgreSQL on `:5432`; activate it with
+`./mvnw spring-boot:run -Dspring-boot.run.profiles=local` only if you want that.)
+
+**Important:** the backend reads `src/main/resources/config.json` relative to its
+working directory, so it must be run from the ``backend`` directory (IntelliJ users
+already set this in the working-directory step above).
 
 #### _Running_
 
@@ -65,6 +78,44 @@ The main class is the ``Synbiohub3Application`` class.
 Note: If packages or dependencies aren't found, click on ``Maven`` in the top right corner
 and click ``Reload all Maven projects``.
 
+#### _Connecting to Virtuoso (local development)_
+
+The backend needs a Virtuoso triplestore for SPARQL queries (set up below). The
+bundled `src/main/resources/config.json` defaults to the `virtuoso3` Docker host,
+because the backend is normally run as a container alongside Virtuoso on the same
+Docker network. When you run the backend directly via `java`/`./mvnw` on the host,
+Virtuoso is reached at `localhost` instead. To point the backend there at runtime
+set these environment variables before running the backend:
+
+```sh
+export SBH_SPARQL_ENDPOINT="http://localhost:8890/sparql"
+export SBH_GRAPH_STORE_ENDPOINT="http://localhost:8890/sparql-graph-crud-auth/"
+./mvnw spring-boot:run
+```
+
+These take precedence over both `config.json` and `config.local.json`, so the
+committed Docker defaults stay intact. (In IntelliJ, add them under
+`Edit Configuration` → `Environment variables`.)
+
+> **Any** key in `config.json` can be overridden this way: prefix the key with
+> `SBH_` in upper snake case (e.g. `sparqlEndpoint` → `SBH_SPARQL_ENDPOINT`,
+> `useSBOLExplorer` → `SBH_USE_SBOL_EXPLORER`, `instanceUrl` → `SBH_INSTANCE_URL`).
+> Values are parsed as JSON, so types are preserved — `SBH_USE_SBOL_EXPLORER=true`
+> is a boolean, `SBH_FETCH_LIMIT=1000` a number — while plain strings like a URL are
+> used as-is. This is the recommended way to supply secrets (`SBH_PASSWORD`,
+> `SBH_SESSION_SECRET`, …) and per-environment settings without editing committed files.
+
+Alternatively — or to override other settings — create
+`backend/data/config.local.json` (gitignored; takes precedence over `config.json`
+on a per-key basis, but is itself overridden by the environment variables above):
+
+```json
+{
+  "sparqlEndpoint": "http://your-host:8890/sparql",
+  "graphStoreEndpoint": "http://your-host:8890/sparql-graph-crud-auth/"
+}
+```
+
 ## **Installing the Backend from Docker**
 
 Docker is recommended for normal use and testing. 

backend/src/main/java/com/synbiohub/sbh3/utils/ConfigUtil.java

@@ -1,15 +1,19 @@
 package com.synbiohub.sbh3.utils;
 
 
+import com.fasterxml.jackson.core.JsonParser;
 import com.fasterxml.jackson.databind.JsonNode;
 import com.fasterxml.jackson.databind.ObjectMapper;
 import com.fasterxml.jackson.databind.node.ObjectNode;
+import com.fasterxml.jackson.databind.node.TextNode;
 import lombok.extern.slf4j.Slf4j;
 import org.springframework.stereotype.Component;
 
 import java.io.File;
 import java.io.IOException;
 import java.nio.file.Files;
+import java.util.HashMap;
+import java.util.Map;
 
 @Slf4j
 @Component
@@ -19,9 +23,73 @@ public class ConfigUtil {
     private static JsonNode localjson;
     private static ObjectMapper mapper = new ObjectMapper();
 
+    /**
+     * Prefix marking environment variables that override config values. Any key in
+     * {@code config.json} (or {@code config.local.json}) can be set this way — e.g.
+     * {@code SBH_SPARQL_ENDPOINT} overrides {@code sparqlEndpoint}. Environment variables take
+     * precedence over both config files, so settings such as the triplestore location can be
+     * supplied at runtime without editing either file. This is what lets the backend run directly
+     * via {@code java} against a Virtuoso on {@code localhost} while the committed defaults target
+     * the {@code virtuoso} Docker host.
+     */
+    private static final String ENV_PREFIX = "SBH_";
+
+    /**
+     * Overrides keyed by the normalized config key (lower-cased, non-alphanumerics stripped) so
+     * that {@code SBH_SPARQL_ENDPOINT} matches {@code sparqlEndpoint} and {@code SBH_USE_SBOL_EXPLORER}
+     * matches {@code useSBOLExplorer} regardless of word boundaries. Built once from the process
+     * environment, which is immutable at runtime.
+     */
+    private static Map<String, JsonNode> envOverrides = Map.of();
+
+    private static String normalizeKey(String key) {
+        return key == null ? "" : key.toLowerCase().replaceAll("[^a-z0-9]", "");
+    }
+
+    /**
+     * Builds the {@link #envOverrides} index. Each {@code SBH_}-prefixed variable's value is parsed
+     * as JSON so booleans/numbers/objects keep their type (e.g. {@code SBH_USE_SBOL_EXPLORER=true}
+     * becomes a boolean, {@code SBH_FETCH_LIMIT=1000} a number). Values that are not valid JSON —
+     * a bare URL, for instance — are kept as plain strings.
+     */
+    private static Map<String, JsonNode> loadEnvOverrides() {
+        Map<String, JsonNode> overrides = new HashMap<>();
+        for (Map.Entry<String, String> entry : System.getenv().entrySet()) {
+            String name = entry.getKey();
+            String value = entry.getValue();
+            if (name == null || !name.startsWith(ENV_PREFIX) || value == null || value.isBlank()) {
+                continue;
+            }
+            String normalized = normalizeKey(name.substring(ENV_PREFIX.length()));
+            if (!normalized.isEmpty()) {
+                overrides.put(normalized, parseEnvValue(value));
+            }
+        }
+        return overrides;
+    }
+
+    private static JsonNode parseEnvValue(String value) {
+        try (JsonParser parser = mapper.getFactory().createParser(value)) {
+            JsonNode node = mapper.readTree(parser);
+            // Require the whole value to be valid JSON; otherwise (e.g. "150mb", a URL) keep it as text.
+            if (node != null && !node.isNull() && !node.isMissingNode() && parser.nextToken() == null) {
+                return node;
+            }
+        } catch (IOException ignored) {
+            // not JSON — fall through to a plain string
+        }
+        return TextNode.valueOf(value);
+    }
+
+    private static JsonNode getEnvOverride(String key) {
+        JsonNode value = envOverrides.get(normalizeKey(key));
+        return (value != null && !value.isNull()) ? value : null;
+    }
+
     public ConfigUtil() {
         localjson = null;
         json = null;
+        envOverrides = loadEnvOverrides();
         try {
             json = mapper.readValue(new File("src/main/resources/config.json"), JsonNode.class);
             if (Files.exists(new File("data/config.local.json").toPath())) {
@@ -38,6 +106,13 @@ public static JsonNode get(String key) throws IOException {
 
         if (key.isEmpty())
             return json;
+
+        // Environment variables take precedence over both config files (see ENV_PREFIX).
+        JsonNode envValue = getEnvOverride(key);
+        if (envValue != null) {
+            return envValue;
+        }
+
         try {
             var item = localjson.get(key); // TODO: need to rebuild the object into memory every time it is written to
             if (item != null && !item.isNull()) {

frontend/.env.development

@@ -1,2 +1,2 @@
-backend="http://localhost:7777"
-backendSS="http://localhost:7777"
+backend="http://localhost:6789"
+backendSS="http://localhost:6789"