Skip to content

Commit 49c293a

Browse files
committed
chore: upgrade bundled SQLCipher and restore source-build fallback
- update the vendored SQLCipher amalgamation to 4.14.0 / SQLite 3.51.3 - add required SQLCipher init/shutdown build defines - switch macOS builds from vendored OpenSSL to CommonCrypto - restore runtime fallback from prebuilt binaries to local source builds - publish the native sources needed for downstream rebuilds - reduce published binaries to N-API 6 and trim legacy CI/docs
1 parent ae62115 commit 49c293a

14 files changed

Lines changed: 153502 additions & 97406 deletions

File tree

.circleci/config.yml

Lines changed: 3 additions & 131 deletions
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ aliases:
113113
- checkout
114114
- run: |
115115
export HOMEBREW_NO_AUTO_UPDATE=1
116-
brew install git-lfs openssl@1.1 || true
116+
brew install git-lfs || true
117117
- run: git lfs install
118118
- run: git lfs pull
119119
- run:
@@ -151,7 +151,7 @@ aliases:
151151
- checkout
152152
- run: |
153153
export HOMEBREW_NO_AUTO_UPDATE=1
154-
brew install git-lfs openssl@1.1 || true
154+
brew install git-lfs || true
155155
- run: git lfs install
156156
- run: git lfs pull
157157
- run:
@@ -233,21 +233,6 @@ jobs:
233233
docker:
234234
- image: circleci/node:16.9.1-stretch
235235

236-
build-14:
237-
<<: *common-build
238-
docker:
239-
- image: circleci/node:14.11.0-stretch
240-
241-
build-12:
242-
<<: *common-build
243-
docker:
244-
- image: circleci/node:12.6.0-stretch
245-
246-
build-10:
247-
<<: *common-build
248-
docker:
249-
- image: circleci/node:10.16.0-stretch
250-
251236
# Node version should match electron's node version.
252237
# See https://github.com/mapbox/node-sqlite3/pull/1367
253238
build-electron-16:
@@ -257,48 +242,6 @@ jobs:
257242
environment:
258243
ELECTRON_VERSION: "16.0.2"
259244

260-
build-electron-11:
261-
<<: *common-electron-linux
262-
docker:
263-
- image: circleci/node:12.18.3-stretch
264-
environment:
265-
ELECTRON_VERSION: "11.2.3"
266-
267-
build-electron-10:
268-
<<: *common-electron-linux
269-
docker:
270-
- image: circleci/node:12.16.3-stretch
271-
environment:
272-
ELECTRON_VERSION: "10.3.2"
273-
274-
build-electron-9:
275-
<<: *common-electron-linux
276-
docker:
277-
- image: circleci/node:12.14.1-stretch
278-
environment:
279-
ELECTRON_VERSION: "9.3.1"
280-
281-
build-electron-8:
282-
<<: *common-electron-linux
283-
docker:
284-
- image: circleci/node:12.13.0-stretch
285-
environment:
286-
ELECTRON_VERSION: "8.5.2"
287-
288-
build-electron-7:
289-
<<: *common-electron-linux
290-
docker:
291-
- image: circleci/node:12.8.1-stretch
292-
environment:
293-
ELECTRON_VERSION: "7.1.14"
294-
295-
build-electron-6:
296-
<<: *common-electron-linux
297-
docker:
298-
- image: circleci/node:12.6.0-stretch # Issues with 12.4.0
299-
environment:
300-
ELECTRON_VERSION: "6.1.9"
301-
302245

303246
build-macos-16:
304247
<<: *common-macos
@@ -316,63 +259,10 @@ jobs:
316259
BUILD_ARM64: true
317260
SKIP_TEST: true
318261

319-
build-macos-14:
320-
<<: *common-macos
321-
environment:
322-
NODE_VERSION: "14.11.0"
323-
ELECTRON_VERSION: "11.2.3"
324-
325-
build-macos-12:
326-
<<: *common-macos
327-
environment:
328-
NODE_VERSION: "12.6.0"
329-
330-
build-macos-10:
331-
<<: *common-macos
332-
environment:
333-
NODE_VERSION: "10.16.0"
334-
335262
# NODE_VERSION should match electron's node version.
336263
# See https://github.com/mapbox/node-sqlite3/pull/1367
337264

338-
build-macos-electron-11:
339-
<<: *common-electron-macos
340-
environment:
341-
ELECTRON_VERSION: "11.2.3"
342-
NODE_VERSION: "12.18.3"
343-
344-
build-macos-electron-10:
345-
<<: *common-electron-macos
346-
environment:
347-
ELECTRON_VERSION: "10.3.2"
348-
NODE_VERSION: "12.16.3"
349-
350-
build-macos-electron-9:
351-
<<: *common-electron-macos
352-
environment:
353-
ELECTRON_VERSION: "9.3.1"
354-
NODE_VERSION: "12.14.1"
355-
356-
build-macos-electron-8:
357-
<<: *common-electron-macos
358-
environment:
359-
ELECTRON_VERSION: "8.5.2"
360-
NODE_VERSION: "12.13.0"
361-
362-
build-macos-electron-7:
363-
<<: *common-electron-macos
364-
environment:
365-
ELECTRON_VERSION: "7.1.14"
366-
NODE_VERSION: "12.8.1"
367-
368-
build-macos-electron-6:
369-
<<: *common-electron-macos
370-
environment:
371-
ELECTRON_VERSION: "6.1.9"
372-
# Issues with 12.4.0
373-
NODE_VERSION: "12.6.0"
374-
375-
# Node 14 - build NAPI 3 and 6
265+
# Build the published N-API 6 binaries
376266
windows-16_x86:
377267
<<: *common-windows
378268
environment:
@@ -406,27 +296,9 @@ workflows:
406296
build_all:
407297
jobs:
408298
- build-16
409-
- build-14
410-
- build-12
411-
- build-10
412299
- build-electron-16
413-
- build-electron-11
414-
# - build-electron-10 # Segfaults on electron-mocha
415-
- build-electron-9
416-
- build-electron-8
417-
- build-electron-7
418-
- build-electron-6
419300
- build-macos-16-arm64
420301
- build-macos-16
421-
- build-macos-14
422-
- build-macos-12
423-
- build-macos-10
424-
- build-macos-electron-11
425-
# - build-macos-electron-10 # Segfaults on electron-mocha
426-
- build-macos-electron-9
427-
- build-macos-electron-8
428-
- build-macos-electron-7
429-
- build-macos-electron-6
430302
- windows-16_x86
431303
- windows-16_x64
432304
- windows-16_arm64

README.md

Lines changed: 21 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
Fork of [node-sqlite3](https://github.com/mapbox/node-sqlite3), modified to use [SQLCipher](https://www.zetetic.net/sqlcipher/).
22

3-
While the `node-sqlite3` project does include support for compiling against sqlcipher, it requires manual work, and does not work out-of-the-box on Electron on Windows. This fork changes the default configuration to bundle SQLCipher directly, as well as OpenSSL where required.
3+
This fork bundles SQLCipher directly and now supports rebuilding from source again when no matching prebuilt binary is available.
44

55
## Supported platforms
66

7-
Binaries are built against N-API 3 and 6, on MacOS, Windows (ia32 and x64) and Linux (x64).
7+
Published binaries are built against N-API 6.
88

9-
Node 10+ and Electron 6+ is supported.
9+
Older N-API 3 binaries are no longer published.
1010

11-
Other platforms/architectures may work by building from source - see the section below.
11+
If no matching binary is available for your platform or runtime, the published package includes the native sources needed to rebuild from source.
1212

1313
# Installation
1414

@@ -49,25 +49,27 @@ db.close();
4949

5050
# SQLCipher
5151

52-
A copy of the source for SQLCipher 4.4.2 is bundled, which is based on SQLite 3.33.0.
52+
A copy of the source for SQLCipher 4.14.0 is bundled, which is based on SQLite 3.51.3.
5353

5454
## Building from source.
5555

56-
Building from source when installing the package is only supported up to version 5.2.0.
56+
Building from source when installing the package is supported again.
5757

58-
The two pre-built versions (N-API 3 and N-API 6) cover all electron and node versions, so building from source should
59-
not be required.
58+
The published tarball includes `binding.gyp`, `deps/`, and `src/` so that `npm install --build-from-source`, `node-gyp rebuild`, and rebuild tools such as `electron-rebuild` can compile the addon when needed.
59+
60+
Platform notes:
61+
62+
1. macOS uses SQLCipher's CommonCrypto provider via `Security.framework` and does not require Homebrew `openssl@1.1`.
63+
2. Linux links against the system `libcrypto`.
64+
3. Windows release binaries continue to use vendored static OpenSSL artifacts from `deps/`.
6065

6166
## Usage with electron-forge / electron-rebuild
6267

63-
[electron-forge](https://www.electronforge.io/) uses [electron-rebuild](https://github.com/electron/electron-rebuild) and attempts to rebuild this library from source by default, in a way
64-
that is _not_ compatible with the way `node-pre-gyp` is used here.
68+
[electron-forge](https://www.electronforge.io/) uses [electron-rebuild](https://github.com/electron/electron-rebuild) and will rebuild native modules from source by default.
69+
70+
That rebuild path is now supported by the published package. If a matching prebuilt binary exists you can still skip rebuilding this module to speed up install times, but it is no longer required as a workaround.
6571

66-
The workaround is to disable the rebuilding:
67-
1. If using Electron 11+, use a node version that supports N-API 6+ (v10.20.0+ / v12.17.0+ / v14.0.0).
68-
2. After `npm install` / `yarn install`, make sure that the folder `node_modules/@journeyapps/sqlcipher/lib/binding/napi-v6-linux-x64` exists.
69-
If not, check the previous step again, remove the `node_modules` folder, and try again.
70-
3. Disable rebuilding of this library using the `onlyModules` option of `electron-rebuild` in your `package.json`:
72+
To skip rebuilding when a matching prebuilt binary is already present, disable rebuilding of this library using the `onlyModules` option of `electron-rebuild` in your `package.json`:
7173

7274
"config": {
7375
"forge": {
@@ -77,18 +79,17 @@ The workaround is to disable the rebuilding:
7779
}
7880
}
7981

80-
Note: [electron-builder](https://www.electron.build/) does not appear to have this issue, and should work directly.
81-
Similarly, using electron directly should just work, but do check that a compatible node version is used (see above).
82+
Note: [electron-builder](https://www.electron.build/) should continue to work directly.
8283

8384
## OpenSSL
8485

8586
SQLCipher depends on OpenSSL.
8687

87-
For Windows, we bundle OpenSSL 1.1.1i. Binaries are generated using [vckpg](https://github.com/microsoft/vcpkg) (e.g., `.\vcpkg\vcpkg install openssl:x64-windows-static`).
88+
On macOS we use CommonCrypto instead of a vendored OpenSSL build.
8889

89-
On Mac we bundle OpenSSL 1.1.1l.
90+
On Linux we dynamically link against the system OpenSSL / `libcrypto`.
9091

91-
On Linux we dynamically link against the system OpenSSL.
92+
For Windows release binaries we keep static OpenSSL artifacts in `deps/`. These can be refreshed with [vcpkg](https://github.com/microsoft/vcpkg) via `deps/openssl-windows.bat`.
9293

9394
# API
9495

SQLCipher.md

Lines changed: 25 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -4,20 +4,17 @@
44

55
## Step 1: Get the SQLCipher source
66

7-
Clone the sqlcipher repo, and build the amalgamation source.
7+
Start from an official SQLCipher release tarball or a release tag, then build the amalgamation source.
88

99
```
10-
git clone git@github.com:sqlcipher/sqlcipher.git
11-
cd sqlcipher
12-
./configure
10+
curl -L https://github.com/sqlcipher/sqlcipher/archive/refs/tags/v4.14.0.tar.gz -o sqlcipher.tar.gz
11+
tar xf sqlcipher.tar.gz
12+
cd sqlcipher-4.14.0
13+
./configure --enable-all --disable-tcl
1314
make sqlite3.c
14-
15-
VERSION=3031000
16-
mkdir sqlcipher-amalgamation-$VERSION
17-
cp sqlite3.c sqlite3.h shell.c sqlite3ext.h VERSION sqlcipher-amalgamation-$VERSION/
1815
```
1916

20-
The above produces 4 files of interest:
17+
The build produces the files we vendor in `deps/sqlcipher-amalgamation`:
2118

2219
```
2320
sqlite3.c
@@ -27,33 +24,31 @@ sqlite3ext.h
2724
VERSION # rename to VERSION.txt
2825
```
2926

30-
Copy these files to: `deps/sqlcipher-amalgamation`.
27+
Copy these files into `deps/sqlcipher-amalgamation`, renaming `VERSION` to `VERSION.txt`.
3128

3229
## Step 2: Get OpenSSL libraries
3330

34-
NodeJS typically includes OpenSSL. However, to support Electron, we statically link against libcrypto from OpenSSL.
31+
macOS no longer vendors OpenSSL in this repository. It uses SQLCipher's CommonCrypto provider through `Security.framework`.
32+
33+
Linux links against the system `libcrypto`.
34+
35+
Windows release binaries still vendor static OpenSSL artifacts because they are needed at compile time for rebuilds and published prebuilds.
3536

36-
Run the following commands to generate the latest OpenSSL libs for Windows:
37+
Run the following command on Windows to regenerate the vendored headers and static libraries:
3738

3839
```
3940
cd deps
4041
.\openssl-windows.bat
4142
```
4243

43-
... this will output the libs in `deps/` (OpenSSL-WinXX), including the header files in `deps/openssl-include`. Every arch-specific folder includes these binaries:
44-
45-
```
46-
libcrypto.lib
47-
libssl.lib
48-
ossl_static.pdb
49-
```
44+
This will refresh the files in `deps/OpenSSL-Win32`, `deps/OpenSSL-Win64`, `deps/OpenSSL-Win64-ARM`, and `deps/openssl-include`.
5045

5146
## Step 3: Test the build
5247

5348
Run:
5449

5550
```sh
56-
./node_modules/.bin/node-pre-gyp install --build-from-source
51+
./node_modules/.bin/node-gyp rebuild
5752
```
5853

5954
Then run the tests:
@@ -62,13 +57,20 @@ Then run the tests:
6257
npm run test
6358
```
6459

60+
If you want to verify the source-build fallback path specifically, temporarily move the matching prebuilt binary out of `lib/binding/` and rerun the tests.
61+
6562

6663
# Notes
6764

68-
The OpenSSL files are specifically required for Electron, which doesn't bundle OpenSSL like NodeJS does. The header and .lib files are required at compile-time. We bundle a statically-linked version of OpenSSL with the library, so the user does not need to manually install OpenSSL.
65+
This repository now builds SQLCipher with:
66+
67+
* `SQLITE_HAS_CODEC`
68+
* `SQLITE_EXTRA_INIT=sqlcipher_extra_init`
69+
* `SQLITE_EXTRA_SHUTDOWN=sqlcipher_extra_shutdown`
70+
* `SQLITE_TEMP_STORE=2`
6971

7072
`deps/sqlite3.gyp` has been modified from the original node-sqlite3 one to:
71-
* Use the bundled OpenSSL on Windows and MacOS.
73+
* Use CommonCrypto on macOS.
74+
* Use the vendored Windows OpenSSL headers and static libraries for Windows release binaries.
7275
* Add additional define statements required by SQLCipher.
7376

74-

binding.gyp

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,8 @@
22
"includes": [ "deps/common-sqlite.gypi" ],
33
"variables": {
44
"sqlite%":"internal",
5-
"sqlite_libname%":"sqlite3"
5+
"sqlite_libname%":"sqlite3",
6+
"module_name%":"node_sqlite3"
67
},
78
"targets": [
89
{
@@ -50,17 +51,6 @@
5051
"src/statement.cc"
5152
],
5253
"defines": [ "NAPI_VERSION=<(napi_build_version)", "NAPI_DISABLE_CPP_EXCEPTIONS=1" ]
53-
},
54-
{
55-
"target_name": "action_after_build",
56-
"type": "none",
57-
"dependencies": [ "<(module_name)" ],
58-
"copies": [
59-
{
60-
"files": [ "<(PRODUCT_DIR)/<(module_name).node" ],
61-
"destination": "<(module_path)"
62-
}
63-
]
6454
}
6555
]
6656
}
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
3.33.0
1+
3.51.3

0 commit comments

Comments
 (0)