Skip to content

Algorithm agility: PSS on the PKCS#11 shape, DSS TSA nonce, EC coverage - #478

Merged
kwart merged 5 commits into
masterfrom
algorithm-agility
Aug 9, 2026
Merged

Algorithm agility: PSS on the PKCS#11 shape, DSS TSA nonce, EC coverage#478
kwart merged 5 commits into
masterfrom
algorithm-agility

Conversation

@kwart

@kwart kwart commented Aug 9, 2026

Copy link
Copy Markdown
Member

Closes #23, closes #33, closes #255.

Design: design-doc/3.2-algorithm-agility.md. Three premises in the issue review were wrong; the doc and jsignpdf-issues-review.md are corrected.

#255 RSASSA-PSS (dss)

  • A PSS-only certificate in a software keystore already signed correctly. The real defect was the PKCS#11 shape: the key reports RSA under an id-RSASSA-PSS certificate, so the token and the parameters disagreed and DSS threw The SignatureAlgorithm within the SignatureValue 'RSA_SHA256' does not match ... 'RSA_SSA_PSS_SHA256_MGF1'.
  • The token no longer overrides DSS's sign(); AbstractSignatureTokenConnection supplies the PSSParameterSpec, and the engine passes parameters.getSignatureAlgorithm().
  • getSignatureInstance requests PSS under the portable RSASSA-PSS name and pins the key's provider. setParameter before initSign resolves the provider immediately, so delayed selection would pin SunRsaSign and reject a non-extractable token key.
  • openpdf structurally cannot emit PSS (no parameter slot in its SignerInfo writer).

#33 TSA nonce (dss)

  • engine.dss.tsa.nonce, default on. openpdf always sent one; DSS omits it unless a NonceSource is set.
  • EmbeddedTsaServer can now report the received nonce and echo a wrong one; the mismatch is rejected.

#23 EC keys

  • Fixed by the OpenPDF 3 migration, not by this PR. Tests on both engines.
  • dss writes ecdsa-with-SHA256, openpdf writes id-ecPublicKey; documented, and the guide recommends -eng dss for EC.

Not covered

  • The --signature-algorithm selector, whose only remaining function is forcing PSS on an rsaEncryption certificate.
  • PKCS#11 hardware. The provider-pinning branch has no CI coverage — needs a manual check with a PSS-capable token.

369 tests pass.

kwart added 2 commits August 9, 2026 09:13
- #255: drive the DSS token from the signature parameters instead of the
  key, so a PSS-only certificate signs when the key reports plain RSA (the
  PKCS#11 shape). Reuse AbstractSignatureTokenConnection for the PSS
  parameters; request PSS under the portable RSASSA-PSS name and pin the
  key's provider, since setParameter-before-initSign defeats delayed
  provider selection and would reject a non-extractable token key.
- #33: send an RFC 3161 nonce from the DSS engine and require the echo,
  switchable via engine.dss.tsa.nonce (default on). openpdf already did.
- #23: EC signing tests on both engines; fixed by the OpenPDF 3 migration.
  Note dss writes ecdsa-with-SHA256 where openpdf writes id-ecPublicKey.
- PrivateKeyInfo carries the keystore provider.

Deferred: the --signature-algorithm selector, needed only to force PSS on
an rsaEncryption certificate. See design-doc/3.2-algorithm-agility.md.
Provider pinning is required, not an optimisation: with the token's provider
the key produces a PSS signature in hardware; without it, JCA resolves to
SunRsaSign and fails with "key must be RSAPrivateKey".
@kwart

kwart commented Aug 9, 2026

Copy link
Copy Markdown
Member Author

PKCS#11 verified on hardware — Gemalto SafeNet eToken 5110 (ID Prime MD), SAC 10.9.6885, JDK 21. Driver offers RSA-PKCS-PSS and SHA256-RSA-PKCS-PSS as hw, sign; the key is P11Key$P11RSAPrivateKeyInternal (non-extractable).

Same key, three attempts, setParameter before initSign as the DSS base class does:

Attempt Result
SHA256withRSA, token provider OK, 256 bytes
RSASSA-PSS, token provider pinned (this PR) OK, 256 bytes
RSASSA-PSS, no provider InvalidKeyException: key must be RSAPrivateKey

So provider pinning is what makes PSS work on this token at all — without it JCA resolves to SunRsaSign, which cannot take an opaque P11 key.

Still not covered: a full PDF signing run on the token, which needs a certificate marked id-RSASSA-PSS over the on-card key (both certs on the test token are rsaEncryption, so DSS picks RSA and skips the PSS path).

CLI run with an on-card key under an id-RSASSA-PSS certificate produces a
PSS SignerInfo (SHA-256/MGF1/salt 32/trailer 1) that verifies.
@kwart

kwart commented Aug 9, 2026

Copy link
Copy Markdown
Member Author

End-to-end verified on the eToken 5110.

Setup: fresh on-card RSA-2048 key (sensitive, always sensitive, never extractable, local) plus a certificate whose SPKI is re-encoded as id-RSASSA-PSS over that key — the shape-2 combination on real hardware, key reporting RSA and certificate saying PSS. Existing identities on the token untouched.

Signed through the CLI, -eng dss -kst PKCS11 -ka pss-test:

SubFilter        : ETSI.CAdES.detached
digestAlg OID    : 2.16.840.1.101.3.4.2.1        (SHA-256)
signatureAlg OID : 1.2.840.113549.1.1.10         (RSASSA-PSS)
  hash           : 2.16.840.1.101.3.4.2.1        (SHA-256)
  maskGen        : 1.2.840.113549.1.1.8          (MGF1)
  saltLength     : 32
  trailerField   : 1
CMS verify       : VALID

Salt length and trailer field match the RFC 4055 convention the BC-name fallback depends on, so both PSS paths are byte-compatible as claimed. On master this same run fails in ensureSignatureValue.

Not covered: third-party validator acceptance (Adobe Reader) of the PSS output.

kwart added 2 commits August 9, 2026 16:35
PssTokenCertGen builds a certificate with an id-RSASSA-PSS SubjectPublicKeyInfo
over a token-exported public key, giving the PSS-cert/RSA-key shape on hardware
without any private-key operation on the token.
@kwart
kwart merged commit 48d7dd2 into master Aug 9, 2026
1 check passed
@kwart
kwart deleted the algorithm-agility branch August 9, 2026 21:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: Signing with RSASSA-PSS Support: Nonce error: Private keys must be instance of RSAPrivate(Crt)Key or have PKCS#8 encoding

1 participant