diff --git a/CHANGELOG.md b/CHANGELOG.md index 72acf229..0d0e663e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/). The format is based on [Keep a Changelog](http://keepachangelog.com/). +## Version 2.1.0 + +### Added + +- Support for `telemetry-to-caas` kind for CaaS (Collector as a Service) with automatic mTLS certificate management and rotation via Zero Trust Identity Service (ZTI/SPIFFE). Falls back to base64-encoded certificates from environment variables when ZTI is not available or explicitly disabled. + ## Version 2.0.1 - 2026-07-03 ### Fixed diff --git a/README.md b/README.md index 6843b802..3c8f13fe 100644 --- a/README.md +++ b/README.md @@ -25,6 +25,7 @@ Documentation can be found at [cap.cloud.sap](https://cap.cloud.sap/docs) and [o - [`telemetry-to-console`](#telemetry-to-console) - [`telemetry-to-dynatrace`](#telemetry-to-dynatrace) - [`telemetry-to-cloud-logging`](#telemetry-to-cloud-logging) + - [`telemetry-to-caas`](#telemetry-to-caas) - [`telemetry-to-jaeger`](#telemetry-to-jaeger) - [`telemetry-to-otlp`](#telemetry-to-otlp) - [Detailed Configuration Options](#detailed-configuration-options) @@ -200,7 +201,7 @@ Please note that in order for logs to be exported via OpenTelemetry, `cds.log()` ## Predefined Kinds -There are five predefined kinds as follows: +There are six predefined kinds as follows: ### `telemetry-to-console` @@ -283,12 +284,86 @@ In order to receive OpenTelemetry credentials in the binding to the SAP Cloud Lo If you are binding your app to SAP Cloud Logging via a [user-provided service instance](https://docs.cloudfoundry.org/devguide/services/user-provided.html), make sure that it has the tag `Cloud Logging`. -> Tip: To add the required tag to an existing user-provided service, you can use: +> Tip: To add the required tag to an existing user-provided service, you can use: > ``` > cf update-user-provided-service {service-name} -t "Cloud Logging" > ``` > For detailed information about binding resolution in CAP, consult [`cds.connect()` → Service Bindings](https://cap.cloud.sap/docs/node.js/cds-connect#service-bindings). +### `telemetry-to-caas` + +Exports traces, metrics, and logs to CaaS (Collector as a Service). +CaaS acts as a managed OpenTelemetry Collector that can route telemetry data to downstream backends like SAP Cloud Logging. + +Use via `cds.requires.telemetry.kind = 'to-caas'`. + +Required additional dependencies: +- `@opentelemetry/exporter-trace-otlp-proto` +- `@opentelemetry/exporter-metrics-otlp-proto` +- `@opentelemetry/exporter-logs-otlp-proto` (if using log export) + +CaaS requires mTLS authentication. There are two ways to provide the mTLS certificates: + +#### Option 1: Zero Trust Identity (ZTI) with SPIRE (Recommended) + +ZTI with SPIRE sidecar automatically provisions and rotates mTLS certificates (SVID files). This is the recommended approach for production. + +1. **Bind ZTI service** to your app: +```yaml +# mta.yaml +requires: + - name: my-zti-instance +``` + +2. **Bind CaaS service** to your app: +```yaml +# mta.yaml +requires: + - name: my-caas-instance +``` + +That's it! The plugin automatically detects ZTI and uses the SVID files provisioned by the SPIRE sidecar for mTLS authentication. + +**How it works**: The SPIRE sidecar provisions SVID certificate files in parallel with app startup. Since these files may not exist immediately, `@cap-js/telemetry` uses a lazy exporter that buffers telemetry data until the credentials become available. Once the SVID files are ready, buffered data is flushed and subsequent telemetry is exported normally. Certificate rotation is handled automatically. + +> **Note**: For optimal buffering behavior in production, ensure `NODE_ENV=production` is set. This enables batch processing with periodic export intervals (5s for traces/logs, 60s for metrics), ensuring buffered telemetry is flushed shortly after ZTI credentials become ready. In development mode, traces and logs use immediate export per-request. + +To explicitly disable ZTI (e.g., for testing), set: +```bash +CDS_REQUIRES_TELEMETRY_USE_ZTI=false +``` + +#### Option 2: Manual Certificate Configuration (Fallback) + +For environments without ZTI or when you want to manage certificates externally, you can provide mTLS credentials manually: + +1. **Bind the CaaS service** to your app with subject/issuer configuration: +```yaml +# mta.yaml +requires: + - name: my-caas-instance + parameters: + config: + subject: "CN=my-app,..." + issuer: "CN=SAP PKI Certificate Service Client CA,..." +``` + +2. **Provide mTLS credentials** via environment variables (base64 encoded): +```yaml +# mta.yaml +properties: + CDS_REQUIRES_TELEMETRY_X509_CERT: '' + CDS_REQUIRES_TELEMETRY_X509_KEY: '' +``` + +Or set directly via Cloud Foundry CLI: +```bash +cf set-env my-app CDS_REQUIRES_TELEMETRY_X509_CERT "" +cf set-env my-app CDS_REQUIRES_TELEMETRY_X509_KEY "" +``` + +The mTLS certificate must be SAP-signed through the BTP Certificate Service. Certificates can be created with validity from 7 days up to 1 year and must be renewed before expiration. For detailed certificate setup and renewal instructions, refer to your project's certificate management documentation. + ### `telemetry-to-jaeger` Exports traces to Jaeger. diff --git a/lib/logging/index.js b/lib/logging/index.js index 08840201..39cfdb1e 100644 --- a/lib/logging/index.js +++ b/lib/logging/index.js @@ -3,7 +3,7 @@ const LOG = cds.log('telemetry') const { getStringFromEnv } = require('@opentelemetry/core') -const { getCredsForCLSAsUPS, augmentCLCreds, _require } = require('../utils') +const { getCredsForCLSAsUPS, augmentCLCreds, augmentCaaSCreds, _require } = require('../utils') const _protocol2module = { grpc: '@opentelemetry/exporter-logs-otlp-grpc', @@ -33,7 +33,13 @@ function _getExporter() { } // use _require for better error message - const loggingExporterModule = _require(loggingExporter.module) + let loggingExporterModule + try { + loggingExporterModule = _require(loggingExporter.module) + } catch { + LOG._warn && LOG.warn(`Logs exporter module '${loggingExporter.module}' not found. Logging telemetry will be disabled.`) + return null + } if (!loggingExporterModule[loggingExporter.class]) throw new Error(`Unknown logs exporter "${loggingExporter.class}" in module "${loggingExporter.module}"`) const config = { ...(loggingExporter.config || {}) } @@ -46,6 +52,26 @@ function _getExporter() { config.credentials ??= credentials.credentials } + if (kind === 'telemetry-to-caas') { + if (!credentials) throw new Error('No CaaS credentials found.') + + augmentCaaSCreds(credentials) + + if (!credentials.httpAgentOptions) { + throw new Error('CaaS requires mTLS. Bind zero-trust-identity service or configure x509 credentials.') + } + + const exporterConfig = { + ...config, + url: credentials.baseUrl + '/v1/logs', + httpAgentOptions: credentials.httpAgentOptions + } + + const exporter = new loggingExporterModule[loggingExporter.class](exporterConfig) + LOG._debug && LOG.debug('Using logs exporter:', exporter) + return exporter + } + const exporter = new loggingExporterModule[loggingExporter.class](config) LOG._debug && LOG.debug('Using logs exporter:', exporter) @@ -76,9 +102,44 @@ module.exports = resource => { const { logs, SeverityNumber } = require('@opentelemetry/api-logs') const { LoggerProvider, BatchLogRecordProcessor, SimpleLogRecordProcessor } = require('@opentelemetry/sdk-logs') + /* + * create processor + */ + const exporter = _getExporter() + if (!exporter) return null + const processor = + _getCustomProcessor(exporter) || + (process.env.NODE_ENV === 'production' + ? new BatchLogRecordProcessor({ exporter }) + : new SimpleLogRecordProcessor({ exporter })) + + /* + * either add processor as delegate in CALM... + */ + if (!resource) { + LOG.warn("@sap/xotel-agent-ext-js found, adding @cap-js/telemetry's log processor as delegate") + try { + const { getCompositeLogRecordProcessor } = require('@sap/xotel-agent-ext-js') + getCompositeLogRecordProcessor().addDelegate(processor) + return + } catch (error) { + LOG.error('Failed to add log processor as delegate:', error) + throw error + } + } + + /* + * ... or initialize and return provider + */ + const loggerProvider = new LoggerProvider({ resource, processors: [processor] }) + logs.setGlobalLoggerProvider(loggerProvider) + // setup logs interception via cds.log.format - cds.on('served', () => { - const loggerProvider = logs.getLoggerProvider() + // Must be done AFTER LoggerProvider is set globally. + let _logInterceptionSetup = false + const setupLogInterception = () => { + if (_logInterceptionSetup) return + _logInterceptionSetup = true const loggers = {} const l2s = { 1: 'ERROR', 2: 'WARN', 3: 'INFO', 4: 'DEBUG', 5: 'TRACE' } @@ -125,37 +186,9 @@ module.exports = resource => { // replace format function of existing loggers for (const each in cds.log.loggers) cds.log.loggers[each].setFormat(format) - }) - - /* - * create processor - */ - const exporter = _getExporter() - const processor = - _getCustomProcessor(exporter) || - (process.env.NODE_ENV === 'production' - ? new BatchLogRecordProcessor(exporter) - : new SimpleLogRecordProcessor(exporter)) - - /* - * either add processor as delegate in CALM... - */ - if (!resource) { - LOG.warn("@sap/xotel-agent-ext-js found, adding @cap-js/telemetry's log processor as delegate") - try { - const { getCompositeLogRecordProcessor } = require('@sap/xotel-agent-ext-js') - getCompositeLogRecordProcessor().addDelegate(processor) - return - } catch (error) { - LOG.error('Failed to add log processor as delegate:', error) - throw error - } } - /* - * ... or initialize and return provider - */ - const loggerProvider = new LoggerProvider({ resource, processors: [processor] }) - logs.setGlobalLoggerProvider(loggerProvider) + cds.on('served', setupLogInterception) + return loggerProvider } diff --git a/lib/metrics/index.js b/lib/metrics/index.js index bd9be759..09e2fddc 100644 --- a/lib/metrics/index.js +++ b/lib/metrics/index.js @@ -6,7 +6,7 @@ const { getStringFromEnv } = require('@opentelemetry/core') const { resourceFromAttributes } = require('@opentelemetry/resources') const { AggregationTemporality, MeterProvider, PeriodicExportingMetricReader } = require('@opentelemetry/sdk-metrics') -const { getDynatraceMetadata, getCredsForDTAsUPS, getCredsForCLSAsUPS, augmentCLCreds, _require } = require('../utils') +const { getDynatraceMetadata, getCredsForDTAsUPS, getCredsForCLSAsUPS, augmentCLCreds, augmentCaaSCreds, _require } = require('../utils') const _protocol2module = { grpc: '@opentelemetry/exporter-metrics-otlp-grpc', @@ -74,6 +74,26 @@ function _getExporter() { config.credentials ??= credentials.credentials } + if (kind === 'telemetry-to-caas') { + if (!credentials) throw new Error('No CaaS credentials found.') + + augmentCaaSCreds(credentials) + + if (!credentials.httpAgentOptions) { + throw new Error('CaaS requires mTLS. Bind zero-trust-identity service or configure x509 credentials.') + } + + const exporterConfig = { + ...config, + url: credentials.baseUrl + '/v1/metrics', + httpAgentOptions: credentials.httpAgentOptions + } + + const exporter = new metricsExporterModule[metricsExporter.class](exporterConfig) + LOG._debug && LOG.debug('Using metrics exporter:', exporter) + return exporter + } + const exporter = new metricsExporterModule[metricsExporter.class](config) LOG._debug && LOG.debug('Using metrics exporter:', exporter) return exporter diff --git a/lib/tracing/index.js b/lib/tracing/index.js index 8a887c3a..58a3295c 100644 --- a/lib/tracing/index.js +++ b/lib/tracing/index.js @@ -11,6 +11,7 @@ const { getCredsForDTAsUPS, getCredsForCLSAsUPS, augmentCLCreds, + augmentCaaSCreds, hasDependency, _require } = require('../utils') @@ -125,6 +126,26 @@ function _getExporter() { config.credentials ??= credentials.credentials } + if (kind === 'telemetry-to-caas') { + if (!credentials) throw new Error('No CaaS credentials found.') + + augmentCaaSCreds(credentials) + + if (!credentials.httpAgentOptions) { + throw new Error('CaaS requires mTLS. Bind zero-trust-identity service or configure x509 credentials.') + } + + const exporterConfig = { + ...config, + url: credentials.baseUrl + '/v1/traces', + httpAgentOptions: credentials.httpAgentOptions + } + + const exporter = new tracingExporterModule[tracingExporter.class](exporterConfig) + LOG._debug && LOG.debug('Using trace exporter:', exporter) + return exporter + } + const exporter = new tracingExporterModule[tracingExporter.class](config) LOG._debug && LOG.debug('Using trace exporter:', exporter) diff --git a/lib/utils.js b/lib/utils.js index f72c814e..b2d6a054 100644 --- a/lib/utils.js +++ b/lib/utils.js @@ -2,6 +2,7 @@ const cds = require('@sap/cds') const LOG = cds.log('telemetry') const fs = require('fs') +const { initializeZTI, isZTIEnabled, getCert, getKey } = require('./zti') const { DiagLogLevel } = require('@opentelemetry/api') const { hrTimeToMilliseconds, getStringFromEnv } = require('@opentelemetry/core') @@ -137,6 +138,70 @@ function getCredsForCLSAsUPS() { } } +// Returns httpAgentOptions factory for ZTI, or null if not configured +function createZTIAgentFactory() { + if (!isZTIEnabled()) return null + if (!initializeZTI()) return null + + const https = require('https') + return () => new https.Agent({ + cert: getCert(), + key: getKey(), + keepAlive: true + }) +} + +// Returns httpAgentOptions factory for static x509 credentials, or null if not configured +function createStaticAgentFactory() { + const { x509 } = cds.env.requires.telemetry || {} + if (!x509?.cert || !x509?.key) return null + + let cert = x509.cert + let key = x509.key + if (cert.startsWith('LS0t') || !cert.startsWith('-----BEGIN')) { + cert = Buffer.from(cert, 'base64').toString('utf-8') + key = Buffer.from(key, 'base64').toString('utf-8') + } + + const https = require('https') + return () => new https.Agent({ cert, key, keepAlive: true }) +} + +function decodeCredentials(creds) { + if (creds.cert.startsWith('LS0t') || !creds.cert.startsWith('-----BEGIN')) { + return { + cert: Buffer.from(creds.cert, 'base64').toString('utf-8'), + key: Buffer.from(creds.key, 'base64').toString('utf-8') + } + } + return { cert: creds.cert, key: creds.key } +} + +function augmentCaaSCreds(credentials) { + if (credentials._augmented) return + credentials._augmented = true + + if (!credentials.otlp?.http) { + throw new Error('No OTLP HTTP endpoint in CaaS credentials') + } + + credentials.baseUrl = credentials.otlp.http + + const ztiAgentFactory = createZTIAgentFactory() + if (ztiAgentFactory) { + credentials.httpAgentOptions = ztiAgentFactory + return + } + + const staticAgentFactory = createStaticAgentFactory() + if (staticAgentFactory) { + credentials.httpAgentOptions = staticAgentFactory + return + } + + LOG._warn && LOG.warn('CaaS mTLS credentials not found') +} + function augmentCLCreds(credentials) { if (credentials._augmented) return credentials._augmented = true @@ -203,6 +268,10 @@ module.exports = { getCredsForDTAsUPS, getCredsForCLSAsUPS, augmentCLCreds, + augmentCaaSCreds, + createZTIAgentFactory, + createStaticAgentFactory, + decodeCredentials, hasDependency, _hrnow, _require diff --git a/lib/zti.js b/lib/zti.js new file mode 100644 index 00000000..de0cf7e1 --- /dev/null +++ b/lib/zti.js @@ -0,0 +1,101 @@ +const cds = require('@sap/cds') +const LOG = cds.log('telemetry') +const fs = require('fs') + +const SVID_DIR = '/home/vcap/app/spire-svids' + +let _paths = null +let _cached = null + +function _getSVIDCertificate() { + if (!_paths) throw new Error('ZTI paths not initialized') + + let stat + try { + stat = fs.statSync(_paths.cert) + } catch (e) { + // Transient stat failure during atomic rename - serve last-known-good + if (_cached) { + LOG._debug && LOG.debug('Stat failure, serving cached credentials') + return _cached + } + throw e + } + + const mtime = stat.mtimeMs + if (_cached?.mtime === mtime) return _cached + + // mtime changed or first load - reload all files atomically + try { + _cached = { + cert: fs.readFileSync(_paths.cert, 'utf8'), + key: fs.readFileSync(_paths.key, 'utf8'), + bundle: fs.readFileSync(_paths.bundle, 'utf8'), + mtime + } + return _cached + } catch (err) { + // Read failure after successful stat - likely mid-rotation + if (_cached) { + LOG._warn && LOG.warn('Failed to reload SVID files, serving cached:', err) + return _cached + } + throw err + } +} + +function getZTIConfig() { + if (!process.env.VCAP_SERVICES) return null + + const vcap = JSON.parse(process.env.VCAP_SERVICES) + const zti = vcap['zero-trust-identity'] + if (!zti || zti.length === 0) return null + + const svidName = zti[0].credentials?.parameters?.['svid-store']?.file?.name + if (!svidName) { + LOG._warn && LOG.warn('zero-trust-identity binding missing svid-store.file.name') + return null + } + + const svidDir = process.env.CDS_REQUIRES_TELEMETRY_ZTI_DIR || SVID_DIR + return { svidDir, svidName } +} + +function initializeZTI() { + const config = getZTIConfig() + if (!config) return false + + _paths = { + cert: `${config.svidDir}/${config.svidName}.svid.pem`, + key: `${config.svidDir}/${config.svidName}.svid.key`, + bundle: `${config.svidDir}/${config.svidName}.bundle.pem` + } + return true +} + +function isZTIEnabled() { + return process.env.CDS_REQUIRES_TELEMETRY_USE_ZTI !== 'false' +} + +function getCert() { + return _getSVIDCertificate().cert +} + +function getKey() { + return _getSVIDCertificate().key +} + +// For testing +function _reset() { + _paths = null + _cached = null +} + +module.exports = { + getZTIConfig, + initializeZTI, + isZTIEnabled, + getCert, + getKey, + _reset +} diff --git a/package.json b/package.json index 08d883a4..98d52596 100644 --- a/package.json +++ b/package.json @@ -157,6 +157,30 @@ "metrics": { "exporter": "env" } + }, + "telemetry-to-caas": { + "vcap": { + "label": "caas-service" + }, + "mtls_service_pattern": "caas-mtls|caas-cert", + "tracing": { + "exporter": { + "module": "@opentelemetry/exporter-trace-otlp-proto", + "class": "OTLPTraceExporter" + } + }, + "metrics": { + "exporter": { + "module": "@opentelemetry/exporter-metrics-otlp-proto", + "class": "OTLPMetricExporter" + } + }, + "logging": { + "exporter": { + "module": "@opentelemetry/exporter-logs-otlp-proto", + "class": "OTLPLogExporter" + } + } } } } diff --git a/test/caas.test.js b/test/caas.test.js new file mode 100644 index 00000000..c6ecea8f --- /dev/null +++ b/test/caas.test.js @@ -0,0 +1,509 @@ +const cds = require('@sap/cds') +const fs = require('fs') +const os = require('os') +const path = require('path') + +// Mock VCAP_SERVICES for CaaS +const MOCK_CAAS_VCAP = { + 'caas-service': [{ + name: 'test-caas', + credentials: { + otlp: { + http: 'https://caas.example.com/otlp', + grpc: 'grpc://caas.example.com:4317' + } + } + }] +} + +// Mock VCAP_SERVICES with ZTI binding +const MOCK_ZTI_VCAP = { + 'caas-service': [{ + name: 'test-caas', + credentials: { + otlp: { + http: 'https://caas.example.com/otlp' + } + } + }], + 'zero-trust-identity': [{ + name: 'test-zti', + credentials: { + parameters: { + 'svid-store': { + file: { name: 'test-svid' } + } + } + } + }] +} + +describe('augmentCaaSCreds', () => { + let originalVcap + + beforeAll(() => { + originalVcap = process.env.VCAP_SERVICES + }) + + afterAll(() => { + if (originalVcap) process.env.VCAP_SERVICES = originalVcap + else delete process.env.VCAP_SERVICES + }) + + beforeEach(() => { + cds.env.requires = cds.env.requires || {} + cds.env.requires.telemetry = { + x509: { + cert: Buffer.from('-----BEGIN CERTIFICATE-----\ntest\n-----END CERTIFICATE-----').toString('base64'), + key: Buffer.from('-----BEGIN PRIVATE KEY-----\ntest\n-----END PRIVATE KEY-----').toString('base64') + } + } + delete require.cache[require.resolve('../lib/utils')] + delete require.cache[require.resolve('../lib/zti')] + }) + + test('sets baseUrl from otlp.http', () => { + process.env.VCAP_SERVICES = JSON.stringify(MOCK_CAAS_VCAP) + delete require.cache[require.resolve('../lib/utils')] + delete require.cache[require.resolve('../lib/zti')] + const { augmentCaaSCreds } = require('../lib/utils') + + const credentials = { + otlp: { + http: 'https://caas.example.com/otlp', + grpc: 'grpc://caas.example.com:4317' + } + } + + augmentCaaSCreds(credentials) + + expect(credentials.baseUrl).toBe('https://caas.example.com/otlp') + }) + + test('sets httpAgentOptions when mTLS credentials found', () => { + process.env.VCAP_SERVICES = JSON.stringify(MOCK_CAAS_VCAP) + delete require.cache[require.resolve('../lib/utils')] + delete require.cache[require.resolve('../lib/zti')] + const { augmentCaaSCreds } = require('../lib/utils') + + const credentials = { + otlp: { http: 'https://caas.example.com/otlp' } + } + + augmentCaaSCreds(credentials) + + expect(credentials.httpAgentOptions).toBeDefined() + expect(typeof credentials.httpAgentOptions).toBe('function') + }) + + test('throws when no OTLP endpoints', () => { + process.env.VCAP_SERVICES = JSON.stringify(MOCK_CAAS_VCAP) + delete require.cache[require.resolve('../lib/utils')] + delete require.cache[require.resolve('../lib/zti')] + const { augmentCaaSCreds } = require('../lib/utils') + + expect(() => augmentCaaSCreds({})).toThrow('No OTLP HTTP endpoint in CaaS credentials') + }) + + test('does not augment twice', () => { + process.env.VCAP_SERVICES = JSON.stringify(MOCK_CAAS_VCAP) + delete require.cache[require.resolve('../lib/utils')] + delete require.cache[require.resolve('../lib/zti')] + const { augmentCaaSCreds } = require('../lib/utils') + + const credentials = { + otlp: { http: 'https://caas.example.com/otlp' } + } + + augmentCaaSCreds(credentials) + const originalBaseUrl = credentials.baseUrl + + credentials.otlp.http = 'https://different.com' + augmentCaaSCreds(credentials) + + expect(credentials.baseUrl).toBe(originalBaseUrl) + }) + + test('no httpAgentOptions when mTLS credentials not found', () => { + cds.env.requires.telemetry = {} // No x509 credentials + process.env.VCAP_SERVICES = JSON.stringify(MOCK_CAAS_VCAP) + delete require.cache[require.resolve('../lib/utils')] + delete require.cache[require.resolve('../lib/zti')] + const { augmentCaaSCreds } = require('../lib/utils') + + const credentials = { + otlp: { http: 'https://caas.example.com/otlp' } + } + + augmentCaaSCreds(credentials) + + expect(credentials.httpAgentOptions).toBeUndefined() + }) +}) + +describe('ZTI SVID File Loading', () => { + let tmpDir + let svidDir + + beforeEach(() => { + tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'zti-test-')) + svidDir = path.join(tmpDir, 'spire-svids') + fs.mkdirSync(svidDir) + + // Set up ZTI environment + process.env.VCAP_SERVICES = JSON.stringify(MOCK_ZTI_VCAP) + process.env.CDS_REQUIRES_TELEMETRY_ZTI_DIR = svidDir + delete process.env.CDS_REQUIRES_TELEMETRY_USE_ZTI + + cds.env.requires = cds.env.requires || {} + cds.env.requires.telemetry = {} + }) + + afterEach(() => { + fs.rmSync(tmpDir, { recursive: true, force: true }) + delete process.env.VCAP_SERVICES + delete process.env.CDS_REQUIRES_TELEMETRY_ZTI_DIR + delete process.env.CDS_REQUIRES_TELEMETRY_USE_ZTI + }) + + test('createZTIAgentFactory returns factory when SVID files exist', () => { + // Create SVID files + fs.writeFileSync(path.join(svidDir, 'test-svid.svid.pem'), '-----BEGIN CERTIFICATE-----\ntest\n-----END CERTIFICATE-----') + fs.writeFileSync(path.join(svidDir, 'test-svid.svid.key'), '-----BEGIN PRIVATE KEY-----\ntest\n-----END PRIVATE KEY-----') + fs.writeFileSync(path.join(svidDir, 'test-svid.bundle.pem'), '-----BEGIN CERTIFICATE-----\nbundle\n-----END CERTIFICATE-----') + + let factory + jest.isolateModules(() => { + const { createZTIAgentFactory } = require('../lib/utils') + factory = createZTIAgentFactory() + }) + + expect(factory).not.toBeNull() + expect(typeof factory).toBe('function') + }) + + test('createZTIAgentFactory returns factory even when SVID files do not exist yet', () => { + // Don't create SVID files - but factory is still created + // The factory will throw when called if files don't exist + + let factory + jest.isolateModules(() => { + const { createZTIAgentFactory } = require('../lib/utils') + factory = createZTIAgentFactory() + }) + + // Factory is created (ZTI is configured via VCAP_SERVICES) + expect(factory).not.toBeNull() + expect(typeof factory).toBe('function') + // Calling it throws because files don't exist + expect(() => factory()).toThrow() + }) + + test('getCert/getKey reload when mtime changes', async () => { + const certPath = path.join(svidDir, 'test-svid.svid.pem') + const keyPath = path.join(svidDir, 'test-svid.svid.key') + const bundlePath = path.join(svidDir, 'test-svid.bundle.pem') + + // Write initial files + fs.writeFileSync(certPath, '-----BEGIN CERTIFICATE-----\nv1\n-----END CERTIFICATE-----') + fs.writeFileSync(keyPath, '-----BEGIN PRIVATE KEY-----\nv1\n-----END PRIVATE KEY-----') + fs.writeFileSync(bundlePath, '-----BEGIN CERTIFICATE-----\nv1\n-----END CERTIFICATE-----') + + let cert1, cert2 + await jest.isolateModulesAsync(async () => { + const { initializeZTI, getCert, _reset } = require('../lib/zti') + _reset() + initializeZTI() + + cert1 = getCert() + + // Wait to ensure mtime changes (filesystem mtime resolution can be ~1s on some systems) + await new Promise(resolve => setTimeout(resolve, 50)) + + // Update files (simulating ZTI rotation) + fs.writeFileSync(certPath, '-----BEGIN CERTIFICATE-----\nv2\n-----END CERTIFICATE-----') + fs.writeFileSync(keyPath, '-----BEGIN PRIVATE KEY-----\nv2\n-----END PRIVATE KEY-----') + + cert2 = getCert() + }) + + expect(cert1).toContain('v1') + expect(cert2).toContain('v2') + }) +}) + +describe('ZTI flag behavior', () => { + beforeEach(() => { + cds.env.requires = { telemetry: {} } + delete process.env.CDS_REQUIRES_TELEMETRY_USE_ZTI + delete require.cache[require.resolve('../lib/utils')] + delete require.cache[require.resolve('../lib/zti')] + }) + + afterEach(() => { + delete process.env.CDS_REQUIRES_TELEMETRY_USE_ZTI + delete process.env.VCAP_SERVICES + }) + + test('detects ZTI config from VCAP_SERVICES', () => { + process.env.VCAP_SERVICES = JSON.stringify(MOCK_ZTI_VCAP) + + jest.isolateModules(() => { + const { getZTIConfig } = require('../lib/zti') + const config = getZTIConfig() + + expect(config).not.toBeNull() + expect(config.svidName).toBe('test-svid') + expect(config.svidDir).toBe('/home/vcap/app/spire-svids') + }) + }) + + test('returns null when no ZTI binding', () => { + process.env.VCAP_SERVICES = JSON.stringify(MOCK_CAAS_VCAP) + + jest.isolateModules(() => { + const { getZTIConfig } = require('../lib/zti') + const config = getZTIConfig() + + expect(config).toBeNull() + }) + }) + + test('createStaticAgentFactory returns factory when x509 configured', () => { + process.env.CDS_REQUIRES_TELEMETRY_USE_ZTI = 'false' + process.env.VCAP_SERVICES = JSON.stringify(MOCK_ZTI_VCAP) + cds.env.requires.telemetry.x509 = { + cert: Buffer.from('-----BEGIN CERTIFICATE-----\nenvvar\n-----END CERTIFICATE-----').toString('base64'), + key: Buffer.from('-----BEGIN PRIVATE KEY-----\nenvvar\n-----END PRIVATE KEY-----').toString('base64') + } + + const { createStaticAgentFactory } = require('../lib/utils') + const factory = createStaticAgentFactory() + + expect(factory).toBeDefined() + expect(typeof factory).toBe('function') + }) + + test('augmentCaaSCreds sets httpAgentOptions factory from ZTI', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'zti-test-')) + const svidDir = path.join(tmpDir, 'spire-svids') + fs.mkdirSync(svidDir) + + process.env.VCAP_SERVICES = JSON.stringify(MOCK_ZTI_VCAP) + process.env.CDS_REQUIRES_TELEMETRY_ZTI_DIR = svidDir + + // Create SVID files + fs.writeFileSync(path.join(svidDir, 'test-svid.svid.pem'), '-----BEGIN CERTIFICATE-----\nzti-cert\n-----END CERTIFICATE-----') + fs.writeFileSync(path.join(svidDir, 'test-svid.svid.key'), '-----BEGIN PRIVATE KEY-----\nzti-key\n-----END PRIVATE KEY-----') + fs.writeFileSync(path.join(svidDir, 'test-svid.bundle.pem'), '-----BEGIN CERTIFICATE-----\nbundle\n-----END CERTIFICATE-----') + + jest.isolateModules(() => { + const { augmentCaaSCreds } = require('../lib/utils') + + const credentials = { + otlp: { http: 'https://caas.example.com/otlp' } + } + augmentCaaSCreds(credentials) + + expect(credentials.httpAgentOptions).toBeDefined() + // httpAgentOptions is now a factory function + expect(typeof credentials.httpAgentOptions).toBe('function') + }) + + // Cleanup + delete process.env.CDS_REQUIRES_TELEMETRY_ZTI_DIR + fs.rmSync(tmpDir, { recursive: true, force: true }) + }) + + test('augmentCaaSCreds sets httpAgentOptions factory from x509 config', () => { + process.env.VCAP_SERVICES = JSON.stringify(MOCK_CAAS_VCAP) + process.env.CDS_REQUIRES_TELEMETRY_USE_ZTI = 'false' + cds.env.requires.telemetry.x509 = { + cert: Buffer.from('-----BEGIN CERTIFICATE-----\nenvvar-cert\n-----END CERTIFICATE-----').toString('base64'), + key: Buffer.from('-----BEGIN PRIVATE KEY-----\nenvvar-key\n-----END PRIVATE KEY-----').toString('base64') + } + + // x509 fallback doesn't involve ZTI state + const { augmentCaaSCreds } = require('../lib/utils') + + const credentials = { + otlp: { http: 'https://caas.example.com/otlp' } + } + augmentCaaSCreds(credentials) + + expect(credentials.httpAgentOptions).toBeDefined() + // httpAgentOptions is now a factory function + expect(typeof credentials.httpAgentOptions).toBe('function') + }) +}) + +describe('ZTI Certificate Rotation', () => { + let tmpDir, svidDir + let originalEnv + + const CERT_V1 = '-----BEGIN CERTIFICATE-----\nCERT_VERSION_1\n-----END CERTIFICATE-----' + const KEY_V1 = '-----BEGIN PRIVATE KEY-----\nKEY_VERSION_1\n-----END PRIVATE KEY-----' + const BUNDLE_V1 = '-----BEGIN CERTIFICATE-----\nBUNDLE_V1\n-----END CERTIFICATE-----' + + const CERT_V2 = '-----BEGIN CERTIFICATE-----\nCERT_VERSION_2\n-----END CERTIFICATE-----' + const KEY_V2 = '-----BEGIN PRIVATE KEY-----\nKEY_VERSION_2\n-----END PRIVATE KEY-----' + const BUNDLE_V2 = '-----BEGIN CERTIFICATE-----\nBUNDLE_V2\n-----END CERTIFICATE-----' + + beforeAll(() => { + originalEnv = { ...process.env } + }) + + beforeEach(() => { + // Create temp directory for SVID files + tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'zti-rotation-test-')) + svidDir = path.join(tmpDir, 'spire-svids') + fs.mkdirSync(svidDir) + + // Set up environment + process.env.VCAP_SERVICES = JSON.stringify(MOCK_ZTI_VCAP) + process.env.CDS_REQUIRES_TELEMETRY_ZTI_DIR = svidDir + delete process.env.CDS_REQUIRES_TELEMETRY_USE_ZTI + + cds.env.requires = { telemetry: {} } + }) + + afterEach(() => { + fs.rmSync(tmpDir, { recursive: true, force: true }) + process.env = { ...originalEnv } + }) + + function writeSVIDFiles(cert, key, bundle) { + fs.writeFileSync(path.join(svidDir, 'test-svid.svid.pem'), cert) + fs.writeFileSync(path.join(svidDir, 'test-svid.svid.key'), key) + fs.writeFileSync(path.join(svidDir, 'test-svid.bundle.pem'), bundle) + } + + function touchWithNewMtime(filepath) { + // Ensure mtime changes (some filesystems have 1-second resolution) + const now = new Date() + now.setSeconds(now.getSeconds() + 2) + fs.utimesSync(filepath, now, now) + } + + test('returns initial certificate on first call', () => { + writeSVIDFiles(CERT_V1, KEY_V1, BUNDLE_V1) + + jest.isolateModules(() => { + const { initializeZTI, getCert, getKey } = require('../lib/zti') + initializeZTI() + + expect(getCert()).toBe(CERT_V1) + expect(getKey()).toBe(KEY_V1) + }) + }) + + test('returns cached certificate when mtime unchanged', () => { + writeSVIDFiles(CERT_V1, KEY_V1, BUNDLE_V1) + + jest.isolateModules(() => { + const { initializeZTI, getCert, _reset } = require('../lib/zti') + _reset() + initializeZTI() + + // First call - reads from disk + const cert1 = getCert() + + // Second call without any file changes - should return cached value + const cert2 = getCert() + + expect(cert1).toBe(CERT_V1) + expect(cert2).toBe(CERT_V1) // Still cached, same mtime + }) + }) + + test('reloads certificate when mtime changes (rotation)', () => { + writeSVIDFiles(CERT_V1, KEY_V1, BUNDLE_V1) + + jest.isolateModules(() => { + const { initializeZTI, getCert, getKey } = require('../lib/zti') + initializeZTI() + + // First call - reads V1 + expect(getCert()).toBe(CERT_V1) + expect(getKey()).toBe(KEY_V1) + + // Simulate certificate rotation: write new files with new mtime + writeSVIDFiles(CERT_V2, KEY_V2, BUNDLE_V2) + touchWithNewMtime(path.join(svidDir, 'test-svid.svid.pem')) + + // Second call - should detect mtime change and reload + expect(getCert()).toBe(CERT_V2) + expect(getKey()).toBe(KEY_V2) + }) + }) + + test('serves cached cert during transient read failure', () => { + writeSVIDFiles(CERT_V1, KEY_V1, BUNDLE_V1) + + jest.isolateModules(() => { + const { initializeZTI, getCert } = require('../lib/zti') + initializeZTI() + + // First call - cache V1 + expect(getCert()).toBe(CERT_V1) + + // Simulate mid-rotation: touch mtime but make key file unreadable + touchWithNewMtime(path.join(svidDir, 'test-svid.svid.pem')) + fs.unlinkSync(path.join(svidDir, 'test-svid.svid.key')) + + // Should serve cached V1 despite read failure + expect(getCert()).toBe(CERT_V1) + }) + }) + + test('throws when files missing and no cache', () => { + // Don't create SVID files + + jest.isolateModules(() => { + const { initializeZTI, getCert } = require('../lib/zti') + initializeZTI() + + expect(() => getCert()).toThrow() + }) + }) + + test('getCert/getKey return rotated values when mtime changes', () => { + writeSVIDFiles(CERT_V1, KEY_V1, BUNDLE_V1) + + jest.isolateModules(() => { + const { initializeZTI, getCert, getKey } = require('../lib/zti') + + initializeZTI() + + // First call - returns V1 + expect(getCert()).toBe(CERT_V1) + expect(getKey()).toBe(KEY_V1) + + // Simulate certificate rotation: write new files with new mtime + writeSVIDFiles(CERT_V2, KEY_V2, BUNDLE_V2) + touchWithNewMtime(path.join(svidDir, 'test-svid.svid.pem')) + + // Second call - returns V2 (rotation detected) + expect(getCert()).toBe(CERT_V2) + expect(getKey()).toBe(KEY_V2) + }) + }) + + test('createZTIAgentFactory returns sync factory that uses getCert/getKey', () => { + writeSVIDFiles(CERT_V1, KEY_V1, BUNDLE_V1) + + jest.isolateModules(() => { + const { createZTIAgentFactory } = require('../lib/utils') + + const factory = createZTIAgentFactory() + expect(factory).not.toBeNull() + expect(typeof factory).toBe('function') + + // Factory is sync and returns an Agent with current certs + const agent = factory() + expect(agent.options.cert).toBe(CERT_V1) + expect(agent.options.key).toBe(KEY_V1) + expect(agent.options.keepAlive).toBe(true) + }) + }) +}) diff --git a/test/tracing-zti.test.js b/test/tracing-zti.test.js new file mode 100644 index 00000000..2f20d326 --- /dev/null +++ b/test/tracing-zti.test.js @@ -0,0 +1,271 @@ +/** + * Integration tests for ZTI + Tracing with dynamic certificate rotation + * + * With dynamic https.Agent cert/key functions, certificates are loaded on-demand + * when new TCP connections are established. The mtime-based caching ensures + * minimal disk reads while supporting automatic certificate rotation. + */ + +const cds = require('@sap/cds') +const fs = require('fs') +const os = require('os') +const path = require('path') +const { trace } = require('@opentelemetry/api') + +// Mock VCAP_SERVICES with ZTI and CaaS +const MOCK_ZTI_VCAP = { + 'caas-service': [{ + name: 'test-caas', + credentials: { + otlp: { + http: 'https://caas.example.com/otlp' + } + } + }], + 'zero-trust-identity': [{ + name: 'test-zti', + credentials: { + parameters: { + 'svid-store': { + file: { name: 'test-svid' } + } + } + } + }] +} + +describe('Tracing with ZTI integration', () => { + let tmpDir + let svidDir + let originalEnv + + beforeAll(() => { + // Save original environment + originalEnv = { + VCAP_SERVICES: process.env.VCAP_SERVICES, + CDS_REQUIRES_TELEMETRY_ZTI_DIR: process.env.CDS_REQUIRES_TELEMETRY_ZTI_DIR, + CDS_REQUIRES_TELEMETRY_USE_ZTI: process.env.CDS_REQUIRES_TELEMETRY_USE_ZTI, + cds_requires_telemetry_kind: process.env.cds_requires_telemetry_kind, + cds_requires_telemetry_tracing_exporter: process.env.cds_requires_telemetry_tracing_exporter + } + }) + + afterAll(() => { + // Restore original environment + Object.entries(originalEnv).forEach(([key, value]) => { + if (value === undefined) delete process.env[key] + else process.env[key] = value + }) + }) + + beforeEach(() => { + // Create temp directory for SVID files + tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'zti-tracing-test-')) + svidDir = path.join(tmpDir, 'spire-svids') + fs.mkdirSync(svidDir) + + // Setup ZTI environment + process.env.VCAP_SERVICES = JSON.stringify(MOCK_ZTI_VCAP) + process.env.CDS_REQUIRES_TELEMETRY_ZTI_DIR = svidDir + delete process.env.CDS_REQUIRES_TELEMETRY_USE_ZTI + process.env.cds_requires_telemetry_kind = 'to-caas' + + // Clear module cache + delete require.cache[require.resolve('../lib/utils')] + delete require.cache[require.resolve('../lib/zti')] + delete require.cache[require.resolve('../lib/index')] + delete require.cache[require.resolve('../lib/tracing')] + delete require.cache[require.resolve('../lib/metrics')] + delete require.cache[require.resolve('../lib/logging')] + }) + + afterEach(() => { + // Cleanup + if (tmpDir && fs.existsSync(tmpDir)) { + fs.rmSync(tmpDir, { recursive: true, force: true }) + } + delete process.env.VCAP_SERVICES + delete process.env.CDS_REQUIRES_TELEMETRY_ZTI_DIR + delete process.env.CDS_REQUIRES_TELEMETRY_USE_ZTI + delete process.env.cds_requires_telemetry_kind + delete process.env.cds_requires_telemetry_tracing_exporter + }) + + test('TracerProvider is registered with ZTI credentials', () => { + // Create SVID files + fs.writeFileSync(path.join(svidDir, 'test-svid.svid.pem'), '-----BEGIN CERTIFICATE-----\ntest\n-----END CERTIFICATE-----') + fs.writeFileSync(path.join(svidDir, 'test-svid.svid.key'), '-----BEGIN PRIVATE KEY-----\ntest\n-----END PRIVATE KEY-----') + fs.writeFileSync(path.join(svidDir, 'test-svid.bundle.pem'), '-----BEGIN CERTIFICATE-----\nbundle\n-----END CERTIFICATE-----') + + cds.env.requires = { + telemetry: { + kind: 'to-caas', + credentials: { + otlp: { + http: 'https://caas.example.com/otlp' + } + }, + tracing: { + exporter: { + module: '@opentelemetry/sdk-trace-base', + class: 'InMemorySpanExporter' + }, + sampler: { + kind: 'AlwaysOnSampler' + }, + propagators: [] + }, + instrumentations: {} + } + } + + // Setup telemetry + const setup = require('../lib/index') + setup() + + // Verify we can get a tracer (not NoopTracer) + const tracer = trace.getTracer('test') + expect(tracer).toBeDefined() + + // Create a span and verify it's a real span (NoopTracer returns zeros for traceId) + const span = tracer.startSpan('test-span') + expect(span.spanContext().traceId).toBeDefined() + expect(span.spanContext().traceId).not.toBe('00000000000000000000000000000000') + span.end() + }) + + test('throws when mTLS credentials are missing (no ZTI files, no x509)', () => { + // Remove ZTI binding from VCAP_SERVICES (so no ZTI agent) + process.env.VCAP_SERVICES = JSON.stringify({ + 'caas-service': [{ + name: 'test-caas', + credentials: { + otlp: { + http: 'https://caas.example.com/otlp' + } + } + }] + }) + + // No x509 credentials either + cds.env.requires = { + telemetry: { + kind: 'telemetry-to-caas', + credentials: { + otlp: { + http: 'https://caas.example.com/otlp' + } + }, + tracing: { + exporter: { + module: '@opentelemetry/sdk-trace-base', + class: 'InMemorySpanExporter' + }, + sampler: { + kind: 'AlwaysOnSampler' + }, + propagators: [] + }, + instrumentations: {} + } + } + + delete require.cache[require.resolve('../lib/zti')] + + // Setup should throw because mTLS is required but no credentials available + const setup = require('../lib/index') + expect(() => setup()).toThrow('CaaS requires mTLS') + }) + + test('standard flow works without ZTI binding using x509 credentials', () => { + // Remove ZTI from VCAP_SERVICES + process.env.VCAP_SERVICES = JSON.stringify({ + 'caas-service': [{ + name: 'test-caas', + credentials: { + otlp: { + http: 'https://caas.example.com/otlp' + } + } + }] + }) + + cds.env.requires = { + telemetry: { + kind: 'to-caas', + credentials: { + otlp: { + http: 'https://caas.example.com/otlp' + } + }, + x509: { + cert: Buffer.from('-----BEGIN CERTIFICATE-----\ntest\n-----END CERTIFICATE-----').toString('base64'), + key: Buffer.from('-----BEGIN PRIVATE KEY-----\ntest\n-----END PRIVATE KEY-----').toString('base64') + }, + tracing: { + exporter: { + module: '@opentelemetry/sdk-trace-base', + class: 'InMemorySpanExporter' + }, + sampler: { + kind: 'AlwaysOnSampler' + }, + propagators: [] + }, + instrumentations: {} + } + } + + delete require.cache[require.resolve('../lib/zti')] + + // Standard setup should work + const setup = require('../lib/index') + setup() + + const tracer = trace.getTracer('test') + const span = tracer.startSpan('test-span') + expect(span.spanContext().traceId).toBeDefined() + span.end() + }) + + test('x509 env var credentials work when USE_ZTI=false', () => { + process.env.CDS_REQUIRES_TELEMETRY_USE_ZTI = 'false' + + delete require.cache[require.resolve('../lib/zti')] + delete require.cache[require.resolve('../lib/utils')] + + cds.env.requires = { + telemetry: { + kind: 'to-caas', + credentials: { + otlp: { + http: 'https://caas.example.com/otlp' + } + }, + x509: { + cert: Buffer.from('-----BEGIN CERTIFICATE-----\nenvvar\n-----END CERTIFICATE-----').toString('base64'), + key: Buffer.from('-----BEGIN PRIVATE KEY-----\nenvvar\n-----END PRIVATE KEY-----').toString('base64') + }, + tracing: { + exporter: { + module: '@opentelemetry/sdk-trace-base', + class: 'InMemorySpanExporter' + }, + sampler: { + kind: 'AlwaysOnSampler' + }, + propagators: [] + }, + instrumentations: {} + } + } + + const setup = require('../lib/index') + setup() + + const tracer = trace.getTracer('test') + const span = tracer.startSpan('test-span') + expect(span.spanContext().traceId).toBeDefined() + span.end() + }) +})