Skip to content

feat: Add @azure/functions-extensions-kafka package for KafkaRecord binding (#103) - #104

Open
Tsuyoshi Ushio (TsuyoshiUshio) wants to merge 3 commits into
Azure:mainfrom
TsuyoshiUshio:feature/kafka-record-extension
Open

feat: Add @azure/functions-extensions-kafka package for KafkaRecord binding (#103)#104
Tsuyoshi Ushio (TsuyoshiUshio) wants to merge 3 commits into
Azure:mainfrom
TsuyoshiUshio:feature/kafka-record-extension

Conversation

@TsuyoshiUshio

Copy link
Copy Markdown
Contributor

Summary

Add @azure/functions-extensions-kafka package that enables Node.js users to bind to raw Apache Kafka records (KafkaRecord type) with full metadata access via Protobuf deserialization.

Issue

Fixes #103
Relates to Azure/azure-functions-kafka-extension#612

Changes

New package: azure-functions-nodejs-extensions-kafka/

File Description
types/index.d.ts TypeScript interfaces: KafkaRecord, KafkaHeader, KafkaTimestamp, KafkaTimestampType
src/kafka/kafkaRecordDecoder.ts Protobuf wire format decoder using protobufjs/minimal
src/kafka/kafkaRecordFactory.ts ModelBindingDataKafkaRecord conversion (single + batch)
src/kafka/registerKafkaRecordFactory.ts Register "AzureKafkaRecord" factory with ResourceFactoryResolver
src/index.ts Auto-registers factory on import, re-exports types
test/kafkaRecordFactory.test.ts 8 tests: full record decode, null key/value, null leader epoch, null header value, single binding, batch binding, null content error, unknown timestamp type fallback

Follows the azure-functions-nodejs-extensions-servicebus pattern exactly.

No changes needed to azure-functions-nodejs-library or azure-functions-nodejs-worker.

Dependencies

  • @azure/functions-extensions-base 0.3.0
  • protobufjs ^7.4.0

Breaking Changes

None. New package — all existing Kafka bindings continue to work.

Testing

  KafkaRecordDecoder
    ✔ should decode a full record with all fields
    ✔ should handle null key and value
    ✔ should handle null leader epoch
    ✔ should handle null header value
  KafkaRecordFactory
    ✔ should build a KafkaRecord from single ModelBindingData
    ✔ should build KafkaRecord[] from batch ModelBindingData
    ✔ should throw when content is null
    ✔ should fallback unknown timestamp type to NotAvailable (0)

  8 passing (14ms)

User Experience

import { KafkaRecord } from '@azure/functions-extensions-kafka';

app.generic('KafkaTrigger', {
    trigger: { type: 'kafkaTrigger', topic: 'my-topic', brokerList: '%BrokerList%', consumerGroup: '$Default' },
    handler: async (record: KafkaRecord, context) => {
        context.log(`Topic: ${record.topic}, Partition: ${record.partition}`);
        const value = Buffer.from(record.value!).toString('utf-8');
    },
});

Tsuyoshi Ushio and others added 2 commits April 23, 2026 21:30
…inding

New package that enables Node.js users to bind to raw Apache Kafka records
with full metadata access via Protobuf deserialization.

New files:
- types/index.d.ts — KafkaRecord, KafkaHeader, KafkaTimestamp, KafkaTimestampType
- src/kafka/kafkaRecordDecoder.ts — Protobuf wire format decoder
- src/kafka/kafkaRecordFactory.ts — ModelBindingData → KafkaRecord conversion
- src/kafka/registerKafkaRecordFactory.ts — 'AzureKafkaRecord' factory registration
- src/index.ts — Auto-registers factory on import, exports types
- test/kafkaRecordFactory.test.ts — 8 tests (decoder + factory)
- package.json, tsconfig.json, webpack.config.js — Build configuration

Follows the ServiceBus extensions package pattern exactly.
No changes needed to nodejs-library or nodejs-worker.

Fixes Azure#103
Relates to Azure/azure-functions-kafka-extension#612

Co-authored-by: Dobby <dobby@microsoft.com>
…fka extension

- Add kafka_extension to CI build matrices (minimal-build.yml, build.yml)
- Add kafka_extension to test matrices (tests.yml) for Node 18/20/22/24
- Add sample app: kafkaTriggerSample with KafkaRecord binding example
- Add CI-required tooling: .eslintrc.json, updateVersion.ts, validateRelease.ts
- Fix lint errors: unbound method reference, tsconfig include paths
- Update package.json with lint/updateVersion scripts and devDependencies

Build: webpack compiled successfully
Tests: 8 passing
Lint: 0 errors

Co-authored-by: Dobby <dobby@microsoft.com>
@TsuyoshiUshio

Copy link
Copy Markdown
Contributor Author

E2E testing (Docker Compose + Kafka broker) is tracked separately in #105 due to the additional infrastructure complexity (Kafka KRaft broker setup, extension bundle dependency, topic auto-creation). This PR covers the package implementation, unit tests, CI build/lint/version integration, and sample app.

…fka sample

- Package README.md with installation, usage, type reference, and architecture
- Sample README.md with Prerequisites, Quick Start, configuration table,
  Confluent Cloud / Event Hubs auth examples, troubleshooting guide
- local.settings.json.example with BrokerList placeholder
- .funcignore for deployment filtering

Co-authored-by: Dobby <dobby@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Add @azure/functions-extensions-kafka package for KafkaRecord binding

1 participant