Kafka

Kafka provides a unified, high-throughput, low-latency platform for handling real-time data feeds. Kafka can connect to external systems (for data import/export)

Official Websitehttps://kafka.js.org/

Documentationhttps://kafka.js.org/docs/getting-started

NodeJS packagehttps://www.npmjs.com/package/kafkajs

Version1.15.0

Source Codehttps://github.com/tulios/kafkajs

Tagsqueue, kafka

Network Connection needs

This integration needs network access to the server where the service is running.

See the Network access page for details about how to achieve that.

KafkaJS snippets available in YepCode editor

note

The title is the triggering text for YepCode to autocomplete the script

Integration

New integration from credential

const kafka = yepcode.integration.kafkajs('credential-slug')

New integration from plain authentication data

const { Kafka } = require('kafkajs')

const kafka = new Kafka({
  clientId: 'my-app',
  brokers: ['kafka1:9092', 'kafka2:9092'],
  ssl: true,
  sasl: {
    mechanism: 'plain',
    username: 'my-username',
    password: 'my-password'
  }
})

Producer

Producer

const producer = kafka.producer()

await producer.connect()
await producer.send({
  topic: 'test-topic',
  messages: [
    { value: 'Hello KafkaJS user!' },
  ],
})

await producer.disconnect()

Consumer

Consumer

const consumer = kafka.consumer({ groupId: 'test-group' })

await consumer.connect()
await consumer.subscribe({ topic: 'test-topic', fromBeginning: true })

await consumer.run({
  eachMessage: async ({ topic, partition, message }) => {
    console.log({
      value: message.value.toString(),
    })
  },
})