waterdrop

2.8.2last stable release 1 month ago
Complexity Score
Low
Open Issues
5
Dependent Projects
7
Weekly Downloadsglobal
233

Downloads

Loading Weekly Download Data

Readme

WaterDrop

WaterDrop is a standalone gem that sends messages to Kafka easily with an extra validation layer. It is a part of the Karafka ecosystem.

It:

  • Is thread-safe
  • Supports sync and async producing
  • Supports transactions
  • Supports buffering
  • Supports producing to multiple clusters
  • Supports multiple delivery policies
  • Supports per-topic configuration alterations (variants)
  • Works with Kafka 1.0+ and Ruby 3.1+
  • Works with and without Karafka

Documentation

Karafka ecosystem components documentation, including WaterDrop, can be found here.

Getting Started

If you want to both produce and consume messages, please use Karafka. It integrates WaterDrop automatically.

To get started with WaterDrop:

  1. Add it to your Gemfile:
bundle add waterdrop
  1. Create and configure a producer:
producer = WaterDrop::Producer.new do |config|
  config.deliver = true
  config.kafka = {
    'bootstrap.servers': 'localhost:9092',
    'request.required.acks': 1
  }
end
  1. Use it as follows:
# sync producing
producer.produce_sync(topic: 'my-topic', payload: 'my message')

# or for async
producer.produce_async(topic: 'my-topic', payload: 'my message')

# or in sync batches
producer.produce_many_sync(
  [
    { topic: 'my-topic', payload: 'my message'},
    { topic: 'my-topic', payload: 'my message'}
  ]
)

# and async batches
producer.produce_many_async(
  [
    { topic: 'my-topic', payload: 'my message'},
    { topic: 'my-topic', payload: 'my message'}
  ]
)

# transactions
producer.transaction do
  producer.produce_async(topic: 'my-topic', payload: 'my message')
  producer.produce_async(topic: 'my-topic', payload: 'my message')
end

Dependencies

No runtime dependency information found for this package.

CVE IssuesActive
0
Scorecards Score
No Data
Test Coverage
No Data
Follows Semver
No
Github Stars
259
Dependenciestotal
3
DependenciesOutdated
0
DependenciesDeprecated
0
Threat Modelling
No
Repo Audits
No

Learn how to distribute waterdrop in your own private RubyGems registry

gem install waterdrop
Processing...
Done

105 Releases

RubyGems on Cloudsmith

Getting started with RubyGems on Cloudsmith is fast and easy.