Skip to content

Releases: karafka/karafka-core

v2.6.2

29 Jun 18:45
374d8fc

Choose a tag to compare

  • [Enhancement] Document that a leaf's default value is intentionally shared by reference across the class template and every config instance produced by Configurable::Node#deep_dup. This uniform rule (the leaf is shallow-copied) is what lets a shared service object passed as a default (e.g. a logger) keep its identity across all configs; the flip side is that an in-place mutation of a mutable container default (config.list << :x) is visible on every instance. Callers that need a per-instance mutable default should assign it inside a configure block or dup it themselves rather than relying on a mutable default: (e.g. default: []). Adds characterization tests covering the shared-default behavior.
  • [Fix] require "pathname" explicitly in lib/karafka-core.rb (with the other top-level requires). Karafka::Core.gem_root returns a Pathname, but the gem never required pathname -- it only worked because Bundler (or another gem) happened to load it first. In an environment where nothing else loads it, gem_root raised NameError: uninitialized constant Pathname.
  • [Enhancement] Document the virtual rule result contract: a rule must return a freshly built Array of [path, message] error pairs on every call. Contract#call takes ownership of that array and prepends the current scope onto each pair in place (avoiding a per-error allocation), so returning a memoized, shared or frozen array is unsupported -- the in-place scoping would accumulate the scope prefix across validations or raise FrozenError. Adds characterization tests for the supported and unsupported patterns.
  • [Fix] Configurable::Node#register raises the documented "already registered" ArgumentError for a name already used by an unread lazy-with-constructor setting. The duplicate guard only checked @configs_refs, but a lazy setting with a constructor is absent from it until first read, so register silently overwrote it; it now checks the defined children.
  • [Fix] Contractable::Contract.nested now pops its path in an ensure. If the block raised while the contract was being defined and the caller rescued it, the path stayed on the nesting stack and was prefixed onto every rule defined afterwards.
  • [Fix] Contract#call no longer raises NoMethodError when validating a non-Hash root with a 1-key or 2-key rule path; it reports the path as missing, consistent with the 3+-key path (and the non-Hash intermediate handling added in 2.6.1).
  • [Fix] Honor excluded_keys containing "cgrp" in StatisticsDecorator only_keys mode. The cgrp branch of the structure-aware fast path lacked the exclusion guard that the brokers and topics branches have, so excluding the consumer-group subtree still decorated it (inconsistent with the full-decoration path).
  • [Fix] Guard the patched rdkafka error callback against a null client pointer. librdkafka can invoke the error callback with a NULL rd_kafka_t (e.g. very early in client construction); calling rd_kafka_name on it dereferenced the null pointer and could segfault the process. Mirrors the upstream ErrorCallback.
  • [Fix] Resolve fatal errors in the patched rdkafka error callback. ERR__FATAL is only a generic marker, so the callback now fetches the real underlying error code and description via RdkafkaError.build_fatal (rd_kafka_fatal_error) instead of reporting the generic fatal code. Mirrors the upstream ErrorCallback.
  • [Fix] A lazy setting declared without a constructor (setting(:x, lazy: true)) no longer raises when its accessor is read. lazy: true only makes sense together with a constructor to (re)evaluate; without one there is nothing to evaluate, so such a setting now behaves like a regular setting backed by its default. Previously reading it raised NoMethodErrornil.arity through the dynamic accessor for a falsy default, or a missing accessor for a truthy default.
  • [Fix] Contract#call no longer raises NoMethodError when a virtual rule returns false. A virtual rule now signals "no errors" with any non-Array result (true/false/nil); only an Array of error pairs is collected. Previously a false return reached false.each (a nil return was already tolerated).
  • [Fix] Manage CallbacksManager callbacks copy-on-write: add/delete rebuild and atomically swap an immutable values snapshot under a mutex, and #call iterates that snapshot directly. The previous values cache (introduced in 2.5.11) was lazily invalidated from within #call, which could not be done atomically against a concurrent add/delete, so a callback racing with dispatch could be silently lost — a removed one kept firing forever, or a newly added one never fired. The copy-on-write read takes no lock and allocates nothing per call, so the race is fixed without reintroducing a per-call #values allocation. librdkafka statistics/error callbacks fire from a background thread while callbacks are registered/unregistered, so this was reachable in practice.
  • [Fix] Manage Notifications subscriptions copy-on-write (#subscribe, #unsubscribe and #clear replace the per-event listener array instead of mutating it in place) so a listener that unsubscribes itself (or another) from within its own handler no longer causes the listener following it to be silently skipped, and concurrent subscribe/unsubscribe during dispatch is safe. Dispatch keeps iterating the live array directly, so there is no per-notification allocation on the hot path.
  • [Fix] Report a freeze duration (_fd) of 0 for statistics keys that are newly introduced in an emission (e.g. a broker or partition that appears mid-stream) instead of the elapsed time since the previous emission. A key that did not exist in the prior emission could not have been "frozen" for any duration, so accumulating the inter-emission gap was incorrect and also made the related StatisticsDecorator spec flaky on slow CIs (_fd depended on the wall-clock gap between the two emissions).
  • [Fix] Make assigning a setting on a frozen Configurable::Node atomic. The ivar-backed writer evaluated @configs_refs[name] = value before instance_variable_set, so a frozen node mutated the canonical store and only then raised FrozenError, leaving the store and the ivar-backed reader permanently out of sync. It now raises before touching any state.
  • [Fix] Configurable::Node#to_h now evaluates a setting's constructor with its default (arity-aware, matching #compile) instead of calling it with no arguments. The documented ->(default) { ... } constructor form previously raised ArgumentError: wrong number of arguments from #to_h whenever the value was not yet in the config store (e.g. #to_h on an unconfigured instance, or an unread lazy setting).
  • [Fix] Honor excluded_keys inside StatisticsDecorator only_keys decoration. A key listed in both only_keys and excluded_keys was still decorated because the direct-access decoration loop never consulted excluded_keys; exclusion now wins, matching the full-decoration path.
  • [Fix] Strip the tests/specs root directory as an anchored prefix (sub(/\A.../)) instead of a global gsub in MinitestLocator and RSpecLocator. When the root directory string recurred later in a test/spec file path, the global replace removed every occurrence and corrupted the derived subject class path; only the leading prefix is now removed.

v2.6.1

15 Jun 15:21
a65b58c

Choose a tag to compare

  • [Enhancement] Speed up Contract#call by ~1.25x for minimal and ~1.4x for fully populated data: resolve rule paths with a single Hash#fetch per level instead of key? + [], inline the per-rule type dispatch into the rules loop, and compare the dig sentinel via #equal? so #== is never dispatched to the validated (user-provided) values. This is the per-message validation path in WaterDrop producers.
  • [Fix] Contract#call with rule paths of 3+ keys no longer raises NoMethodError when an intermediate value is not a Hash and reports the path as missing instead, consistent with the 2-key path behavior.
  • [Change] Reject reserved setting names with an ArgumentError in Configurable::Node#setting and #register: internal state names (node_name, children, nestings, compiled, configs_refs, local_defs) and the node public API names (setting, configure, to_h, deep_dup, register, compile). Previously such names silently shadowed the node own accessors, breaking deep_dup or to_h, and assignments like config.children = value corrupted the node internal state.
  • [Enhancement] Skip the event name mapping hash lookup in Monitor#instrument when no namespace is used and the event id is already a String, which is the case for all events in the Karafka ecosystem (~1.2x faster dispatch on the common no-subscribers path). Symbol event ids and namespaced monitors keep going through the mapping.
  • [Enhancement] Mirror config values into instance variables and use attr_reader based readers in Configurable::Node, yielding ~1.4x faster flat and ~1.6x faster nested settings reads on hot paths. @configs_refs remains the canonical store; non-identifier setting names (e.g. registered names with dashes) keep the previous hash-based accessors.
  • [Enhancement] Instantiate each Configurable::Node through a per-layout anonymous subclass so the ivar-backed settings do not grow object shape variations on the shared Node class (which would degrade ivar access and trigger Ruby performance warnings). deep_dup reuses the template's subclass, so duplicated configs share object shapes.
  • [Fix] Symbolize setting names at definition time (setting, same as register) and on config store writes so String setting names work end to end (accessors, #to_h, recompilation state) and cannot corrupt node internal state when matching reserved internal names (previously string-named settings were quietly broken as accessors and the store disagreed on the key type).
  • [Change] Config nodes are now instances of anonymous Node subclasses: is_a?(Karafka::Core::Configurable::Node) still holds, but instance_of?(Node) is now false and node.class.name is nil.
  • [Change] Assigning a setting on a frozen config node now raises FrozenError (previously the write silently mutated internal storage despite the freeze).

v2.6.0

10 Jun 16:05
8d73cc2

Choose a tag to compare

  • [Enhancement] Add Node#register to allow runtime key-value registration on compiled nodes without going through the static setting DSL. Useful for dynamic registries (e.g. named clusters) where setting names are not known at class-load time.
  • [Enhancement] Replace version-gated Warning[:performance] with a Warning.categories-based loop that enables all opt-in Ruby warning categories automatically, picking up new categories (e.g. strict_unused_block in Ruby 3.4+) without future patches.

v2.5.13

08 Apr 09:34
4ce84e0

Choose a tag to compare

  • [Enhancement] Extract decorate_partitions method from StatisticsDecorator to allow subclasses to filter which partitions are decorated (e.g. skip unassigned partitions in a consumer context).

v2.5.12

02 Apr 17:29
5384f09

Choose a tag to compare

  • [Fix] Exclude test/ directory from gem releases to reduce package size.

v2.5.11

02 Apr 10:32
035ab48

Choose a tag to compare

  • [Enhancement] Specialize Contract#dig for common 1-key and 2-key paths to avoid iterator overhead, yielding ~1.5x faster single-key lookups and ~1.45x faster two-key nested lookups.
  • [Enhancement] Replace Node#build_accessors @local_defs Array with Hash for O(1) membership checks instead of O(n) Array#include?, yielding up to ~5x faster accessor lookups at 50 settings.
  • [Enhancement] Use frozen EMPTY_ARRAY constant for Contract#call and #validate! default scope parameter to avoid allocating a new Array on every invocation, yielding ~1.36x faster call dispatch and saving 1 Array allocation per call.
  • [Enhancement] Pre-resolve @events_methods_map method name before the listener notification loop in Notifications#instrument to avoid repeated Hash lookup per listener, yielding ~1.12x faster event dispatch with multiple listeners.
  • [Enhancement] Cache a frozen success Result singleton via Result.success to eliminate 1 object allocation per successful Contract#call on the happy path.
  • [Enhancement] Skip nestings block re-evaluation in Node#deep_dup to avoid recreating children that are immediately overwritten, yielding ~14x faster deep_dup and reducing allocations from ~620 to ~66 objects for large configs.
  • [Enhancement] Cache CallbacksManager#call values snapshot and invalidate on add/delete to avoid allocating a new Array on every invocation while preserving thread-safety snapshot semantics, saving 1 Array allocation per call.
  • [Enhancement] Store execution time separately in Event and build the merged payload hash lazily on #payload access, eliminating 1 Hash allocation per Notifications#instrument call when listeners use #[] access (the common pattern), yielding ~1.7x faster event dispatch.
  • [Enhancement] Replace StatisticsDecorator#diff pending-writes buffer with keys.each direct-write iteration, eliminating the buffer and write-back loop for ~13% faster decoration at scale (10 brokers, 20 topics, 2000 partitions).
  • [Enhancement] Reorder StatisticsDecorator#diff type checks to test Numeric before Hash, matching the ~80% numeric value distribution in librdkafka statistics.
  • [Enhancement] Support only_keys option in StatisticsDecorator to decorate only specified numeric keys (e.g. consumer_lag, committed_offset). When combined with excluded_keys, reduces decoration cost from ~80ms to ~8.5ms per call on large clusters (10 brokers, 20 topics, 2000 partitions) by using structure-aware navigation of the librdkafka statistics tree and direct key access instead of full-hash iteration.

v2.5.10

02 Mar 12:02
1b03a47

Choose a tag to compare

  • [Enhancement] Introduce MinitestLocator helper for minitest/spec subject class auto-discovery from test file paths.

v2.5.9

02 Mar 11:17
91dd614

Choose a tag to compare

  • [Enhancement] Optimize StatisticsDecorator to eliminate per-hash Array allocations by using each_pair with a per-call pending-writes buffer instead of current.keys.each, reducing allocations from tens of thousands to one per call at scale.
  • [Enhancement] Inline StatisticsDecorator#append and #suffix_keys_for into #diff to reduce method call overhead by ~96% (from ~915k to ~39k calls at 6400 partitions).
  • [Enhancement] Support excluded_keys option in StatisticsDecorator to skip decoration of unused librdkafka statistics subtrees (e.g. broker toppars, window stats), yielding ~28% additional speedup at scale.

v2.5.8

23 Nov 14:25
eb4a3ef

Choose a tag to compare

  • [Enhancement] Memoize StatisticsDecorator suffix keys to reduce string allocations (#268).

v2.5.7

28 Sep 15:19
3f4eb49

Choose a tag to compare

  • [Enhancement] Optimize Node#deep_dup to reduce array allocations during configuration deep copying.
  • [Enhancement] Optimize Result#initialize to use shared EMPTY_HASH constant for successful validations.
  • [Enhancement] Optimize StatisticsDecorator#append string concatenation to reduce allocations.
  • [Enhancement] Optimize Notifications#instrument payload handling to avoid unnecessary hash merges for empty payloads.
  • [Change] Remove Ruby 3.1 support due to EOL.