Proposed Enhancements for the Outbox Library

[Alek96]

First, I want to express my appreciation for the work you’ve done. You’ve built an excellent Spring Boot library for implementing the transactional outbox pattern, and it already covers many critical features for production-grade systems, including:

• Event Ordering
• Database-backed retry mechanisms
• Monitoring & Observability

I understand the library is still evolving, so I’d like to share a few improvement ideas that may be useful.

---

1. First-Attempt Online Delivery

It would be beneficial if the library attempted to send an event immediately after the transaction commits, using a background thread.

Benefits:

• Significantly reduces end-to-end latency.
• Still safe, as the scheduled partition-based workers remain responsible for retries and for picking up any messages the first attempt didn’t handle (e.g., application crash or executor saturation).

Key requirements:

• Before sending, verify that the event is the oldest pending one for its aggregate to preserve ordering.
• If the event cannot be sent immediately, the existing scheduler handles it.

---

2. Retry Configuration per Event Type

Different event types often have different delivery characteristics (e.g., HTTP POST calls to external services with varying response times).
Allowing retry policy configuration per event type, similar to how Resilience4j handles configs, would give users much finer control.

This could be implemented using Spring Boot properties.

---

3. Ordered vs. Non-Ordered Event Types

In many systems, some events must follow strict ordering, while others do not. Supporting both patterns would make the library more flexible.

One approach to achieve it:

• Introduce an order_date column.
• For ordered events: set order_date = created_at.
• For non-ordered events: set order_date = null.
• Then retrieval logic can use a unified SQL predicate that works for both kinds of events:

select o.aggregateId, min(o.createdAt) as minCreated
 from OutboxRecordEntity o
where o.partitionNo in :partitions
  and o.status = :status
  and o.nextRetryAt <= :now
  and not exists (
      select 1 from OutboxRecordEntity older
      where older.aggregateId = o.aggregateId
      and older.completedAt is null
      and older.order_date < o.order_date 
  )
group by o.aggregateId
order by minCreated asc

---

4. Trace Propagation

It would be valuable for the library to automatically propagate tracing information (e.g., traceId) similarly to how Spring Boot and Micrometer handle it.

Ideas:

• Store the originating traceId in the outbox record.
• Allow enabling/disabling tracing propagation per event type.

This feature greatly improves cross-service observability.

---

5. Customizable Table Name Prefix

Some teams use a shared database across multiple services (e.g., “distributed monolith”), and rely on table prefixes such as APP_X_ or APP_Y_ to avoid collisions.

It would be helpful if the library allowed specifying a prefix for all outbox tables, similar to how Quartz supports table prefix configuration.

---

6. Task Executor per Event Type

Spring supports executor selection for @Async via qualifiers.
Adding a similar capability to the outbox, letting different event types use different executors, would improve performance tuning and isolation, especially in high-throughput systems.

---

7. Retryable / Non-Retryable Exceptions per Event Type

In some cases, certain exceptions should cause an event to fail immediately rather than be retried.
For example, if the application lacks permission to publish to a specific message queue, retrying will not resolve the issue.

To improve flexibility and reduce unnecessary retries, it would be useful to support:

• A per-event-type list of retryable exceptions
• A per-event-type list of non-retryable exceptions that should fail the event instantly

---

If you'd like, I can help refine any of these ideas further or turn them into concrete design proposals or pull requests.

[rolandbeisel]

Thank you so much for taking the time to thoughtfully outline these improvement ideas! 🙏 Your analysis is excellent, and I appreciate the depth of consideration you've put into each proposal. It's clear you have production experience with event-driven systems, and your suggestions directly address real-world challenges.

I want to be transparent: the library is currently undergoing major architectural changes with the extension and changes of partitioning support. This is foundational work that needs to stabilize before layering these additional features on top.
The partitioning work is critical because it:

• Enables distributed processing across multiple instances
• Provides the foundation for scalability
• Changes how events are selected and distributed (impacts most proposals here)

Once this is solid, these feature proposals become much easier to implement cleanly.

I'd love to implement these features!

Even though I'm not in a position to accept contributions on these specific features right now, I want you to know:

• These ideas are tracked - I'm keeping them in the roadmap
• Design is mostly solid - Your proposals are well-thought-out; we won't need major rework
• I'm open to collaboration later - Once partitioning is stable, we can definitely work together on any of these

Thank you again for the thoughtful contribution. This is exactly the kind of feedback that helps guide a library's evolution. I'm genuinely excited about implementing these features once I've solidified the foundation.

[Alek96]

Over the past months, we reviewed each item and, in several cases, implemented a handler-oriented version of the proposal. Below is a final status update for each point.

1) First-Attempt Online Delivery (immediate publishing / post-commit emission)

Status: Not supported (intentionally).

We did consider an “emit immediately after transaction commit” path, but decided to keep the system strictly polling-driven.

Why: the current model is partition-owned and non-competitive:

• Records are stored in the same DB transaction as the business change.
• Each record is deterministically assigned to a partition.
• Each instance owns a subset of partitions.
• A single thread processes a given partition, so there is no competition.

This keeps processing simple and efficient (select + update, no distributed locks, predictable behavior).

Adding immediate delivery would introduce a second concurrent consumer (the post-commit async emitter) competing with the polling scheduler. Avoiding double processing would require additional “ownership/locking” mechanics (e.g., optimistic lock update, timeout management, extra DB round trips, edge cases around crashes and timeouts). Given that polling latency can be tuned and the current model is clean and scalable, we chose not to add this complexity at this time.

2) Retry Configuration per Event Type

Status: Supported, but handler-oriented (not event-type-oriented).

Instead of configuring retries “per event type”, the design is centered around the handler that processes a payload. This provides similar flexibility while keeping configuration close to the execution point.

Example:

@Component
class PaymentHandler {
    @OutboxHandler
    @OutboxRetryable(AggressiveRetryPolicy::class)
    fun handlePayment(payload: PaymentEvent) {
        paymentGateway.process(payload)
    }
}

3) Ordered vs. Non-Ordered Event Types

Status: Supported.

Ordering is enforced by the key. If you provide a key, records for that key are processed in-order. If you omit the key, the record is treated as non-ordered (i.e., not constrained by per-key sequencing).

Ordered:

outbox.schedule(
    payload = OrderCreatedEvent(order.id, order.customerId),
    key = "order-${order.id}"
)

Non-ordered:

outbox.schedule(
    payload = OrderCreatedEvent(order.id, order.customerId),
)

4) Trace Propagation

Status: Partially supported (context propagation supported; restoration is user-controlled).

The library supports capturing and propagating “context” across the async boundary (e.g., trace IDs, correlation IDs, tenant/user metadata). Context capture is supported via a custom OutboxContextProvider. Context restoration is currently done explicitly by the application within the handler, so users can integrate with their specific tracing setup.

5) Customizable Table Name Prefix

Status: Supported (JDBC module).

A configurable table prefix is available via: namastack.jdbc.table-prefix: my-prefix. This helps avoid collisions in shared-database setups.

6) Task Executor per Event Type

Status: Not supported.

The processing model is polling-based and designed so that each poll cycle completes before the next begins. For this reason, processing runs on a single executor to preserve the scheduler’s sequencing and operational predictability. Per-event/handler executors were not added.

7) Retryable / Non-Retryable Exceptions per Event Type

Status: Supported, but handler-oriented.

Rather than being event-type-based, retryability is configured via the handler’s retry policy. Users can define which exceptions are retryable vs. fail-fast in the policy used by @OutboxRetryable (same mechanism as in item 2).

---

Closing this discussion since each item has been evaluated and either implemented (sometimes with a handler-based shape) or intentionally deferred to keep the core polling/partition design simple and robust.

[rolandbeisel]

Thanks for the summary @Alek96 !