MT-Replica-Set

中文版

mt-replica-set is a high-performance matching engine replica set component based on ultra-fast zero-copy IPC. By physically decoupling Ingress, Matching, and Egress stages, it achieves sub-20 microsecond latency and high throughput.

🚀 Key Features

• Zero-Copy IPC: Powered by xchannel 2.0.0 using mmap-backed memory files, eliminating data copy overhead between processes.
• User-space Async IO: Built on monoio with io-uring support on Linux and FusionDriver on macOS.
• Core Affinity: Support for physical core binding to minimize context switching.
• Graceful Shutdown: Cascaded shutdown using Poison Pills (SBE Template ID 500) triggered by SIGINT.

🏗 Architecture

graph LR
    User([User/Client]) -- SBE/TCP --> Ingress
    Ingress -- xchannel --> InputJournal[(Input Journal)]
    InputJournal -- mmap --> Matching
    
    Matching -- ExecutionReport --> RB_OMS[(OMS Stream)]
    Matching -- PublicTrade/Depth --> RB_MD[(MD Stream)]
    
    RB_OMS -- mmap --> Egress_OMS[Egress OMS]
    RB_MD -- mmap --> Egress_MD[Egress MD]
    
    Egress_OMS -- SBE/TCP --> OMS([OMS Service])
    Egress_MD -- SBE/TCP --> MD([Market Data])

    subgraph "Replica Set Pipeline (4-Process Architecture)"
    Ingress
    Matching
    Egress_OMS
    Egress_MD
    end

Loading

Process Roles

1. Ingress: Manages TCP connections and timestamps incoming SBE orders into input_journal.xch.
2. Matching: Consumes from journal, executes matching, and dispatches split results (Private vs Public) to target RingBuffers.
3. Egress OMS: Dedicated process for Order Management reports (ExecutionReport).
4. Egress MD: Dedicated process for Market Data (PublicTrade, DepthUpdate).

⚙️ Core Mapping & SQPOLL

For maximum performance, all IO-bound processes (Ingress/Egress) use io_uring with SQPOLL enabled and kernel threads bound to dedicated physical cores.

Process	App Core	SQPOLL Core	Role
Ingress	Core 1	Core 5	TCP Inbound
Matching	Core 2	N/A	Engine Hot Path
Egress OMS	Core 3	Core 7	Order Reports
Egress MD	Core 4	Core 8	Market Data

📈 Performance

Recent benchmarks show exceptional stability and speed:

Metric	Value
Throughput	> 210,000 msg/s
Mean Latency	~19.2 µs
P50 Latency	18.3 µs

See full Performance Report for details.

🛠 Documentation

• Access Guide: SBE protocol definitions and connection details.
• Deployment Guide: Production setup, core binding, and process management.
• SBE Schema: Complete message definitions in XML.

🚥 Quick Start

Build

cargo build --release

Run (4-Process Setup)

Start processes in order:

1. Matching (node_type: dense):

   cargo run --release -- matching

2. Egress OMS:

   cargo run --release -- egress --type oms --target 127.0.0.1:9091

3. Egress MD:

   cargo run --release -- egress --type md --target 127.0.0.1:9092

4. Ingress:

   cargo run --release -- ingress --addr 0.0.0.0:8080

Test

Run full pipeline integration tests:

cargo test --test pipeline_test -- --nocapture

⚖️ License

Apache License 2.0.