test: add regression test framework with CI for contributors#316
Conversation
Docker-based test infrastructure for OLR regression testing: - Dockerfile.dev: dev image with all deps + gtest - Per-environment configs in tests/1-environments/ (free-23 first) - generate.sh: 7-stage pipeline (SQL → redo capture → schema → LogMiner → OLR → compare → golden file) - test_pipeline.cpp: parameterized gtest, auto-discovers fixtures from both 2-prebuilt/ (Git LFS) and 3-generated/ (CI artifact) - compare.py: content-based OLR vs LogMiner matching - CI: sql-tests-free23 (generates fixtures), redo-log-tests (replays) - Includes basic-crud scenario as initial SQL test input
concurrent-updates, data-types, ddl-add-column, interleaved-transactions, large-transaction, lob-operations, long-spanning-txn, many-columns, multi-table, null-handling, number-precision, partitioned-table, rollback, special-chars, timestamp-variants, wide-rows
- tests/1-environments/xe-21/ with PDB=XEPDB1 - sql-tests-xe21.yaml CI workflow - redo-log-tests downloads from both free-23 and xe-21 - All 17 scenarios pass on XE 21.3.0
|
@bersler I have been using Debezium + LogMiner in my corporate envs for quite some time, and like everyone else that came to you, I am evaluating whether the envs can benefit from integrating with OLR. Reading the "Important Note on AI-Generated Patches and Forks" section in README, I guess you likely have received some AI-made contriutions without proper verifications, and likely asked or expected YOU to do the verification. Having a test framework backed with public visible CI pipeline helps in the way that you can request contributors to contribute in the form of test cases. Hope that helps. Disclaimer: this PR is created by AI (Claude) as well. I did not take a careful look at the test cases, but they look basic enough, and being docker-based helps to have consistent results across everyone's machine. |
| @@ -0,0 +1,345 @@ | |||
| /* Full pipeline I/O tests for OpenLogReplicator | |||
There was a problem hiding this comment.
We're trying to understand the intended split between the two, because right
now they look like duplicated logic.
As far as we can tell:
generate.shbrings up Oracle, runs the SQL scenario, captures the redo
logs, and once produces the reference by running OLR and comparing its
output against LogMiner.test_pipeline.cppthen re-runs OLR on those same redo logs and
compares the result line-by-line against that reference.
So the actual OLR-on-redo run is done twice against effectively the same data.
It would make sense if test_pipeline.cpp were a standalone regression test —
i.e. commit the 3-generated/ fixtures and run only test_pipeline.cpp (no
Oracle, no LogMiner) on every change. But as it stands it's a mix: the C++
runner needs the fixtures freshly generated by the Oracle-backed generate.sh
each time, so it doesn't really stand on its own.
Question: what was the intended division of responsibility here? Was
test_pipeline.cpp meant to be the real regression gate (with committed
fixtures, runnable without Oracle), or is the LogMiner comparison inside
generate.sh already the source of truth? Right now the same thing runs in
both places and we'd like to remove the duplication — which direction did you
have in mind?
Motivation
I know that OLR has a "private, proprietary test suite with 1,000+ edge-case scenarios", which is nice.
This PR is NOT meant to replace existing test coverage. Instead, just to provide a basic safety net for contributions.
This PR adds a lightweight regression test framework that:
What's included
generate.sh): captures redo logs + golden output from a live Oracle instancetest_pipeline.cpp): replays captured redo logs against golden files — no Oracle needed at test timesql-tests-*: spins up Oracle in Docker, runs SQL scenarios, captures redo logs and golden output as artifacts. Runs on pushes to the test branch.redo-log-tests: downloads the captured fixtures, builds OLR, replays redo logs and compares output against golden files. Runs on PRs — no Oracle needed.See how the GHA workflows work in my fork: Redo Logs Test, SQL Tests against Free 23ai
Workflow for contributors