Repository navigation
Add QA test framework for percona-binlog-server (branch 0.4) - #14
Merged
Merged
Conversation
Introduces pbs/, a standalone test suite and runner for Percona-Lab/percona-binlog-server (branch 0.4), covering fetch/pull streaming, GTID rewrite mode, crash/restart resilience, purge+resume, storage encryption, and binlog event-format diversity. Framework layout: - util.py: stateless leaf helpers (assertions, key=value config parsing, log tailing, the orphaned-binlog-index repair primitive). - servers.py: process wrappers for mysqld/an existing server/sysbench/ binlog_server, plus resilience helpers built on top of BinlogServer (retry-after-repair for a known post-hard-kill storage inconsistency, catch-up polling). - setup.py: test config loading/validation, main_config.json generation, shared argparse flags, and the ScenarioRunner base class that owns the common setup/teardown lifecycle every test shares so each test only implements its own scenario() steps. - config.json: central config (basedir, binlog_server_bin, sysbench_*, encryption/keyring settings, S3/connection settings). Tests (tests/): - gtid_rewrite_test.py / nogtid_norewrite_test.py: fetch+pull streaming in GTID+rewrite and position/no-rewrite modes. - rewrite_mode_transition_test.py: switching an existing storage directory from non-rewrite to rewrite mode mid-stream. - pull_restart_resilience_test.py: repeated SIGKILL+restart of `pull` under continuous load (rapid and gapped restart phases). - pull_purge_resume_test.py: repeated kill -> purge_binlogs -> resume cycles, verifying correct GTID-based resumption after a purge. - binlog_event_diversity_test.py: varies binlog_format/row_image/ transaction_compression across a battery of DDL/DML/transaction scenarios to exercise diverse binlog event types. - Optional per-test *_test.config files (restart counts, timing, iteration counts) for pull_restart_resilience_test.py and pull_purge_resume_test.py, separate from the shared config.json. Self-healing: detects and repairs the specific post-hard-kill storage inconsistency where a rotated binlog file's metadata was written but the index commit referencing it wasn't, verified against upstream's actual storage/index-validation source rather than assumed. Encryption test coverage: when a cipher is left unset, one is chosen at random per run (instead of always exercising a single default) from the combinations percona-binlog-server's own source confirms are valid, while enforcing the real compatibility constraint between the keyring's KEK cipher mode and the data-encryption key length. pbs_test_runner.py: runs one or more tests sequentially or across a small pool of parallel workers, with --tests/--run-all-tests selection, an -e/--encryption override, and automatic preservation of a failed test's logs before its working directory is reused by the next test. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
tplavcic
requested changes
Oct 6, 2026
tplavcic
requested changes
Oct 8, 2026
binlog_server logs to stdout via a temporary cout_logger for any failure that happens before main_config.json has been successfully parsed (e.g. a version/schema mismatch) -- the file-based logger, and binsrv.log itself, only get set up after that succeeds. fetch() only ever captured stderr, so this whole class of failure showed up as a content-free "fetch failed (exit 1): " with no binsrv.log and the real reason silently dropped. Now includes whichever of stdout/stderr actually has content. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces pbs/, a standalone test suite and runner for Percona-Lab/percona-binlog-server (branch 0.4), covering fetch/pull streaming, GTID rewrite mode, crash/restart resilience, purge+resume, storage encryption, and binlog event-format diversity.
Framework layout:
Tests (tests/):
pullunder continuous load (rapid and gapped restart phases).Self-healing: detects and repairs the specific post-hard-kill storage inconsistency where a rotated binlog file's metadata was written but the index commit referencing it wasn't, verified against upstream's actual storage/index-validation source rather than assumed.
Encryption test coverage: when a cipher is left unset, one is chosen at random per run (instead of always exercising a single default) from the combinations percona-binlog-server's own source confirms are valid, while enforcing the real compatibility constraint between the keyring's KEK cipher mode and the data-encryption key length.
pbs_test_runner.py: runs one or more tests sequentially or across a small pool of parallel workers, with --tests/--run-all-tests selection, an -e/--encryption override, and automatic preservation of a failed test's logs before its working directory is reused by the next test.