Skip to content

feat(event): run native queue and event plugin simultaneously#6852

Open
h45hc47 wants to merge 1 commit into
tronprotocol:developfrom
h45hc47:feat(event)-run-native-queue-and-event-plugin-simultaneously
Open

feat(event): run native queue and event plugin simultaneously#6852
h45hc47 wants to merge 1 commit into
tronprotocol:developfrom
h45hc47:feat(event)-run-native-queue-and-event-plugin-simultaneously

Conversation

@h45hc47

@h45hc47 h45hc47 commented Jun 22, 2026

Copy link
Copy Markdown

What does this PR do?

Makes the native ZMQ message queue and the event plugin (e.g. the MongoDB plugin) two independent event sinks, so a node can publish triggers to both at the same time.

Previously they were mutually exclusive: when useNativeQueue = true, the plugin's path/server/dbconfig were never loaded and only the native queue started; the plugin only ran when useNativeQueue = false.

Key changes:

  • Args: always copy path/server/dbconfig into EventPluginConfig, regardless of useNativeQueue.
  • EventPluginLoader: add a separate useEventPlugin flag (derived from a non-empty plugin path). start() launches the plugin first — so the plugin's listeners exist before any topic is registered — and then the native queue. Each post*Trigger is dispatched to every active sink; setPluginTopic is null-guarded for the now-possible double trigger-config pass; isBusy() is gated on the plugin being active instead of on the native queue being off.
  • HistoryEventService: honor the plugin's back-pressure (isBusy) during historical sync even when the native queue is enabled, so the plugin queue is not flooded. Native-only pacing is unchanged.
  • config.conf / reference.conf: document that useNativeQueue and a plugin path can be combined.

Why are these changes required?

Operators often want both transports at once: low-latency ZMQ delivery for real-time consumers AND durable persistence in MongoDB for querying and indexing. Today they must pick one. With a single event.subscribe block (native queue on + a plugin path set), both now receive every trigger.

Example configuration enabling both:

event.subscribe = {
  enable = true
  native   = { useNativeQueue = true, bindport = 5555, sendqueuelength = 1000 }
  path     = "/home/trx/event-plugin/plugin-mongodb-1.0.0.zip"
  server   = "127.0.0.1:27017"
  dbconfig = "db|log|pass"
  ...
}

This PR has been tested by:

  • Unit Tests
    • Built on JDK 8 (jdk1.8.0_271) (from GreatVoyage-v4.8.1.1 tag): :common and :framework compileJava succeed.
    • EventPluginLoaderTest#testIsBusy — updated for the new useEventPlugin semantics — PASSED.
    • HistoryEventServiceTest#test — covers the reordered back-pressure path — PASSED.
  • Manual Testing
    • Verified in a Docker deployment: a mainnet full node running with useNativeQueue = true and a MongoDB plugin path set starts cleanly, writes events to the MongoDB collections, and binds the native ZMQ queue on port 5555 simultaneously.

Follow up

A future enhancement could add an explicit useEventPlugin config key instead of inferring plugin activation from a non-empty path.

Extra details

In combined mode the trigger-config pass runs once per launcher; this is idempotent (boolean flags and setTopic overwrite with the same values). Every trigger is now serialized and dispatched to both sinks, a modest overhead versus single-sink mode. Behavior is unchanged for the two existing single-sink configurations (native-only, or plugin-only).

@github-actions github-actions Bot requested review from 0xbigapple and xxo1shine June 22, 2026 13:21
@h45hc47 h45hc47 closed this Jun 22, 2026
@h45hc47 h45hc47 reopened this Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant