Feature: run each plugin in its own subprocess with skip/abort on Ctrl+C - #1019
Open
Abu-Huraira21 wants to merge 3 commits into
Open
Feature: run each plugin in its own subprocess with skip/abort on Ctrl+C#1019Abu-Huraira21 wants to merge 3 commits into
Abu-Huraira21 wants to merge 3 commits into
Conversation
Adds --mp_per_plugin/--mp flag. Each plugin runs in a spawned subprocess; first Ctrl+C terminates just that plugin and continues, a second within 5s aborts the run. SIGUSR1/SIGUSR2 act as alternate skip triggers. Parent merges icons/LAVA-artifact deltas back after each subprocess completes. Rebased onto upstream's _HTML/_Script_Logs output layout and lava_db_name-based db path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Upstream renamed report_folder_base -> output_folder_base; the mp_per_plugin subprocess payload still referenced the old attribute, crashing on the first plugin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two mp_per_plugin-only bugs found via log diff between an --mp run and a plain run: - lava_open_existing() built lava_data without the 'meta' key, so lava_process_artifact()'s lava_data['meta']['modules'] access raised KeyError: 'meta' for every plugin. - Context.set_output_params() only ran once in the parent at startup; a spawned subprocess is a fresh interpreter, so any plugin calling Context.get_output_params() (mister_skinnylegs plugins, media linking helpers) raised "Context not set. OutputParameters not available." Now set per-subprocess from the known output folder layout. Also wire meta_modules_delta through the result queue and merge it in the parent, mirroring the existing icons/lava_artifacts delta pattern — otherwise the final _lava_data.lava's meta.modules would end up empty under --mp, since every plugin runs in an isolated subprocess. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a --mp_per_plugin / --mp flag that runs each plugin in an isolated subprocess instead of
in-process. Solves a common pain point during triage: a single slow/hung plugin currently forces you to
kill the whole run (losing all progress) since there's no way to skip past it.
Behavior
plugin.
sudo).
otherwise double-fire the handler.
How it works
resolved files_found, so no re-searching needed), and re-opens the existing LAVA sqlite DB.
lava_artifacts_delta, and meta_modules_delta (per-module metadata for _lava_data.lava). No plugin state
itself is shared.
Testing
Verified against a full local extraction run (625 plugins, --mp_per_plugin) — output (_lava_data.lava,
HTML/TSV reports, LAVA sqlite) matches an in-process run byte-for-byte aside from ordering-independent
fields. Also exercised the skip/abort paths manually via Ctrl+C mid-run.