Skip to content

Commit 97b8a8b

Browse files
committed
FEATURE-ci-configured-symbol-tool: Honor configured symbol inspector in both checks
Pass the actual generated NM command through Make and installed package inspection, preserving command arguments and failing closed without inspector fallback. Keep the 24 compiled-definition and demangling requirements unchanged. Verification: 10 new checker/generated-Make/lane tests, 31 existing affected consumer fixtures and 4 package workflow tests pass; Python compilation, bootstrap/Automake and diff checks pass. Explicit local GNU 2.42 inspection passes on the exact retained FreeBSD archive. Two pre-edit configure sanity failures remain failed/incomplete without retry; native post-fix FreeBSD checks remain CI/v3 PR owned.
1 parent 4003ed9 commit 97b8a8b

5 files changed

Lines changed: 269 additions & 3 deletions

File tree

‎Makefile.am‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -628,8 +628,10 @@ EXTRA_DIST += scripts/lib/example_inventory.py scripts/lib/example_consumers.py
628628

629629
EXTRA_DIST += docs/v3-source-layout.md scripts/check-v3-compiled-operations.py
630630

631+
# Export preserves quotes in configured commands without shell re-evaluation.
632+
check-v3-compiled-operations: export V3_COMPILED_OPERATIONS_NM = $(NM)
631633
check-v3-compiled-operations: $(top_builddir)/src/libhttpserver.la
632-
python3 $(top_srcdir)/scripts/check-v3-compiled-operations.py $(top_builddir)/src/.libs/libhttpserver.a
634+
python3 $(top_srcdir)/scripts/check-v3-compiled-operations.py --nm="$$V3_COMPILED_OPERATIONS_NM" $(top_builddir)/src/.libs/libhttpserver.a
633635

634636
.PHONY: check-v3-compiled-operations
635637

‎scripts/check-v3-compiled-operations.py‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
#!/usr/bin/env python3
22
"""Check the installed-library implementation contract for representative operations."""
33
import argparse
4+
import shlex
45
import subprocess
56

67
# These operations must have a library-owned definition even when no consumer
@@ -35,9 +36,16 @@
3536

3637
def main():
3738
parser = argparse.ArgumentParser(description=__doc__)
39+
parser.add_argument('--nm', default='nm', help='configured symbol-tool command, including options')
3840
parser.add_argument('library')
3941
args = parser.parse_args()
40-
result = subprocess.run(['nm', '-C', args.library],
42+
try:
43+
command = shlex.split(args.nm)
44+
except ValueError as error:
45+
parser.error('invalid NM command: ' + str(error))
46+
if not command or not command[0]:
47+
parser.error('NM command is empty')
48+
result = subprocess.run(command + ['-C', args.library],
4149
check=True, text=True, capture_output=True)
4250
definitions = [line for line in result.stdout.splitlines()
4351
if ' T ' in line]

‎scripts/check-v3-installed-consumer.py‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
import os
88
from pathlib import Path
99
import platform
10+
import re
1011
import shlex
1112
import shutil
1213
import subprocess
@@ -137,8 +138,11 @@ def _package_lane(args, source, output, mode, inventory):
137138
matrix.check_linkage(compiled_graph, linkage, prefix)
138139
command([str(compiled_executable)], consumer_dir, runtime_env)
139140
public[linkage]['compiled_operations'] = dict(compile=compiled_provenance, dependencies=compiled_graph)
141+
configured_nm = re.search(r'^NM = (.*)$', (build / 'Makefile').read_text(), re.M)
142+
if configured_nm is None:
143+
raise ValueError('generated Makefile is missing configured NM')
140144
command([sys.executable, str(source / 'scripts/check-v3-compiled-operations.py'),
141-
str(prefix / 'lib/libhttpserver.a')], consumer_dir, runtime_env)
145+
'--nm=' + configured_nm[1], str(prefix / 'lib/libhttpserver.a')], consumer_dir, runtime_env)
142146
private_dir = lane / ('integration-' + linkage)
143147
private_dir.mkdir()
144148
private_libraries = libraries + (shlex.split(env['V3_TLS_LIBS']) if mode == 'yes' else [])

‎scripts/test-v3-installed-consumer.py‎

Lines changed: 200 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,206 @@ def test_default_make_overrides_ambient_make(self):
111111
self.exercise_selection('make', ambient_make=True)
112112

113113

114+
115+
@unittest.skipIf(os.name == 'nt', 'POSIX executable fixture; native Windows is CI owned')
116+
class CompiledOperationsFixture(unittest.TestCase):
117+
"""Execute the checker with a controlled symbol-producer boundary."""
118+
# Independent examples of the existing compiled-definition contract.
119+
symbols = (
120+
'httpserver::http::fields::append(',
121+
'httpserver::server::server_options::validate(',
122+
'httpserver::server::resource_budget::reserve(',
123+
'httpserver::server::route_pattern::parse(',
124+
'httpserver::server::make_sync_route(',
125+
'httpserver::manual_executor::run_one(',
126+
'httpserver::detail::current_executor_slot(',
127+
'httpserver::detail::task_frame_base::finish(',
128+
'httpserver::stop_token::cancellation_awaiter::await_suspend(',
129+
'httpserver::detail::fire_trigger(',
130+
'httpserver::exchange::admit_body(',
131+
'httpserver::body_reader::collect(',
132+
'httpserver::response_writer::finish(',
133+
'httpserver::response_definition::owned_bytes(',
134+
'httpserver::send_definition(',
135+
'httpserver::detail::send_cursor::pull(',
136+
'httpserver::detail::http1_head_parser::',
137+
'httpserver::detail::hpack_decoder::',
138+
'httpserver::detail::qpack_decoder::',
139+
'httpserver::websocket::options::validate(',
140+
'httpserver::detail::http2_dispatch::post(',
141+
'httpserver::detail::valid_http1_host(',
142+
'httpserver::detail::unescape_buf_raw(',
143+
'httpserver::detail::owned_completion_storage::claim_read(',
144+
)
145+
146+
def setUp(self):
147+
self.temp = tempfile.TemporaryDirectory()
148+
self.addCleanup(self.temp.cleanup)
149+
self.root = Path(self.temp.name).resolve()
150+
self.checker = Path(__file__).with_name('check-v3-compiled-operations.py').resolve()
151+
self.selected = self.root / "selected tools' directory" / 'symbol tool'
152+
self.selected.parent.mkdir()
153+
self.calls = self.root / 'selected.json'
154+
self.default_calls = self.root / 'default-used'
155+
self.default = self.root / ('nm.exe' if os.name == 'nt' else 'nm')
156+
self.write_tool(self.default, f'from pathlib import Path; Path({str(self.default_calls)!r}).touch(); raise SystemExit(91)')
157+
self.archive = self.root / 'installed archive.a'
158+
self.archive.write_bytes(b'archive identity fixture')
159+
self.environment = dict(os.environ, PATH=str(self.root) + os.pathsep + os.environ['PATH'])
160+
self.selection = shlex.join([str(self.selected), '-B', '--selected-option'])
161+
self.producer()
162+
163+
def write_tool(self, path, code):
164+
path.write_text('#!' + sys.executable + '\n' + code + '\n')
165+
path.chmod(0o755)
166+
167+
def producer(self, symbols=None, status=0):
168+
text = '\n'.join('00000000 T ' + name for name in self.symbols) if symbols is None else symbols
169+
self.write_tool(self.selected, 'import json,sys; from pathlib import Path; '
170+
f'Path({str(self.calls)!r}).write_text(json.dumps(sys.argv[1:])); '
171+
f'print({text!r}); raise SystemExit({status})')
172+
173+
def check(self, selection=None):
174+
return subprocess.run([sys.executable, str(self.checker), '--nm=' + (self.selection if selection is None else selection),
175+
str(self.archive)], env=self.environment, text=True, capture_output=True)
176+
177+
178+
class CompiledOperationsToolTest(CompiledOperationsFixture):
179+
def test_selected_command_preserves_options_spaces_demangling_and_archive(self):
180+
result = self.check()
181+
self.assertEqual(result.returncode, 0, result.stderr)
182+
self.assertEqual(json.loads(self.calls.read_text()), ['-B', '--selected-option', '-C', str(self.archive)])
183+
self.assertFalse(self.default_calls.exists())
184+
185+
def test_omitted_selection_preserves_default_nm_cli(self):
186+
self.producer()
187+
shutil.copyfile(self.selected, self.default)
188+
result = subprocess.run([sys.executable, str(self.checker), str(self.archive)],
189+
env=self.environment, text=True, capture_output=True)
190+
self.assertEqual(result.returncode, 0, result.stderr)
191+
self.assertEqual(json.loads(self.calls.read_text()), ['-C', str(self.archive)])
192+
193+
def test_missing_or_undefined_definition_remains_fatal(self):
194+
for kind in ('missing', 'undefined', 'lowercase'):
195+
with self.subTest(kind=kind):
196+
lines = ['00000000 T ' + name for name in self.symbols[1:]]
197+
if kind != 'missing':
198+
lines.append((' U ' if kind == 'undefined' else '00000000 t ') + self.symbols[0])
199+
self.producer('\n'.join(lines))
200+
result = self.check()
201+
self.assertNotEqual(result.returncode, 0)
202+
self.assertIn('Missing compiled library operations:\n' + self.symbols[0], result.stderr)
203+
self.assertFalse(self.default_calls.exists())
204+
205+
def test_nonzero_selected_tool_is_fatal_even_with_complete_definitions(self):
206+
self.producer(status=23)
207+
result = self.check()
208+
self.assertNotEqual(result.returncode, 0)
209+
self.assertTrue(self.calls.exists(), result.stderr)
210+
self.assertIn('23', result.stderr)
211+
self.assertFalse(self.default_calls.exists())
212+
213+
def test_empty_output_remains_missing_definitions(self):
214+
self.producer('')
215+
result = self.check()
216+
self.assertNotEqual(result.returncode, 0)
217+
self.assertIn('Missing compiled library operations', result.stderr)
218+
self.assertFalse(self.default_calls.exists())
219+
220+
def test_empty_malformed_or_unavailable_selection_never_uses_default(self):
221+
for selection in ('', ' ', "''", "'unclosed", str(self.root / 'absent-tool')):
222+
with self.subTest(selection=selection):
223+
result = self.check(selection)
224+
self.assertNotEqual(result.returncode, 0)
225+
self.assertFalse(self.calls.exists())
226+
self.assertFalse(self.default_calls.exists())
227+
228+
229+
@unittest.skipIf(os.name == 'nt', 'POSIX Autotools fixture; Windows is CI owned')
230+
class ConfiguredSymbolEntryPointsTest(CompiledOperationsFixture):
231+
"""Real generated Makefiles and lane subprocesses; no full library build."""
232+
def configure(self, source, build, environment):
233+
result = subprocess.run(['bash', str(source / 'configure'), '--disable-v3-tls', '--disable-examples',
234+
'--prefix=' + str(self.root / 'matrix/tls-no/prefix')],
235+
cwd=build, env=environment, text=True, capture_output=True)
236+
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
237+
238+
def test_generated_owning_make_target_uses_configured_nm(self):
239+
source = Path(__file__).resolve().parent.parent
240+
build = self.root / 'build'
241+
build.mkdir()
242+
self.configure(source, build, dict(self.environment, NM=self.selection))
243+
result = subprocess.run([shutil.which('gmake') or shutil.which('make'), '-o', 'src/libhttpserver.la',
244+
'check-v3-compiled-operations'], cwd=build, env=self.environment,
245+
text=True, capture_output=True)
246+
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
247+
self.assertEqual(json.loads(self.calls.read_text()),
248+
['-B', '--selected-option', '-C', './src/.libs/libhttpserver.a'])
249+
self.assertFalse(self.default_calls.exists())
250+
251+
def exercise_lane(self, *, missing=False, selection=None, error='23'):
252+
source = self.root / 'source'
253+
source.mkdir()
254+
# A small genuine configured project reaches the existing package lane
255+
# without replacing its compile, dependency inspection or symbol check.
256+
(source / 'configure.ac').write_text('AC_INIT([symbol-lane], [1])\n'
257+
'AM_INIT_AUTOMAKE([foreign])\nAC_PROG_CC\n'
258+
'AC_ARG_ENABLE([v3-tls])\nAC_ARG_ENABLE([examples])\n' +
259+
('' if missing else 'AC_SUBST([NM], [' + shlex.quote(self.selection if selection is None else selection) + '])\n') +
260+
'AC_CONFIG_FILES([Makefile])\nAC_OUTPUT\n')
261+
(source / 'Makefile.am').write_text(('' if missing else 'NM = @NM@\n') +
262+
'all-local:\n\t$(CC) -c $(srcdir)/fixture.c -o fixture.o\n\tar cr libhttpserver.a fixture.o\n'
263+
'install-exec-local:\n\tmkdir -p $(prefix)/lib/pkgconfig\n'
264+
'\tcp libhttpserver.a $(prefix)/lib/libhttpserver.a\n'
265+
'\tprintf "Name: libhttpserver\\nDescription: fixture\\nVersion: 1\\nLibs: -lhttpserver\\n" > $(prefix)/lib/pkgconfig/libhttpserver.pc\n'
266+
'check-v3-native-linkage check-headers check-hygiene check-v3-installed-package-fixtures:\n\t@:\n')
267+
(source / 'fixture.c').write_text('int fixture(void) { return 1; }\n')
268+
for name, content in (('package_consumer.cpp', 'int main() { return 0; }\n'),
269+
('compiled_operations.cpp', 'int main() { return 0; }\n'),
270+
('compiled_operations_worker.cpp', 'int worker() { return 0; }\n')):
271+
path = source / 'test/installed' / name
272+
path.parent.mkdir(parents=True, exist_ok=True)
273+
path.write_text(content)
274+
(source / 'examples').mkdir()
275+
(source / 'examples/host_poll.hpp').write_text('// fixture\n')
276+
(source / 'scripts').mkdir()
277+
shutil.copyfile(self.checker, source / 'scripts' / self.checker.name)
278+
generated = subprocess.run(['autoreconf', '--install', '--force'], cwd=source,
279+
text=True, capture_output=True)
280+
self.assertEqual(generated.returncode, 0, generated.stdout + generated.stderr)
281+
output = self.root / 'matrix'
282+
output.mkdir()
283+
spec = importlib.util.spec_from_file_location('configured_symbol_lane', self.checker.with_name('check-v3-installed-consumer.py'))
284+
runner = importlib.util.module_from_spec(spec)
285+
spec.loader.exec_module(runner)
286+
args = SimpleNamespace(compiler=shutil.which('c++'), cc=shutil.which('cc'),
287+
make=shutil.which('gmake') or shutil.which('make'), jobs=1, prerequisite_prefix=[],
288+
pkg_config_dir=[], provider_prefix=self.root, toolchain_prefix=None, optimized=False)
289+
inventory = matrix.load()
290+
inventory.update(linkages=['static'], loops=[])
291+
self.producer(status=23) # Halt at the selected inspector with real failure evidence.
292+
with patch.dict(os.environ, dict(self.environment, NM='unrelated-ambient-nm'), clear=True):
293+
with self.assertRaisesRegex((RuntimeError, ValueError), error):
294+
runner.package_lane(args, source, output, 'no', inventory)
295+
self.assertFalse(self.default_calls.exists())
296+
return output
297+
298+
def test_installed_lane_uses_generated_selection_over_ambient_nm(self):
299+
output = self.exercise_lane()
300+
self.assertEqual(json.loads(self.calls.read_text()), ['-B', '--selected-option', '-C',
301+
str(output / 'tls-no/prefix/lib/libhttpserver.a')])
302+
self.assertEqual((output / 'tls-no/prefix/lib/libhttpserver.a').read_bytes(),
303+
(output / 'tls-no/build/libhttpserver.a').read_bytes())
304+
305+
def test_installed_lane_missing_configured_nm_fails_without_default(self):
306+
self.exercise_lane(missing=True, error='configured NM')
307+
self.assertFalse(self.calls.exists())
308+
309+
def test_installed_lane_empty_configured_nm_fails_without_default(self):
310+
self.exercise_lane(selection='', error='NM command is empty')
311+
self.assertFalse(self.calls.exists())
312+
313+
114314
class InstalledShellTest(unittest.TestCase):
115315
"""Controlled process boundary; these are not native Windows receipts."""
116316
def setUp(self):
Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# FEATURE-ci-configured-symbol-tool
2+
3+
Status: In Progress
4+
Base: v3 `4003ed97e112fefc5fa68b67e4e5868daae1bad2`
5+
Plan: `.groundwork-plans/FEATURE-ci-configured-symbol-tool-plan.md`
6+
Journal: primary `.debug/ci-6b2-freebsd-nm-assessment-7tg4peed/journal.md`
7+
8+
Honor the build's actual configured NM at the compiled-operation Make target and
9+
installed consumer checkpoint. The retained FreeBSD failure invoked bare `nm -C`
10+
despite generated `NM = /usr/local/bin/nm -B`. Its exact crashed tool version and
11+
internal crash cause are unknown; this change repairs the configured-tool handoff.
12+
13+
- [x] Support explicit selected symbol-tool command/arguments with safe argv execution.
14+
- [x] Pass actual configured NM consistently at both real entrypoints.
15+
- [x] Exercise checker and generated Make/lane entrypoints with a conflicting
16+
default tool, options, quoted spaces, and installed archive identity.
17+
- [x] Preserve all 24 required names, uppercase T definitions, demangling and
18+
fatal missing/undefined/nonzero/empty/malformed-tool behavior without fallback.
19+
- [x] Complete focused checks, Python compilation and build-definition validation.
20+
- [ ] Caller-owned validation/finalization.
21+
22+
Native FreeBSD post-fix checks remain INCOMPLETE and CI/v3 PR owned. No full native
23+
build or baseline suite, SDK installation, CI retry, merge or publication is part
24+
of this implementation. Other closed investigations remain closed.
25+
26+
Pre-edit focused baseline: 9/11 tests pass; two unchanged InstalledMakeTest cases
27+
fail in configure sanity with `newly created file is older than distributed files!`.
28+
Those results remain failed/incomplete; the exact timestamp cause is unproven.
29+
30+
## Implementation verification
31+
32+
- 10 new checker/generated-Make/lane regressions PASS, with meaningful pre-edit
33+
RED for missing CLI and both entrypoints selecting the default PATH inspector.
34+
- 31 existing shell/inspection/matrix consumer fixtures PASS; 4 package workflow
35+
checks PASS. The earlier two InstalledMakeTest sanity failures remain failed
36+
and incomplete; they were not retried. Other unrelated broad tests were not run.
37+
- Bootstrap and Automake generation PASS (existing .PHONY/AC_OUTPUT warnings);
38+
all three changed Python modules compile; `git diff --check` PASS.
39+
- One offline explicit `--nm=/usr/bin/x86_64-linux-gnu-nm -B` check passes on the
40+
exact retained 122420834-byte FreeBSD archive, SHA-256
41+
`3d1f70de66312ad5fbefaeb706a5b32d650a56945d362850c963d04d30360852`.
42+
This uses local GNU Binutils 2.42, executes no foreign program, and confirms the
43+
unchanged 24-operation contract. Native configured GNU 2.44 is unverified.
44+
45+
Literal RED/GREEN/build-generation logs, source-bound static checks and archive
46+
receipt are durable in the primary journal's `implementation/` directory. No
47+
production C++, installed headers, ABI, runtime/dependency policy, required symbol
48+
names, uppercase T predicate or demangling requirement changed. Selected tool
49+
failure never triggers another inspector. Local implementation is ready for
50+
caller-owned validation; status stays In Progress through that handoff. Native
51+
post-fix FreeBSD/other-platform evidence remains INCOMPLETE and CI/v3 PR owned.
52+
No push, merge, cleanup or CI controls were performed by this executor.

0 commit comments

Comments
 (0)