Skip to content

Mixed McXtrace obstacles #118

Open
Open
Mixed McXtrace obstacles#118

Description

@willend
willend
opened on Feb 13, 2025

As a follow-up to the completion of #113 tested a little with
McXtrace 3.5.23 and mccode-antlr 0.10.0:

mxrun --cogen=mcxtrace-antlr ESRF_BM29.instr 
INFO: No output directory specified (--dir)
INFO: Using directory: "ESRF_BM29_20250213_074816"
INFO: Regenerating c-file: ESRF_BM29.c
2025-02-13 07:48:17.354 | DEBUG    | mccode_antlr.translators.c:sort_include_hierarchy:41 - sort includes :Source_flat(0):read_table-lib
2025-02-13 07:48:17.354 | DEBUG    | mccode_antlr.translators.c:sort_include_hierarchy:66 - sorted to read_table-lib
No initialization present?

-----------------------------------------------------------

Generating single GPU kernel or single CPU section layout:

-----------------------------------------------------------

Generating GPU/CPU -DFUNNEL layout:
-> GPU kernel from component Origin
-> GPU kernel from component Source
-> GPU kernel from component FirstPinhole
-> GPU kernel from component SecondPinhole
-> GPU kernel from component Sample
-> GPU kernel from component Beamstop
-> GPU kernel from component QMonitor

-----------------------------------------------------------
INFO: Recompiling: ./ESRF_BM29.out
./ESRF_BM29.c:10096:7: error: call to undeclared function 'RESTORE_XRAY'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 10096 |       RESTORE_XRAY(INDEX_CURRENT_COMP, x, y, z, kx, ky, kz, phi, t, Ex, Ey, Ez, p);
       |       ^
./ESRF_BM29.c:10221:7: error: call to undeclared function 'RESTORE_XRAY'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 10221 |       RESTORE_XRAY(INDEX_CURRENT_COMP, x, y, z, kx, ky, kz, phi, t, Ex, Ey, Ez, p);
       |       ^
2 errors generated.

Error occurs since the current mcxtrace-antlr uses RESTORE_NEUTRON.

Tiny PR incoming that renders McXtrace code-generation functional, so far only tested on this ESRF instr. :-)

Activity

  1. willend commented on Feb 13, 2025

    @willend
    willend
    on Feb 13, 2025
    ContributorAuthor

    Now running a full mxtest: there are still more issues incoming but many McXtrace instrs now run correctly with antlr-driven cogen.

  2. willend commented on Feb 18, 2025

    @willend
    willend
    on Feb 18, 2025
    ContributorAuthor

    @g5t if you squeeze out an 0.10.1 including #119 we could claim that mccode-antlr has

    • feature-complete support for McStas instruments via mcstas-antlr
    • basic support for McXtrace instruments via mcxtrace-antlr

    (I am think that would go together well with an announced set of McCode 3.5.24s where I would announce this plus also the availability of NCrystal on Windows. Will 'hit the button for 3.5.24 today and announce it tomorrow :-) )

  3. g5t commented on Feb 18, 2025

    @g5t
    g5t
    on Feb 18, 2025
    Collaborator

    @g5t if you squeeze out an 0.10.1 including #119 we could claim that mccode-antlr has

    * feature-complete support for McStas instruments via `mcstas-antlr`

* basic support for McXtrace instruments via `mcxtrace-antlr`

    (I am think that would go together well with an announced set of McCode 3.5.24s where I would announce this plus also the availability of NCrystal on Windows. Will 'hit the button for 3.5.24 today and announce it tomorrow :-) )

    Why not? As per PyPI - Version it's now available on PyPI. I think the feedstock for Conda-Forge should pickup the new version automatically after not too long too.

  4. g5t
    changed the title [-]Last(?) McXtrace obstacle - RESTORE_XRAY[/-] [+]Undefined McXtrace obstacles[/+] on Mar 26, 2025

  5. g5t commented on Mar 26, 2025

    @g5t
    g5t
    on Mar 26, 2025
    Collaborator

    @willend I've changed the title of this issue since RESTORE_XRAY was fixed, but you mentioned

    there are still more issues incoming but many McXtrace instrs now run correctly with antlr-driven cogen.

    Feel free to report those problems here or in new issues as you see fit, and when you have time.

  6. willend commented on Mar 27, 2025

    @willend
    willend
    on Mar 27, 2025
    ContributorAuthor

    Will start by a rerun of "full example suite" based on latest mcxtrace-antlr and current McCode main.

  7. willend commented on Mar 27, 2025

    @willend
    willend
    on Mar 27, 2025
    ContributorAuthor

    https://new-nightly.mcxtrace.org/todays-datafiles/2025-03-27_output.html has run (and should run with ANTLR once/day from now on).

    I am packing up a .tgz (unpacked 3Gig, sorry) with the whole folder structure for a later look at today's status...

  8. g5t commented on Jun 13, 2026

    @g5t
    g5t
    on Jun 13, 2026
    Collaborator

    It was unclear to me at the time that RESTORE_XRAY's functionality came with another code generator change, namely the addition of allow_backprop to the x-ray particle state structure.

    This change broke compilation of mcxtrace instruments produced by mcxtrace-antlr which use mxtrace-r from McCode v3.6.9 or later.
    Tests that could have identified this problem failed to do so because they have been run against a pinned version of McCode, namely v3.5.31.

  9. g5t commented on Sep 22, 2026

    @g5t
    g5t
    on Sep 22, 2026
    Collaborator

    This long-standing issue may have had some progress made against it's nebulous title. At the very least, I found that one McXtrace example instrument file has comments delimited by % -- yet another grammar rule which I did not know was allowed.
    https://github.com/mccode-dev/McCode/blob/29895c31a433b01ae0de95c9c6b654203559f06a/mcxtrace-comps/examples/Tests_optics/Test_Mono/Test_Mono.instr#L107-L108

    Also previously absent from mccode-antlr were C-style type casts, which are now modeled. These appear to have only been used in McXtrace instruments, so the fix may unlock more compatibility between mcxtrace-antlr and mcxtrace.

    Both the % comments and C-style type casts are added in #357 and will form part of the v0.30.0 release

  10. willend
    changed the title [-]Undefined McXtrace obstacles[/-] [+]Mixed McXtrace obstacles[/+] on Oct 20, 2026

  11. willend commented on Oct 20, 2026

    @willend
    willend
    on Oct 20, 2026
    ContributorAuthor

    @g5t I've started a Claude-driven investigation of what is missing here and there. Issues and PR's to appear eventually in both code-bases.

  12. willend commented on Oct 20, 2026

    @willend
    willend
    on Oct 20, 2026
    ContributorAuthor

    Status check: mccode-antlr main (0.35.1, 3bed330) against McCode main, full McXtrace mxtest --nsigma 5 --mpi=auto on macOS arm64. MCCOGEN was set to mcxtrace-antlr -I$CONDA_PREFIX/share/mcxtrace/resources, as in the McStas CI workflows.

    Not tested: Linux, Windows, GPU.

  13. g5t commented on Oct 21, 2026

    @g5t
    g5t
    on Oct 21, 2026
    Collaborator
    * Separate antlr issue found on the way: IR cache sidecars (`*.comp.<build>.json`) are not invalidated when a `SHARE INHERIT` parent changes. `SAXSPDB` kept a stale copy of `SAXSPDBFast`'s SHARE until the sidecar was deleted.

    FYI, this comment convinced me to follow-through on something I'd been considering for some time: turning off the on-disk component cache by default #402 (which also fixes the SHARE INHERIT bug you identified).

  14. willend commented on Oct 21, 2026

    @willend
    willend
    on Oct 21, 2026
    ContributorAuthor

    How about local .c/.h share snippets?

    Sorry, let me rephrase:
    Is there any handle to ensure that a local .c/.h takes precedence over what mccode-antlr finds upstream?
    Or in other words - does what I have in e.g. the McCode basictest workflow make sense / should I do something different:
    https://github.com/mccode-dev/McCode/blob/6d0e1a761054e2986de62b5fb20abb85a4a284fc/.github/workflows/mcstas-basictest.yml#L283

  15. g5t commented on Oct 21, 2026

    @g5t
    g5t
    on Oct 21, 2026
    Collaborator

    If I've understood correctly, you're asking about the possibility to modify some file (any file, really) that's in one of the mccode-pooch-registries *-registry.txt files, then ensure CI uses the local checkout rather than the tagged version from the repository?

    If that's right, then you want the commands added in #311 and #315 (part of v0.28.0),

    • cache register to make your own registry text files
    • and cache populate --registry-dir to use them;

    so the file cache gets whatever is in the local repository checkout (including mccode-r.h.in, which isn't installed and therefore isn't coverd by -I ${CONDA_PREFIX}/share/mscstas/resources).

    You should be able to put something like this in CI step to make sure mccode-antlr uses whatever you checkout into a directory called src:

      mccode-antlr cache register src common/lib/share mccode/nlib/share mccode/xlib/share -o regs/libc-registry.txt
  mccode-antlr cache register src mcstas-comps -o regs/mcstas-registry.txt
  mccode-antlr cache populate --from-path src --flavor mcstas --registry-dir regs

    There's a potential pitfall if you decide to specify a --tag to either of those commands: they both need a valid tag value and use whatever the confuse configuration has if you don't specify anything. So if you specify a tag for either, you have to specify the same tag for both and it needs to be a tag that mccode-pooch-registries knows about. (It's fine to reuse a tag for this, so leaving the default should work fine)

  16. khurrumsaleem
    added a commit that references this issue on Oct 22, 2026

    CI: populate mccode-antlr cache from registries minted from the checkout

    648c243
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions