Go source does not reproduce the bundled gomtp.dylib

[aj0413]

Summary

The Go source currently committed under ffi/kalam/native cannot reproduce the precompiled lib/gomtp.dylib shipped with SwiftMTP v1.2.4.

The dylib implements a newer multi-device backend API and additional USB synchronization logic that are absent from the repository source. This prevents contributors from auditing, debugging, or rebuilding the backend used by the application.

Evidence

The generated header declares this API:

extern void FetchAvailableDevices(on_cb_result_t* onDonePtr);
extern void Initialize(char* initInputJson, on_cb_result_t* onDonePtr);
extern void CancelTransfer(char* cancelTransferInputJson, on_cb_result_t* onDonePtr);
extern void Dispose(char* deviceInputJson, on_cb_result_t* onDonePtr);

However, the checked-in Go source has an older API:

//export Initialize
func Initialize(onDonePtr *C.on_cb_result_t)

It does not contain implementations for FetchAvailableDevices, CancelTransfer, or InitializeByDeviceId.

Static inspection of lib/gomtp.dylib shows that the compiled backend contains:

main.FetchAvailableDevices
main.CancelTransfer
main.getContainer
main.containersMap
main.containersLock
main.libusbGlobalLock
main.libusbContextLock
main.libusbLocked
github.com/ganeshrvel/go-mtpx.InitializeByDeviceId

It also contains behavior and diagnostic strings absent from the source:

FetchAvailableDevices: USB busy, returning %d cached devices
FetchAvailableDevices: Merging active device from cache: %s
FetchAvailableDevices: Cached device %s is stale
USB diag: device may be occupied by other processes

Example inspection command:

nm -arch arm64 -nm lib/gomtp.dylib |
  grep -E 'FetchAvailableDevices|CancelTransfer|containersMap|libusbLocked'

The binary reports that it was built with Go 1.26.1:

strings -arch arm64 lib/gomtp.dylib | grep 'go1\.'

The repository artifact at v1.2.4 has SHA-256:

ff774eb9bbab4f149c78d2890a81adc1e607d9e12d3816a52c4beadb9ad1bc1e

Build-script mismatch

ffi/kalam/native/scripts/build.sh still produces:

lib/kalam.dylib
lib/kalam.h

Xcode instead links:

lib/gomtp.dylib
lib/gomtp.h

Therefore, running the documented build script does not recreate the backend used by the application.

Git history indicates that a precompiled kalam.dylib was added and updated directly. Commit be9a80f later renamed that artifact to gomtp.dylib, but the corresponding newer Go source was not committed.

Dependency mismatch

The binary identifies these pinned modules:

github.com/ganeshrvel/go-mtpfs v1.0.4-0.20240426083057-1c3302b3c476
github.com/ganeshrvel/go-mtpx v0.0.0-20240426092756-18f12db021cc
github.com/ganeshrvel/usb v0.0.0-20210103155855-14d96f5ae403

However, the compiled go-mtpfs includes a custom getUSBOccupyingPIDs implementation not present in the pinned upstream source. This suggests that locally modified dependency source was used without recording it through a committed fork, vendor directory, or replace directive.

Why this matters

This currently prevents contributors from:

• Rebuilding the application entirely from source.
• Auditing the exact USB and filesystem behavior being distributed.
• Fixing backend defects and validating that the fix reaches the release.
• Reproducing the generated gomtp.h.
• Verifying dependency patches.
• Producing deterministic release artifacts.

It also affects an observed connection bug. Static analysis indicates that the compiled busy-device path can serialize a nil device slice as:

{"data": null}

The Swift frontend expects an array and reports:

MTPManager: Failed to decode available devices payload

Without the matching Go source, this cannot be corrected at its origin.

Requested resolution

Please consider:

1. Committing the exact Go source used to build the current gomtp.dylib.
2. Publishing any locally modified dependency forks or vendored patches.
3. Updating go.mod with the actual dependency revisions or replace directives.
4. Updating the build script to generate lib/gomtp.dylib and lib/gomtp.h.
5. Documenting the required Go, SDK, macOS deployment target, and libusb versions.
6. Adding a CI check that builds the dylib and verifies its exported symbols against gomtp.h.
7. Rebuilding the checked-in binary from the committed source.

This issue is about source and release reproducibility; it is not an assertion that the existing binary is malicious.

[aj0413]

btw @Neighbor-Z I did generate this issue description from Codex, after I noticed discrepancy while trying to debug a connectivity issue.

I was using it during the course of investigation, as I find it good for decompilation analysis and such.

I don't know your stance on AI usage, but figured you'd be cool with it given the Apple AI Platform support notice.

Mainly I'd just appreciate if you could upload the actual source code (and relevant patches to go-mtpx and go-mtpfs) so that one can reproduce your builds and debug.

I also think, in light of this, that you should consider (or, if open, I wouldn't mind contributing) adding github action workflows for handling releases, with built in GH artifact attestation. This would also mean you can remove the pre-built dylibs :)

I'm not a swift dev (never worked with it before), but I think I understand that the pre-compiled libs was to make life simpler. Problem is, that as someone that works with security tooling, the binaries and source code mismatch were massive red flags and you kinda ran into the immediate problem with that workflow where you can be jumping around a bunch and forget to upload things.

At this point, I'm heavily considering forking and working in parralel, just so my paranoia is satisfied 😅 but I'm also not particularly interested in being a FOSS maintainer tbh; just another random dev that hated the state of OpenMTP, but needs something to fill its void

[Neighbor-Z]

Thanks for bringing this to my attention. I do have been thinking about how to better organize the Go source code. However, at this point there are still quite a few factors that would make working on that part of the codebase inefficient.

My plan is to implement full multi-session MTP support, but that effort is still in its early stages and will involve significant changes to the codebase. This may require me to create a separate repo to manage the project. In the meantime, I'd rather focus on improving the user experience and polishing the current release instead of rushing to add new features.

If you're interested, please checkout the related repos https://github.com/Neighbor-Z/go-mtpx/ and https://github.com/Neighbor-Z/go-mtpfs/. They're slightly few commits behind my current work.

Lastly, please let me know what the connectivity issue you're debugging with is.

[aj0413]

Appreciet linking to the other repos and I understand the desire to focus on current goals, but this does create a trust issue, you understand? I know most people don't bother checking, but one of the goals of OSS is that the code is transparent and auditable.

Your time is your own obvsiously, so I'll probably go ahead and just quickly whip up fork(s?) to avoid the issue; maybe you can eventually use it as a reference point when your ready to clean up the management and release process on your side? I'd appreciate greatly if you could at least push the branches for go-mtpx and go-mtpfs upstream along with ffi changes consuming em in this repo.

That way I can at least try and stay in sync with your work and actually reproduce the gomtp.dylib

As for the issue, it was that the USB controller would report that it was busy or something, but the reported PIDS would point to SwiftMTP itself and Edge. Neither of which were the true culprits; eventually after some time this cleared itself and started working. I know Preview was open before trying to open SwiftMTP and that it historically can interfere with OpenMTP, so I had closed it once I noticed it was open and I assume this might be some locking/caching related thing.

I notice you have retry logic around connections, so maybe an exponential backoff would be appropriate? Try 5 times with longer wait periods?

[Neighbor-Z]

That's a good point, and it's one of the reasons I'm also considering getting this App notarized by Apple or distributing it through the Mac App Store. At this point, those options are much more realistic and would provide a tangible improvement to the user experience.

I have pushed the new commit to the repo https://github.com/Neighbor-Z/go-mtpfs/. Please check the PID related logics out in that repo.

As for the retry logic... yes, it does exist, but I don't actually want the App to keep retrying that many times. Instead, I think it's better for the users to adjust the USB cable or the device settings as needed. Once everything's ready, they can simply click the Connect button manually.