Make rd_file use pybind#1252
Conversation
648f0a2 to
5052b19
Compare
There was a problem hiding this comment.
Pull request overview
This PR migrates the Python ResdataFile implementation from the legacy cwrap-based rd_file.py wrapper to a pybind11-based extension module, while also tightening index-file validation/error behavior and updating the C++ API to use FileType (enum class) and size_t where appropriate. It also fixes documented behavior for report_steps / num_report_steps() on non-restart files and expands test coverage around index corruption/truncation cases.
Changes:
- Replace the Python
rd_file.pywrapper with a pybind11 module (lib/resdata/rd_file_pybind.cpp) and add type stubs (python/resdata/resfile/rd_file.pyi). - Introduce pybind bindings for
FileMode+ newFileTypeenum via_rd_util, and update codebase usage from oldrd_file_enumconstants toFileType. - Rework index serialization/validation and add extensive tests for invalid index scenarios and non-restart-file restart APIs.
Reviewed changes
Copilot reviewed 71 out of 71 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/rd_tests/test_rd_file.py | Adds/adjusts Python tests for index filename storage, index corruption handling, and non-restart-file report_* behavior. |
| python/resdata/resfile/rd_file.pyi | New typing stub for the pybind-provided ResdataFile API. |
| python/resdata/resfile/rd_file.py | Removes the legacy cwrap-based Python implementation (module now provided by pybind). |
| python/resdata/resfile/_open_rd_file.py | Reintroduces open_rd_file() as a small Python context-manager wrapper around ResdataFile. |
| python/resdata/resfile/init.py | Updates exports to import open_rd_file from the new helper module and ResdataFile from the pybind module. |
| python/resdata/rd_util.py | Switches FileMode/FileType and _get_file_type to the new _rd_util pybind module and adjusts typing. |
| python/resdata/_rd_util.pyi | Extends the stub to export FileType and _get_file_type() along with FileMode. |
| lib/util/util.cpp | Removes legacy util_fwrite_string / util_fread_alloc_string helpers (index I/O moved to stream-based code). |
| lib/util/stringlist.cpp | Updates a comment to reflect C++ sources (vector.cpp/util.cpp) rather than C files. |
| lib/tests/test_well_info.cpp | Migrates tests from open_rd_file/C API to rd::File::open() and get_global_view(). |
| lib/tests/test_rd_sum.cpp | Migrates summary tests to rd::File::open() and C++ member calls (get_kw, write, etc.). |
| lib/tests/test_rd_kw.cpp | Updates overflow test to use iostreams and expects std::length_error from FileKW::read(). |
| lib/tests/test_rd_file_kw.cpp | Switches index header read/write tests from FILE* to std::ifstream/ofstream with exceptions enabled. |
| lib/tests/test_geertsma.cpp | Migrates various uses to rd::File::open() and get_global_view(). |
| lib/resdata/well_state.cpp | Updates comments and aligns assumptions with the FileView/C++ API naming. |
| lib/resdata/well_pybind.cpp | Updates load_rstfile dispatch to accept rd::File instances directly; improves type error message. |
| lib/resdata/well_info.cpp | Migrates well-loading paths from rd_file_type* to rd::File* and uses FileType. |
| lib/resdata/tests/well_ts.cpp | Updates rd_select_filelist usage to FileType::RESTART. |
| lib/resdata/tests/well_state_load.cpp | Migrates to rd::File::open() / get_global_view() and C++ has_kw(). |
| lib/resdata/tests/well_state_load_missing_RSEG.cpp | Migrates to rd::File::open() / get_global_view(). |
| lib/resdata/tests/well_segment_load.cpp | Migrates keyword lookups to rd::File::get_kw() and view accessors. |
| lib/resdata/tests/well_dualp.cpp | Migrates to rd::File::open() and C++ keyword access. |
| lib/resdata/tests/well_conn_CF.cpp | Migrates to rd::File::open() and C++ keyword access. |
| lib/resdata/tests/test_rd_file.cpp | Migrates to rd::File::open() and C++ save_kw() / size() usage; switches counts to size_t. |
| lib/resdata/tests/test_rd_file_index.cpp | Migrates fast-open/index tests to exceptions (rd::File::fast_open) and size_t. |
| lib/resdata/tests/rdxx_kw.cpp | Migrates keyword access to rd::File::get_kw(). |
| lib/resdata/tests/rd_rsthead.cpp | Migrates to rd::File::open() and FileType checks. |
| lib/resdata/tests/rd_restart_test.cpp | Migrates to rd::File::open() + find_sim_time() API (now optional). |
| lib/resdata/tests/rd_nnc_test.cpp | Migrates to rd::File::open() / num_named_kw() and new blockview() overload. |
| lib/resdata/tests/rd_lgr_test.cpp | Migrates to rd::File::open() / num_named_kw(). |
| lib/resdata/tests/rd_grid_volume.cpp | Migrates filename selection and file open to FileType + rd::File::open(). |
| lib/resdata/tests/rd_grid_init_fwrite.cpp | Migrates to rd::File::open() / get_kw(). |
| lib/resdata/tests/rd_grid_export.cpp | Migrates to rd::File::open() / has_kw() / get_kw(). |
| lib/resdata/tests/rd_grid_dx_dy_dz.cpp | Migrates to rd::File::open() / get_kw(). |
| lib/resdata/tests/rd_filenames.cpp | Migrates filename/type tests to FileType and updates includes. |
| lib/resdata/tests/rd_file_equinor.cpp | Removes close-stream/load-all legacy tests and migrates writable path to rd::File. |
| lib/resdata/tests/rd_dualp.cpp | Migrates to FileType + rd::File::open() / get_kw(). |
| lib/resdata/tests/rd_coarse_test.cpp | Migrates to FileType + rd::File::open() / get_kw(). |
| lib/resdata/rd_util.cpp | Replaces legacy rd_file_enum with FileType throughout and updates related APIs. |
| lib/resdata/rd_util_pybind.cpp | Adds _rd_util pybind module for FileMode, FileType, and _get_file_type(). |
| lib/resdata/rd_unsmry_loader.cpp | Migrates UNSMRY loading to rd::File::open() and C++ methods. |
| lib/resdata/rd_sum.cpp | Migrates summary-file type branching and filename selection to FileType. |
| lib/resdata/rd_sum_file_data.cpp | Migrates file checks and summary loading to rd::File and FileType. |
| lib/resdata/rd_sum_data.cpp | Migrates unified summary filename selection to FileType. |
| lib/resdata/rd_subsidence.cpp | Migrates INIT file handle from rd_file_type* to rd::File* and uses get_kw(). |
| lib/resdata/rd_subsidence_pybind.cpp | Updates binding to accept rd::File* directly for INIT file parameter. |
| lib/resdata/rd_smspec.cpp | Migrates header parsing to rd::File and updates keyword lookup/aliases to C++ API. |
| lib/resdata/rd_grid.cpp | Migrates many internals to rd::File/FileType and replaces several int indices with size_t. |
| lib/resdata/rd_grav.cpp | Migrates INIT file handle type to rd::File*, introduces helpers for phases/version reading, and improves errors. |
| lib/resdata/rd_grav_pybind.cpp | Updates binding to accept rd::File* directly for INIT file parameter. |
| lib/resdata/rd_grav_common.cpp | Migrates keyword access to rd::File C++ methods. |
| lib/resdata/rd_file.cpp | Replaces legacy rd_file_type C API with rd::File implementation, stream-based index I/O, and exception-based fast-open validation. |
| lib/resdata/rd_file_view.cpp | Switches index serialization to iostreams, migrates to size_t/optional APIs, and tightens index read error handling. |
| lib/resdata/rd_file_pybind.cpp | New pybind11 module implementing ResdataFile directly in C++ (replacing the old _file bridge). |
| lib/resdata/rd_file_kw.cpp | Migrates FileKW index header I/O to std::istream/std::ostream and adds allocation guards via reserve(). |
| lib/resdata/file_mode_pybind.cpp | Removes the old standalone _file_mode pybind module (replaced by _rd_util). |
| lib/resdata/cwrap_pybind.cpp | Removes from_cwrap<rd_file_type> support now that ResdataFile is no longer a cwrap object. |
| lib/private-include/detail/resdata/rd_unsmry_loader.hpp | Updates stored file handle type to std::unique_ptr<rd::File>. |
| lib/private-include/detail/resdata/rd_sum_file_data.hpp | Updates check_file signature to accept rd::File*. |
| lib/include/resdata/well/well_info.hpp | Updates WellInfo APIs to accept rd::File* instead of rd_file_type*. |
| lib/include/resdata/rd_util.hpp | Replaces rd_file_enum typedef with enum class FileType and updates function signatures. |
| lib/include/resdata/rd_subsidence.hpp | Updates API to accept const rd::File* for INIT file. |
| lib/include/resdata/rd_grav.hpp | Updates API to accept const rd::File* for INIT file. |
| lib/include/resdata/rd_grav_common.hpp | Updates API to accept const rd::File* for INIT file. |
| lib/include/resdata/rd_file.hpp | Replaces the legacy C API declarations with the new rd::File class interface. |
| lib/include/resdata/rd_file_view.hpp | Updates FileView index/container types to size_t/optional and migrates I/O to streams. |
| lib/include/resdata/rd_file_kw.hpp | Updates FileKW header read/write APIs to use iostreams. |
| lib/include/ert/util/util.hpp | Removes declarations for the deleted util_fwrite_string / util_fread_alloc_string. |
| lib/CMakeLists.txt | Renames the rd-file pybind module to rd_file and replaces _file_mode with _rd_util; adjusts test executable list. |
| applications/resdata/rd_unpack.cpp | Migrates unpack tool to FileType + rd::File C++ API and size_t. |
| applications/resdata/rd_pack.cpp | Migrates pack tool to FileType + rd::File C++ API. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| void FileView::write_index(std::ostream &ostream) const { | ||
| int n = static_cast<int>(size()); | ||
| ostream.write(reinterpret_cast<const char *>(&n), sizeof(n)); |
There was a problem hiding this comment.
This is existing behavior and we need binary compatible files so we cannot change this.
5052b19 to
9f38498
Compare
9f38498 to
19980dd
Compare
ec9de2c to
41b079f
Compare
fe1caf6 to
734c08c
Compare
e6977fa to
b890557
Compare
b890557 to
98b6627
Compare
To be sure has_sim_time and has_sim_days are consistent with restart_sim_date and restart_sim_days respectively, they have to not raise and iterate over SEQNUM as it may differ from the amount of DOUBHEAD so these functions were refactored.
This changes the error message in case we do not have permissions to view index_file_name or file_name.
98b6627 to
72358d2
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 72 out of 72 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
lib/resdata/rd_file_view.cpp:313
- The
find_sim_timedoc comment refers to callingiselect_rstblock()for LGR handling, but there is no such function in this codebase (and the surrounding API no longer exposes a block-selection mechanism). This makes the guidance misleading for users trying to interpret the returned index on LGR files.
Each report step only has one occurence of SEQNUM, but one INTEHEAD
for each LGR; i.e. one should call iselect_rstblock() prior to
calling this function. */
This fixes a bug with the report_steps and num_report_steps
methods, which would throw KeyError on files without "SEQNUM".
It now returns []/0 as documented.
The
has_kwandreport_datesfunctions had imprecise docstrings,which were updated.
In several places, changes from int to size_t had to be propagated to
avoid narrowing.
Also changes the return value of
iget_restart_sim_daysfrom sentinelsto raising
IndexErrorwhen the index is too large.Also changes
has_sim_time(t)to just bebool(find_sim_time(t)). Beforethere was a discrepancy between
has_sim_timeandfind_sim_timewherefind_sim_timework correctly on non-unified files, buthas_sim_timedid not.