Skip to content

Make rd_file use pybind#1252

Open
eivindjahren wants to merge 58 commits into
mainfrom
use_pybind_for_rd_file
Open

Make rd_file use pybind#1252
eivindjahren wants to merge 58 commits into
mainfrom
use_pybind_for_rd_file

Conversation

@eivindjahren

@eivindjahren eivindjahren commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

This fixes a bug with the report_steps and num_report_steps
methods, which would throw KeyError on files without "SEQNUM".
It now returns []/0 as documented.

The has_kw and report_dates functions had imprecise docstrings,
which were updated.

In several places, changes from int to size_t had to be propagated to
avoid narrowing.

Also changes the return value of iget_restart_sim_days from sentinels
to raising IndexError when the index is too large.

Also changes has_sim_time(t) to just be bool(find_sim_time(t)). Before
there was a discrepancy between has_sim_time and find_sim_time where
find_sim_time work correctly on non-unified files, but has_sim_time did not.

@eivindjahren
eivindjahren force-pushed the use_pybind_for_rd_file branch 5 times, most recently from 648f0a2 to 5052b19 Compare July 16, 2026 13:52
@eivindjahren
eivindjahren requested a review from Copilot July 16, 2026 14:45

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the Python ResdataFile implementation from the legacy cwrap-based rd_file.py wrapper to a pybind11-based extension module, while also tightening index-file validation/error behavior and updating the C++ API to use FileType (enum class) and size_t where appropriate. It also fixes documented behavior for report_steps / num_report_steps() on non-restart files and expands test coverage around index corruption/truncation cases.

Changes:

  • Replace the Python rd_file.py wrapper with a pybind11 module (lib/resdata/rd_file_pybind.cpp) and add type stubs (python/resdata/resfile/rd_file.pyi).
  • Introduce pybind bindings for FileMode + new FileType enum via _rd_util, and update codebase usage from old rd_file_enum constants to FileType.
  • Rework index serialization/validation and add extensive tests for invalid index scenarios and non-restart-file restart APIs.

Reviewed changes

Copilot reviewed 71 out of 71 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/rd_tests/test_rd_file.py Adds/adjusts Python tests for index filename storage, index corruption handling, and non-restart-file report_* behavior.
python/resdata/resfile/rd_file.pyi New typing stub for the pybind-provided ResdataFile API.
python/resdata/resfile/rd_file.py Removes the legacy cwrap-based Python implementation (module now provided by pybind).
python/resdata/resfile/_open_rd_file.py Reintroduces open_rd_file() as a small Python context-manager wrapper around ResdataFile.
python/resdata/resfile/init.py Updates exports to import open_rd_file from the new helper module and ResdataFile from the pybind module.
python/resdata/rd_util.py Switches FileMode/FileType and _get_file_type to the new _rd_util pybind module and adjusts typing.
python/resdata/_rd_util.pyi Extends the stub to export FileType and _get_file_type() along with FileMode.
lib/util/util.cpp Removes legacy util_fwrite_string / util_fread_alloc_string helpers (index I/O moved to stream-based code).
lib/util/stringlist.cpp Updates a comment to reflect C++ sources (vector.cpp/util.cpp) rather than C files.
lib/tests/test_well_info.cpp Migrates tests from open_rd_file/C API to rd::File::open() and get_global_view().
lib/tests/test_rd_sum.cpp Migrates summary tests to rd::File::open() and C++ member calls (get_kw, write, etc.).
lib/tests/test_rd_kw.cpp Updates overflow test to use iostreams and expects std::length_error from FileKW::read().
lib/tests/test_rd_file_kw.cpp Switches index header read/write tests from FILE* to std::ifstream/ofstream with exceptions enabled.
lib/tests/test_geertsma.cpp Migrates various uses to rd::File::open() and get_global_view().
lib/resdata/well_state.cpp Updates comments and aligns assumptions with the FileView/C++ API naming.
lib/resdata/well_pybind.cpp Updates load_rstfile dispatch to accept rd::File instances directly; improves type error message.
lib/resdata/well_info.cpp Migrates well-loading paths from rd_file_type* to rd::File* and uses FileType.
lib/resdata/tests/well_ts.cpp Updates rd_select_filelist usage to FileType::RESTART.
lib/resdata/tests/well_state_load.cpp Migrates to rd::File::open() / get_global_view() and C++ has_kw().
lib/resdata/tests/well_state_load_missing_RSEG.cpp Migrates to rd::File::open() / get_global_view().
lib/resdata/tests/well_segment_load.cpp Migrates keyword lookups to rd::File::get_kw() and view accessors.
lib/resdata/tests/well_dualp.cpp Migrates to rd::File::open() and C++ keyword access.
lib/resdata/tests/well_conn_CF.cpp Migrates to rd::File::open() and C++ keyword access.
lib/resdata/tests/test_rd_file.cpp Migrates to rd::File::open() and C++ save_kw() / size() usage; switches counts to size_t.
lib/resdata/tests/test_rd_file_index.cpp Migrates fast-open/index tests to exceptions (rd::File::fast_open) and size_t.
lib/resdata/tests/rdxx_kw.cpp Migrates keyword access to rd::File::get_kw().
lib/resdata/tests/rd_rsthead.cpp Migrates to rd::File::open() and FileType checks.
lib/resdata/tests/rd_restart_test.cpp Migrates to rd::File::open() + find_sim_time() API (now optional).
lib/resdata/tests/rd_nnc_test.cpp Migrates to rd::File::open() / num_named_kw() and new blockview() overload.
lib/resdata/tests/rd_lgr_test.cpp Migrates to rd::File::open() / num_named_kw().
lib/resdata/tests/rd_grid_volume.cpp Migrates filename selection and file open to FileType + rd::File::open().
lib/resdata/tests/rd_grid_init_fwrite.cpp Migrates to rd::File::open() / get_kw().
lib/resdata/tests/rd_grid_export.cpp Migrates to rd::File::open() / has_kw() / get_kw().
lib/resdata/tests/rd_grid_dx_dy_dz.cpp Migrates to rd::File::open() / get_kw().
lib/resdata/tests/rd_filenames.cpp Migrates filename/type tests to FileType and updates includes.
lib/resdata/tests/rd_file_equinor.cpp Removes close-stream/load-all legacy tests and migrates writable path to rd::File.
lib/resdata/tests/rd_dualp.cpp Migrates to FileType + rd::File::open() / get_kw().
lib/resdata/tests/rd_coarse_test.cpp Migrates to FileType + rd::File::open() / get_kw().
lib/resdata/rd_util.cpp Replaces legacy rd_file_enum with FileType throughout and updates related APIs.
lib/resdata/rd_util_pybind.cpp Adds _rd_util pybind module for FileMode, FileType, and _get_file_type().
lib/resdata/rd_unsmry_loader.cpp Migrates UNSMRY loading to rd::File::open() and C++ methods.
lib/resdata/rd_sum.cpp Migrates summary-file type branching and filename selection to FileType.
lib/resdata/rd_sum_file_data.cpp Migrates file checks and summary loading to rd::File and FileType.
lib/resdata/rd_sum_data.cpp Migrates unified summary filename selection to FileType.
lib/resdata/rd_subsidence.cpp Migrates INIT file handle from rd_file_type* to rd::File* and uses get_kw().
lib/resdata/rd_subsidence_pybind.cpp Updates binding to accept rd::File* directly for INIT file parameter.
lib/resdata/rd_smspec.cpp Migrates header parsing to rd::File and updates keyword lookup/aliases to C++ API.
lib/resdata/rd_grid.cpp Migrates many internals to rd::File/FileType and replaces several int indices with size_t.
lib/resdata/rd_grav.cpp Migrates INIT file handle type to rd::File*, introduces helpers for phases/version reading, and improves errors.
lib/resdata/rd_grav_pybind.cpp Updates binding to accept rd::File* directly for INIT file parameter.
lib/resdata/rd_grav_common.cpp Migrates keyword access to rd::File C++ methods.
lib/resdata/rd_file.cpp Replaces legacy rd_file_type C API with rd::File implementation, stream-based index I/O, and exception-based fast-open validation.
lib/resdata/rd_file_view.cpp Switches index serialization to iostreams, migrates to size_t/optional APIs, and tightens index read error handling.
lib/resdata/rd_file_pybind.cpp New pybind11 module implementing ResdataFile directly in C++ (replacing the old _file bridge).
lib/resdata/rd_file_kw.cpp Migrates FileKW index header I/O to std::istream/std::ostream and adds allocation guards via reserve().
lib/resdata/file_mode_pybind.cpp Removes the old standalone _file_mode pybind module (replaced by _rd_util).
lib/resdata/cwrap_pybind.cpp Removes from_cwrap<rd_file_type> support now that ResdataFile is no longer a cwrap object.
lib/private-include/detail/resdata/rd_unsmry_loader.hpp Updates stored file handle type to std::unique_ptr<rd::File>.
lib/private-include/detail/resdata/rd_sum_file_data.hpp Updates check_file signature to accept rd::File*.
lib/include/resdata/well/well_info.hpp Updates WellInfo APIs to accept rd::File* instead of rd_file_type*.
lib/include/resdata/rd_util.hpp Replaces rd_file_enum typedef with enum class FileType and updates function signatures.
lib/include/resdata/rd_subsidence.hpp Updates API to accept const rd::File* for INIT file.
lib/include/resdata/rd_grav.hpp Updates API to accept const rd::File* for INIT file.
lib/include/resdata/rd_grav_common.hpp Updates API to accept const rd::File* for INIT file.
lib/include/resdata/rd_file.hpp Replaces the legacy C API declarations with the new rd::File class interface.
lib/include/resdata/rd_file_view.hpp Updates FileView index/container types to size_t/optional and migrates I/O to streams.
lib/include/resdata/rd_file_kw.hpp Updates FileKW header read/write APIs to use iostreams.
lib/include/ert/util/util.hpp Removes declarations for the deleted util_fwrite_string / util_fread_alloc_string.
lib/CMakeLists.txt Renames the rd-file pybind module to rd_file and replaces _file_mode with _rd_util; adjusts test executable list.
applications/resdata/rd_unpack.cpp Migrates unpack tool to FileType + rd::File C++ API and size_t.
applications/resdata/rd_pack.cpp Migrates pack tool to FileType + rd::File C++ API.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/resdata/rd_file.cpp
Comment on lines +442 to +444
void FileView::write_index(std::ostream &ostream) const {
int n = static_cast<int>(size());
ostream.write(reinterpret_cast<const char *>(&n), sizeof(n));

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is existing behavior and we need binary compatible files so we cannot change this.

Comment thread lib/resdata/tests/rd_restart_test.cpp Outdated
Comment thread lib/resdata/rd_file_pybind.cpp

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 71 out of 71 changed files in this pull request and generated 2 comments.

Comment thread python/resdata/resfile/rd_file.pyi Outdated
Comment thread lib/resdata/rd_file_pybind.cpp
@eivindjahren
eivindjahren force-pushed the use_pybind_for_rd_file branch from 9f38498 to 19980dd Compare July 17, 2026 08:29
@eivindjahren
eivindjahren requested a review from Copilot July 17, 2026 08:32
Comment thread lib/resdata/rd_grid.cpp Fixed
Comment thread lib/resdata/rd_grid.cpp Fixed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 77 out of 78 changed files in this pull request and generated 3 comments.

Comment thread lib/resdata/rd_grid.cpp
Comment thread lib/resdata/tests/rd_restart_test.cpp
Comment thread lib/include/resdata/rd_file.hpp Outdated
@eivindjahren
eivindjahren force-pushed the use_pybind_for_rd_file branch 3 times, most recently from ec9de2c to 41b079f Compare July 17, 2026 10:20
Comment thread lib/resdata/tests/rd_restart_test.cpp Fixed
@eivindjahren
eivindjahren force-pushed the use_pybind_for_rd_file branch 5 times, most recently from fe1caf6 to 734c08c Compare July 17, 2026 10:49
@eivindjahren
eivindjahren requested a review from Copilot July 17, 2026 10:53

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 77 out of 78 changed files in this pull request and generated 2 comments.

Comment thread lib/resdata/rd_grav.cpp Outdated
Comment thread lib/resdata/rd_file.cpp
@eivindjahren eivindjahren moved this to Ready for Review in SCOUT Jul 20, 2026
@eivindjahren
eivindjahren force-pushed the use_pybind_for_rd_file branch 2 times, most recently from e6977fa to b890557 Compare July 20, 2026 07:40
@eivindjahren
eivindjahren requested a review from Copilot July 20, 2026 07:41

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 72 out of 72 changed files in this pull request and generated 2 comments.

Comment thread lib/resdata/rd_file_view.cpp
Comment thread lib/resdata/rd_file_view.cpp Outdated
@eivindjahren
eivindjahren force-pushed the use_pybind_for_rd_file branch from b890557 to 98b6627 Compare July 20, 2026 07:50
To be sure has_sim_time and has_sim_days are consistent with
restart_sim_date and restart_sim_days respectively, they have to
not raise and iterate over SEQNUM as it may differ from the amount
of DOUBHEAD so these functions were refactored.
This changes the error message in case we do not
have permissions to view index_file_name or file_name.
@eivindjahren
eivindjahren force-pushed the use_pybind_for_rd_file branch from 98b6627 to 72358d2 Compare July 20, 2026 08:05
@eivindjahren eivindjahren changed the title Use pybind for rd file Make rd_file use pybind Jul 20, 2026
@eivindjahren
eivindjahren requested a review from Copilot July 20, 2026 08:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 72 out of 72 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

lib/resdata/rd_file_view.cpp:313

  • The find_sim_time doc comment refers to calling iselect_rstblock() for LGR handling, but there is no such function in this codebase (and the surrounding API no longer exposes a block-selection mechanism). This makes the guidance misleading for users trying to interpret the returned index on LGR files.
Each report step only has one occurence of SEQNUM, but one INTEHEAD
for each LGR; i.e. one should call iselect_rstblock() prior to
calling this function. */

Comment thread lib/include/resdata/rd_file.hpp Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 72 out of 72 changed files in this pull request and generated 1 comment.

Comment thread lib/resdata/rd_file_view.cpp

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 72 out of 72 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 72 out of 72 changed files in this pull request and generated no new comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Ready for Review

Development

Successfully merging this pull request may close these issues.

3 participants