fix(rnd-21448): Fixtures.to_df() returns empty values for nested vessel/corporate_entities fields

[hansenfan]

RELATED TICKETS

https://vortexa.atlassian.net/jira/software/c/projects/RND/boards/465?selectedIssue=RND-21448

! Do not post related PRs here, unless they are open source !

CHANGELOG

• Add convert_fixture_to_flat_dict to group vessel.corporate_entities by layer name before flattening
• Fix tones → tonnes typo in model, DEFAULT_COLUMNS, and docstrings
• Correct docstring column names to match actual API response

TESTS

• Verified before/after with live API: vessel.corporate_entities.effective_controller.label and
  time_charterer.label now populate correctly in to_df(), along with other fields. Has the ability to fill 'vessel.corporate_entities.effective_controller.label' which by definition exists in the schema, and if it exists like ticket says, but across all the fixtures it doesn't exist

pytest -v tests/endpoints/test_fixtures_real.py
pytest -v tests/api/test_fixture_flattening.py

COMMENTS

• The ticket mentions vessels.corporate_entities.charterer.label but the API returns vessel (singular) and never includes charterer in corporate_entities, only at root level
• Also worried that tones -> tonnes may be a breaking change
• Also ticket said vessels, but it's vessel in the api structure and is only a single object, not a list

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

vortexasdk/api/fixture.py:26

• Renaming the Fixture field from tones to tonnes is a breaking change for SDK consumers who may still access fixture.tones. Consider keeping a deprecated alias property so existing code keeps working while the correct field name is used going forward.

    id: ID
    vessel: Optional[VesselEntity] = None
    laycan_from: Optional[str] = None
    laycan_to: Optional[str] = None
    tonnes: Optional[int] = None
    fixing_timestamp: Optional[str] = None
    vtx_fulfilled: Optional[bool] = None
    origin: Optional[Entity] = None
    destination: Optional[Entity] = None
    product: Optional[Entity] = None
    charterer: Optional[Entity] = None

[hansenfan]

Would appreciate some input on this. I think because changing to tonnes could be a breaking change it is doing this to make sure that people don't have to modify anything but idk if that's a good practice

[jack-bliss]

tones -> tonnes probably is a breaking change, i'm not sure we can release that without checking with product first. we may be able to support both for a while, and then release a new major version removing the incorrect spelling

[hansenfan]

@jack-bliss ya that makes sense. although thinking about it, if it was already broken before we'd be making a breaking change to something that was already broken so maybe it'd be fine? idk if that's a valid thought

[jack-bliss]

Verified: tones → tonnes is a bug fix, not a breaking change.

Jack and I (Claude) tested the current master SDK against the live API (/v5/search/fixtures) and confirmed:

1. The API returns "tonnes" (correct spelling) in the response JSON.

2. The current "tones" field has never worked:

   • to_list() → fixture.tones is always None (Pydantic can't match API's "tonnes" to a field named tones without an alias)
   • to_df() with default columns → "tones" column is empty string for all rows (the flattener operates on the raw dict which has key "tonnes", so requesting "tones" yields no match)

3. "tonnes" already works if requested explicitly:

   df = Fixtures().search(...).to_df(columns=["id", "tonnes"])
   # df["tonnes"] → 100000.0, 100000.0, ...  (actual data)

4. Test output (449 fixtures, 2020-01-01 to 2020-01-02):

   df['tones'] empty count: 449 / 449
   df['tones'] sample: ['', '', '']
   
   df['tonnes'] sample: [100000.0, 100000.0, '']
   

No customer could have been relying on "tones" since it was always empty/None. This rename makes the field actually populate for the first time.

[hansenfan]

It is kinda confusing tho since to_list was populating tonnes despite the spelling mistake. Claude said it has something to do with Pydantic accepting tones as tonnes but I couldn't find where. I think the to_df() was handled differently and that's why there were issues with it

[jack-bliss]

Also verified: vessels → vessel (singular) in the docstring is also a bug fix, not a breaking change.

Same approach — Jack and I (Claude) tested the current master SDK against the live API:

df = Fixtures().search(...).to_df(columns=[
    'vessel.name', 'vessel.imo', 'vessel.corporate_entities.0.label',
    'vessels.name', 'vessels.imo', 'vessels.corporate_entities.0.label',
])

Results:

df['vessel.name'] sample: ['MAYLO', 'MAYLO', 'EVITA']         ← populated
df['vessel.imo'] sample: [9331153.0, 9331153.0, 9408530.0]    ← populated
df['vessel.corporate_entities.0.label'] sample: ['Vitol', 'Vitol', 'UNKNOWN']  ← populated

df['vessels.name'] sample: ['', '', '']   ← always empty
df['vessels.imo'] sample: ['', '', '']    ← always empty
df['vessels.corporate_entities.0.label'] sample: ['', '', '']  ← always empty

The API returns "vessel" (singular, single object — not a list). The DEFAULT_COLUMNS already correctly used "vessel.name", "vessel.id" etc. The plural "vessels.*" only appeared in the docstring and would have never returned data for anyone copying those column names. The PR's docstring correction is safe.

[jack-bliss]

It is kinda confusing tho since to_list was populating tonnes

this isn't what i've found when running against master - can you show an example of it working?

[hansenfan]

Oh wait yeah you're right, it doesn't work. The ticket says people currently use to_list() instead of to_df() since the to_df() functionality isn't working, but I guess not since to_list() is also cooked