[rjb32]

Convert every column type and encoding the engine outputs.

[rjb32]

Note

AI-generated summary.

Summary

Goal. The Python client raised dataframeToNumpy: unhandled column kind on four column kinds the engine sends. One is MATCH (n:T) WITH {id: n.i} AS m RETURN sum(m.id % 10), a nullable cell. The value_types regress added in #1359 fails on all four, in native and embedded mode.

Approach. dataframeToNumpy was a switch of 55 hand-listed (container, type) cases. It now hands each column to ColumnSingleDispatcher over OutputtedTypes::Allowed with a generic lambda, the way the wire encoder dispatches. One per-row function, toPyElement, returns a numpy element or a Python object, and that return type picks the column's form. The concepts in net/encoders/OutputValues.h classify the types, and ColumnTypeGenerator names the dtype.

Where to look.

• python/turingdb/_common/NanobindUtils.cpp:297 toPyElement — which rows become numpy elements. Every column's form follows from its return type.
• python/turingdb/_common/NanobindUtils.cpp:353 the convert lambda in dataframeToNumpy — constants, numeric vectors moved into numpy without a copy, and every other vector.
• python/turingdb/_common/NanobindUtils.cpp:266 toPyObject — the Python object for a value with no array form.
• net/encoders/OutputValues.h:119 ColumnTypeGenerator — no caller since the JSON protocol went. Null now reads Null, the name Python already reported.

 dataframeToNumpy
   for each column
-    switch (col->getKind())                 # 55 hand-listed cases
-      default: throw "unhandled column kind"
+    ColumnSingleDispatcher<OutputtedTypes::Allowed>::dispatch(col, convert)
+      convert(column)
+        constant         toPyElement(value), repeated as an array or a list
+        numeric vector   the vector itself, moved into numpy
+        other vector     toPyElement(row) for each row, into an array or a list

array    Int64, UInt64, Double, Bool (uint8), DateTime and Duration (int64 µs, NaT for null),
         label, edge type, property type and label set IDs, change IDs, commit hashes (uint64)
object   String, Embedding (one array per row), Path (its IDs), List, Map, Cell, Point,
         ValueType (its name), Null, node and edge IDs (None when invalid); None for any null

The forms are the ones the switch produced, with one exception: a constant node or edge column is now a list, like a node or edge vector, instead of a raw uint64 array. A point converts through ValueToPyObject::point(), which main added. Its pointColumn and optional point() overload served only the switch and are gone.

Evidence

• Before: regress/value_types at Regress: test every column type and encoding the Python and npm clients read #1359, with the client built from that branch: 78 passed and 10 failed. The same 5 cases fail native and embedded:

  FAIL a nullable String constant       ColumnConst<std::optional<std::string_view>>
  FAIL a nullable List constant         ColumnConst<std::optional<db::ListView>>
  FAIL a nullable Cell column           ColumnVector<std::optional<db::CellView>>
  FAIL a nullable Cell column from an aggregate
       MATCH (n:T) WITH {id: n.i} AS m RETURN sum(m.id % 10) AS total
       TuringDBException: dataframeToNumpy: unhandled column kind: db::ColumnVector<std::optional<db::CellView> >
  FAIL a nullable Cell constant         ColumnConst<std::optional<db::CellView>>
  

  After: value_types 88 of 88, the point case included. test_suite_queries 333 passed, 0 failed, 138 skipped. entity_list_values and query_parameters pass native and embedded. journey_planner_npm 168 of 168.

Run on main at 49d454e84 with this stack on top: the server, the simpledb sample and both Python modules built from it, Python 3.12. Each main.py ran against a private port rather than through run.sh, which kills every turingdb on the host. The npm test used the package CI built for main. The middle commit b9900c303 also builds on its own. CI runs the suite through run_regress.

Merge Danger

Door: two-way

Reverting restores the switch. Nothing on disk or on the wire changes.

Blast Radius: Python client

Every query result from both Python clients goes through the new dispatch. Numeric vectors still move into numpy without a copy. A constant node or edge column comes back as a list (pandas reads both as UInt64). A column kind outside OutputtedTypes::Allowed now raises the dispatcher's Unsupported unary operation on column of type … in place of unhandled column kind. turing_py_nanobind_utils now links turing_db_net_encoders explicitly; both modules already reached it through other links.