Repository navigation
Quote Arrow field names that are not plain identifiers in generated DDL - #1202
Open
Shriraj1901 wants to merge 1 commit into
Open
Shriraj1901 wants to merge 1 commit into
Shriraj1901 wants to merge 1 commit into
Conversation
arrow_schema_to_column_defs() copied each Arrow field name straight into the column definition, so create_table_from_arrow_schema() produced invalid DDL for names that are common in Parquet and pandas data, such as "unit price", "order-id" or "2nd". A dotted name like "a.b" was read by the server as a Nested column instead of a single column. Arrow field names are literal, so any name that is not a plain identifier is now back-quoted (and escaped) with the same helper the SQLAlchemy dialect uses. Plain names such as "id" are returned unchanged, so existing DDL output does not change.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
arrow_schema_to_column_defs() copied each Arrow field name straight into the column definition, so create_table_from_arrow_schema() produced invalid DDL for names that are common in Parquet and pandas data, such as "unit price", "order-id" or "2nd". A dotted name like "a.b" was read by the server as a Nested column instead of a single column.
Arrow field names are literal, so any name that is not a plain identifier is now back-quoted (and escaped) with the same helper the SQLAlchemy dialect uses. Plain names such as "id" are returned unchanged, so existing DDL output does not change.
Summary
Checklist
Delete items not relevant to your PR:
Note
Low Risk
Localized DDL helper change with unit tests; plain identifiers are unchanged so existing output is preserved.
Overview
PyArrow → ClickHouse DDL now back-quotes Arrow column names when they are not plain SQL identifiers.
arrow_schema_to_column_defs()andcreate_table_from_arrow_schema()route each field name through_arrow_field_to_column_name(), which leaves names likeidor_ok9unchanged and wraps others with the existing_format_identifier()helper (spaces, hyphens, dots, leading digits, Unicode, embedded backticks). GeneratedCREATE TABLEstatements therefore match common Parquet/pandas column names instead of producing rejected DDL or misinterpretinga.bas Nested.Reviewed by Cursor Bugbot for commit 0fbf6ff. Bugbot is set up for automated code reviews on this repo. Configure here.