Skip to content

Speed up eq/neq of a byte-view array against a short scalar #10688

Description

@giladkl

Is your feature request related to a problem or challenge?

Filtering a Utf8View/BinaryView column against a short string constant (col = 'x', col <> '') is a very common predicate in analytical query engines, and it is one of the hottest kernels in a scan. Today arrow_ord::cmp::eq handles this through the generic ArrayOrd path: every row is reached through a closure over an (array, index) pair, and settling a row walks the full 128-bit view through a sequence of branches. For a constant that are short, much work can be eliminated.

Describe the solution you'd like

A constant of four bytes or fewer is described entirely by a view’s low 64 bits, which hold the length and the first four bytes. When neither side is a dictionary or REE array and the scalar is non-null, eq/neq can mask those bits and compare them against the constant resolved once up front, settling each row with a single narrow integer compare over the flat &[u128] view slice. That loop is branch-free and vectorizes. Every other shape falls through to the existing generic path unchanged.

Describe alternatives you've considered

No response

Additional context

Opening now a PR to fix...

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementAny new improvement worthy of a entry in the changelog

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions