Skip to content

can we use stable sort for tdigest #739

Description

@Zaid-Work

the sort on centroids (v,w) -> (value, weight) in tdigest merge uses the sort based on value (v) and index -> therefore if we have centroids like (200,1)(100,3)(300,2)(100,1) then at merge it can be sorted to 2 states :

  1. (100,1)(100,3)(200,1)(300,2)
  2. (100,3)(100,1)(200,1)(300,2)

due to which the merging result can differ

other than that we can use sorting based on value (v) and weight (w) as in
https://github.com/tdunning/t-digest/blob/main/core/src/main/java/com/tdunning/math/stats/Sort.java

and this will make the sort stable and remove variance in result

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions