Skip to content

Sources: a guided start and quicker building (#104) - #114

Merged
phornstein merged 16 commits into
mainfrom
feat/104-quicker-building
Nov 2, 2026
Merged

phornstein merged 16 commits into
mainfrom
feat/104-quicker-building

Conversation

@phornstein

@phornstein phornstein commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Phase 3 of the flow UX plan approved in #104. Stacked on #113, and needs stareSDK 0.2.33 (RecordTable draggableColumns; kineticlogic-io/stareSDK#32, merged and waiting for its release).

  • Shorter step list: Map, Filter and Throttle, then Output and NATS topic. Reject, Entity links, Affiliation and Tracker are under More steps.
  • Quicker building:
    • A step dropped onto a connection it can sit in goes between its two steps.
    • One dropped just right of a step's output that can feed it is connected from there.
    • Tidy lays the flow out left to right (dagre).
  • Guided start:
    • The input card offers Capture samples while there are none.
    • An empty Map's card offers Use suggested mapping once a capture suggested one.
  • Drag a field to map it: in the input's samples, a column can be dragged onto a Map.
    • The field goes where the probe's analysis of the captured samples proposed, else to an extension field named after it (ext.<name>).
    • No feed's field names are written into the editor.
    • The Map opens so you can check it.

joinDropped, guessTarget and mapField in sourceFlow.ts are tested with neutral paths.

Checked (dev server, against stareSDK 0.2.33): tsc -b, oxlint 0 errors, vitest 76/76, vite build. Headless on 8091 with ais-stream, without saving:

  • The step list groups are right.
  • Tidy put the input, Map and the two outputs in columns 0, 320 and 640.
  • 25 input columns are draggable, and dropping one on the Map updates its rule.
    No feed-specific code. The probe's suggestion (crates/ot-source/src/probe.rs) had tables of particular feeds' field names (sog, cog, shipname, mmsi, icao, hex, alt_baro…), with the units and identifier schemes they implied. It now reads only:
  • The values: coordinates (a fractional pair in one object, the one past ±90 the longitude), times, CoT types, and the key (fixed-length tokens, the most distinct).
  • OpenTrack's own field names: from the schema's targets, with or without the unit, and only where the values fit.

It suggests no identifiers or unit conversions. The tests use neutral fixtures, and the admin guide and CHANGELOG say so.

Live capture of ais-stream on 8091 suggested:

  • MetaData.latitude / MetaData.longitude, by OpenTrack's names
  • MetaData.time_utc, by its values
  • MetaData.MMSI as the key, by its values
  • name and callsign

ot-source 117/117 and ot-server 228/228 pass; clippy is clean across the workspace.

🤖 Generated with Claude Code

phornstein and others added 7 commits October 27, 2026 19:19
The dry run (/sources/validate) answers with steps: {id: {in, out}}, what
each step took and passed on over all the stored samples: frames and
records for the input, records or observations for the rest, and what an
output delivered.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…onnections (#104)

- The selected step's samples are in a Tray along the bottom of the
  workspace: a RecordTable, one row per sample, with what the step added
  or changed tinted and what it removed struck through, beside what
  reached it; or each sample as JSON. The settings panel holds settings
  only.
- Each step shows in → out from the dry run, and each connection what
  crossed it; a step that takes samples and passes none on is amber.

Needs stareSDK 0.2.32 (Tray, RecordTable).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Map makes observations out of records, so every field would show as
changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the tray (#104)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Clicking a sample in the tray follows it: the steps it passed light up,
the connections it took are drawn in the accent colour, the step that
dropped or held it says why, and the steps it never reached fade. The
tray says which sample is followed; clicking it again, or the badge's
remove, stops.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The step list shows Map, Filter and Throttle, then the outputs, with
  Reject, Entity links, Affiliation and Tracker under More steps.
- A step dropped onto a connection it can sit in goes between its two
  steps; one dropped just right of a step's output that can feed it is
  connected from there. Tidy lays the flow out left to right.
- The input's card offers Capture samples while there are none; an empty
  Map's card offers Use suggested mapping once a capture suggested one.
- A field's column in the input's samples can be dragged onto a Map: it
  is mapped where the capture suggested, else where its name usually goes
  (lat, mmsi, ShipName…), else to an extension field of its own, and the
  Map opens to check.

Needs stareSDK 0.2.33 (RecordTable draggableColumns).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s name (#104)

No feed's field names in the editor: a field dropped on a Map goes where
the probe's analysis of the captured samples proposed, else to an
extension field named after it. The tests use neutral paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@phornstein phornstein added ui Web UI sources Sources, transports, codecs, pipelines labels Oct 28, 2026
phornstein and others added 5 commits October 27, 2026 19:51
Tray and RecordTable (0.2.32), RecordTable draggableColumns (0.2.33).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ames (#104)

The probe's suggestion had tables of particular feeds' field names
(sog, cog, shipname, mmsi, icao, hex, alt_baro…) and the units and
identifier schemes they implied. It now reads only the values and
OpenTrack's own field names:
- a position from two fractional coordinates in one object, the one
  past ±90 the longitude (the one with the most decimal places beside
  it the latitude); coordinates it cannot tell apart are asked about;
- observed_at from fields of times within the last ten years;
- a CoT type from its form;
- the key from fixed-length tokens, the most distinct;
- a field named like an OpenTrack field (from the schema's targets,
  with or without its unit) whose values fit, assuming OpenTrack's unit.
It suggests no identifiers or unit conversions. The tests use neutral
fixtures; two comments no longer name a feed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… only (#104)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@phornstein
phornstein marked this pull request as ready for review October 28, 2026 01:08
Base automatically changed from feat/104-follow-sample to main November 2, 2026 19:18
…ding

# Conflicts:
#	crates/ot-source/src/pipeline.rs
#	ui/src/index.css
#	ui/src/pages/sources/SourceEditor.tsx
#	ui/src/pages/sources/flow/SamplesTray.tsx
#	ui/src/pages/sources/flow/StepNode.tsx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sources Sources, transports, codecs, pipelines ui Web UI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant