Show LabelBOT popup despite no suggestions by gkourie · Pull Request #1478 · biigle/core

gkourie · 2026-05-29T11:49:15Z

When LabelBOT returns no suggestions, an empty array will be sent to the front end. The LabelBOT popup will show a message to type a label in the typeahead. For now the temp annotation will be saved in the database without a label and it will have the editing style in the front end until it's confirmed with a label, if not then it will be deleted.

Copilot

Pull request overview

This PR changes LabelBOT’s empty-result behavior so annotations can still be created when no suggestions are found, then prompts the user to manually choose a label or dismiss the temporary annotation.

Changes:

Backend LabelBOT now returns an empty label list instead of throwing a 404.
Image/video annotation creation can save annotations without immediately attaching a label.
LabelBOT popup now displays an empty-suggestions message and supports manual label selection/deletion flow.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`app/Services/LabelBot/LabelBotService.php`	Returns an empty suggestion array when no similar labels are found.
`app/Http/Controllers/Api/ImageAnnotationController.php`	Skips label authorization/attachment when no label is available.
`app/Http/Controllers/Api/VideoAnnotationController.php`	Mirrors the image annotation empty-label persistence behavior.
`resources/assets/js/annotations/components/labelbotPopup.vue`	Adds no-suggestions UI and temporary annotation deletion behavior.
`resources/assets/js/annotations/annotatorContainer.vue`	Refreshes annotations after manually attaching a label when no previous label existed.
`resources/assets/sass/annotations/components/_labelbot-popup.scss`	Adds styling for the no-suggestions popup message.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+            if (!this.selectedLabel || this.selectedLabel.id !== label.id) {
+                this.dismissLabelbotAnnotation = false;
                this.$emit('update', {label: label, annotation: this.annotation});


+        if (this.noLabels && this.dismissLabelbotAnnotation) {
+            this.deleteLabelAnnotation();



        if (empty($topNLabels)) {
-            throw new NotFoundHttpException("LabelBOT could not find similar annotations.");
+            return [];


Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

+        $response->assertSuccessful();
+
+        // We expect an empty array
+        $response->assertJsonFragment([]);


            this.emitClose();
-            Events.emit('labelbot.chose_label_1');
+            if (this.noLabels) {
+                this.deleteLabelAnnotation();


+        if (this.noLabels && this.deletePendingLabelbotAnnotation) {
+            this.deleteLabelAnnotation();


+            if ($label) {
+                VideoAnnotationLabel::create([
+                    'label_id' => $label->id,
+                    'user_id' => $request->user()->id,
+                    'annotation_id' => $annotation->id,
+                ]);
+            }


mzur · 2026-06-01T09:23:34Z

Please request my review again once you are through with the AI.

gkourie · 2026-06-02T12:14:06Z

@mzur While trying to implement this for videos, I found some other issues:

When LabelBOT returns no results, the video disappears (this should be fixed with empty LabelBOT popup):

Screencast.from.2026-06-02.13-58-18.webm

If LabelBOT popup is shown and the video is seeked, then the popup and its line feature are shown on the next frames without the annotation:

Screencast.from.2026-06-02.14-04-56.webm

Or was that intentional for 2 ? Otherwise, I could try to fix it here.

mzur

Good that you fixed issue 1. Regarding issue 2 I feel that we thought about this when LabelBOT was implemented for videos but I might be wrong (or there was a regression). Please fix that too.

Comments below:

…estions

gkourie · 2026-07-31T11:24:03Z

No unlabeled annotations will be saved to the database anymore. The OpenLayers feature element is now temporarily attached to the annotation element. If LabelBOT returns no results, the feature element ID is used as the popup ID (key), and its geometry is used for popup placement. For video annotations, the pending annotation is attached in the same way.

The two issues mentioned previously should now be fixed. I also noticed that you, @mzur, already opened issue #1510 regarding the video screen crash.

Another issue I found was related to tracking annotations. When creating an ROI in a video, then seeking to another time, and finally clicking "Finish and Track Annotation" while LabelBOT is enabled, the video screen crashes. This happens because there are no available annotations to use as an anchor at the new current time:

Screencast.from.2026-07-31.12-17-16.webm

This was already fixed for non-tracking annotations, and I have now applied the same fix for tracking annotations as well.

When clicking the play button while a LabelBOT popup is open, the popup remains visible until the video is paused or the user interacts with the popup. If the popup is empty, pausing the video will remove the entire pending annotation if no label has been entered. If the popup is not empty, the Top-1 is already saved.

…estions

mzur

I merged the current master so the code works with my new DINO ONNX (generated with a newer Torch version).

I think the popup should close when there are no labels and the user clicks play. It closes anyway once the user clicks pause, so we can just close it earlier.

If the popup timeout is enabled, two Esc presses are needed to close the popup. It should close right away.

Maybe the whole logic of remove callbacks, direct calls to the OpenLayers source and delete-pending events can be consolidated a little more. They all do similar things but it feels very inconsistent.

There are several missing semicolons in the code. Please rerun the AI review in the next iteration before requesting my own review.

mzur · 2026-08-12T07:02:38Z

+    list-style-type: none;
+    max-width: 300px;
+    margin: 0;
+    padding: $padding-small-vertical;
+    background-color: $body-bg;
+    color: $text-muted;


Most is already handled by .labelbot-labels.

Suggested change

list-style-type: none;

max-width: 300px;

margin: 0;

padding: $padding-small-vertical;

background-color: $body-bg;

color: $text-muted;

padding: $padding-small-vertical;

color: $text-muted;

mzur · 2026-08-12T07:04:37Z

+            let annotationFeature;
+            if (this.noLabels) {
+                annotationFeature = this.annotation.feature;
+                this.pendingAnnotation = true;


This case should automatically focus the label typeahead when the popup opens.

mzur · 2026-08-12T07:19:09Z

+            pendingAnnotation.feature = undefined;
+            delete pendingAnnotation.feature;
+
+            const track = pendingAnnotation.track


Tracking video annotations is broken (and was before this PR) when LabelBOT is not enabled or did not return results. The track attribute is not passed to newAnnotation below.

mzur · 2026-08-12T07:19:39Z

+                // in case LabelBOT returns no results
+                newAnnotation.feature = pendingAnnotationFeature;
+                newAnnotation.pendingAnnotation = pendingAnnotation;
+                newAnnotation.track = track;


This can be removed when the track attribute is added to newAnnotation above.

Suggested change

newAnnotation.track = track;

mzur · 2026-08-12T07:28:46Z

-                return annotation;
-            });
+            .then((response) =>  {
+                if (response.status === 204) return null;


Please add a comment what this case means.

Thinking about this, maybe we should not use 204 after all, because it signals some kind of success. Maybe we should use 422 with a special error message for the feature_vector and then catch this and check the message here? A 422 will have better semantics here too because the promise chain will be skipped on an error response.

mzur · 2026-08-12T08:18:38Z

            this.lineFeature.set('unselectable', true);
-            this.lineFeature.set('color', this.labels[0].color);
+            if (this.noLabels) {
+                this.lineFeature.set('color', '5bc0de');


Add a comment that this is the "info" color.

mzur · 2026-08-12T08:29:37Z

                e.feature.set('color', '5bc0de');
                e.feature.setStyle(Styles.editing);

+                newAnnotation.feature = e.feature, // we need this temporarily if LabelBOT returns no results


Suggested change

newAnnotation.feature = e.feature, // we need this temporarily if LabelBOT returns no results

newAnnotation.feature = e.feature; // we need this temporarily if LabelBOT returns no results

mzur · 2026-08-12T08:30:51Z

            shouldHaveProgressBar: true,
            maybeGetsAttention: false,
            typeaheadFocused: false,
+            pendingAnnotation: false,


Better use:

Suggested change

pendingAnnotation: false,

isPendingAnnotation: false,

mzur · 2026-08-12T08:32:31Z

+        // Query an empty DB
+        $response = $this->post("/api/v1/images/{$this->image->id}/annotations", [
+            'shape_id' => Shape::pointId(),
+            'feature_vector' => range(1, 384),
+            'confidence' => 0.5,
+            'points' => [10, 11],
+        ]);
+        // We expect no content
+        $response->assertStatus(204);
+


This should be added as a separate test case.

mzur · 2026-08-12T08:32:38Z

+        // Query an empty DB
+        $response = $this->json('POST', "api/v1/videos/{$this->video->id}/annotations", [
+            'shape_id' => Shape::pointId(),
+            'feature_vector' => range(1, 384),
+            'points' => [[10, 11]],
+            'frames' => [0.0],
+        ]);
+        // We expect no content
+        $response->assertStatus(204);
+


gkourie linked an issue May 29, 2026 that may be closed by this pull request

Improve case with empty LabelBOT suggestions #1337

Open

gkourie requested review from Copilot and mzur May 29, 2026 11:49

Copilot started reviewing on behalf of gkourie May 29, 2026 11:49 View session

Copilot AI reviewed May 29, 2026

View reviewed changes

gkourie requested a review from Copilot May 29, 2026 16:44

Copilot started reviewing on behalf of gkourie May 29, 2026 16:45 View session

Copilot AI reviewed May 29, 2026

View reviewed changes

mzur removed their request for review June 1, 2026 09:23

gkourie requested a review from mzur June 2, 2026 13:34

mzur requested changes Jun 10, 2026

View reviewed changes

Fix empty LabelBOT popup logic

ac0a62a

gkourie force-pushed the 1337-improve-case-with-empty-labelbot-suggestions branch from 207345e to ac0a62a Compare June 18, 2026 18:43

gkourie added 6 commits June 18, 2026 20:48

Merge branch 'master' into 1337-improve-case-with-empty-labelbot-sugg…

a612914

…estions

Fix temp feature removal at popup close

621dcf5

Add temp feature to the new annotation only if LabelBOT is active

9b22774

Set empty popup line feature color to info

1cb402b

Handle empty LabelBOT popup for video

76f1514

Merge branch 'master' into 1337-improve-case-with-empty-labelbot-sugg…

3bac2c5

…estions

gkourie requested a review from mzur July 31, 2026 11:24

mzur linked an issue Aug 12, 2026 that may be closed by this pull request

LabelBOT crashes with videos and no suggestions #1510

Open

Merge branch 'master' into 1337-improve-case-with-empty-labelbot-sugg…

da8268a

…estions

mzur requested changes Aug 12, 2026

View reviewed changes

		if (this.noLabels && this.dismissLabelbotAnnotation) {
		this.deleteLabelAnnotation();

		if (this.noLabels && this.deletePendingLabelbotAnnotation) {
		this.deleteLabelAnnotation();

	newAnnotation.feature = e.feature, // we need this temporarily if LabelBOT returns no results
	newAnnotation.feature = e.feature; // we need this temporarily if LabelBOT returns no results

Uh oh!

Conversation

gkourie commented May 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

mzur commented Jun 1, 2026

Uh oh!

gkourie commented Jun 2, 2026

Uh oh!

mzur left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gkourie commented Jul 31, 2026

Uh oh!

mzur left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mzur left a comment •

edited

Loading