Recover mobile screenshot paste through an explicit clipboard action

docs/engineering/mobile_screenshot_paste_2026-09-14.md

@@ -0,0 +1,77 @@
+# Android screenshot paste: 14 September 2026 observations
+
+- **Kind:** Incident evidence record
+- **Lifecycle:** Frozen
+- **Authority:** Evidence only; delivery status belongs to the native task
+- **Owner:** Michael Witbrock / Codex DGX
+- **Evidence as of:** 14 September 2026
+- **Task:** `#V#task_agent_b61e77311c89e38adcf879ff20c995b1`
+- **Review trigger:** A changed Android Chrome input implementation or new native evidence
+
+## Diagnosis
+
+The supplied original screenshot shows Gboard Clipboard, a screenshot thumbnail,
+the main “Talk with Von here…” composer, and Chrome's “Chrome does not support
+image pasting here” toast. It is not the person/agent Messages reply composer.
+The reported phone is a Pixel 8 Pro, Android 17, Chrome 152; its exact installed
+build and feature state were not independently observed.
+
+The existing main-composer handler already extracts image files from
+`clipboardData.items`, falls back to `clipboardData.files`, and cancels default
+paste only after it captures an image. Adding another extractor cannot repair
+an insertion rejected before a DOM event.
+
+Native operator evidence on Android 17, Chrome 145.0.7632.218 and Gboard
+17.2.2.895242737-preload-arm64-v8a reproduced the refusal with zero image/paste/
+input events in the production textarea and experimental rich control. Captions
+and native multiline text paste survived. Native Documents selection reached
+the uploader. The rich control's IME `contentMimeTypes` was null.
+
+Pinned Chromium 152.0.7977.127 source has two relevant conditions:
+
+1. The Java input connection advertises media MIME types only when
+   `ANDROID_MEDIA_INSERTION` is enabled:
+   [ImeAdapterImpl.java](https://github.com/chromium/chromium/blob/671995b43301fa179149ff18b1952efad92abe60/content/public/android/java/src/org/chromium/content/browser/input/ImeAdapterImpl.java#L567).
+2. Native MIME discovery returns image types only for a richly editable element:
+   [ime_adapter_android.cc](https://github.com/chromium/chromium/blob/671995b43301fa179149ff18b1952efad92abe60/content/browser/android/ime_adapter_android.cc#L569).
+
+The feature is [disabled by default in that source](https://github.com/chromium/chromium/blob/671995b43301fa179149ff18b1952efad92abe60/content/public/common/content_features.cc#L56).
+The retained 145 source has the same gate. Von's textarea does not qualify as
+richly editable; changing it alone also leaves the feature gate. Null MIME types
+and zero DOM events are consistent with the feature being disabled on the
+emulator, but its effective field-trial state was not read. Source defaults do
+not prove the phone's effective feature state.
+
+## Explicit clipboard recovery evidence
+
+A second native operator trial used the exact `initialiseImagePicker` helper
+from PR #658 at `c9ef82f098ba552425a39763e726187eecaf5e87`. A trusted native
+**Paste image** tap produced Chrome's normal clipboard permission prompt.
+Native **Allow** returned a 109-byte PNG, decoded as 24 × 16 pixels, with all
+captions preserved. No permission pre-grant, synthetic paste, changed Chrome
+flag, backend upload, or model request was used in that trial. Chromium
+re-encoded the clipboard PNG; byte identity with the copied PNG is not claimed.
+
+The helper can therefore bypass the keyboard's unsupported insertion route.
+The task reuses that helper in the main composer's More actions menu and its
+existing upload callback. PR #658's broader picker layout, cancellation changes
+and PWA acceptance remain separate work.
+
+## Provenance and limits
+
+Private originals, exported reports and screenshots remain on the native task,
+not in Git. Canonical file-copy reads supplied bytes; all five supplied SHA256
+hashes were checked in run `dec7adcb7cd441ce8a36020eeb068285`.
+
+| Source | File-copy concept | SHA256 |
+| --- | --- | --- |
+| Original screenshot | `#V#computer_file_copy_f8a50dc5cf4e4a8591c89099cfb73f33` | `6204bdce1672cc7bb313a7588207a712e634fb0ad379e629fcc5c3eab35dbf3f` |
+| Native Gboard report | `#V#computer_file_copy_9fe8f1351c4d4919847e3533f8f29d3a` | `a0773217150eba450849fe784942b14a533c5c73372fb2b4617ee916916650b2` |
+| Native Clipboard API evidence ZIP | `#V#computer_file_copy_085ec20da79c402da44f42441c120990` | `4aa3bcf2f6d397d75cf4311362f160b5f6ffc9902a0922cf41c136068ee5b197` |
+
+The native capability trial and local production-composer integration checks
+are separate observations. Neither establishes physical Pixel/Chrome 152,
+full authenticated Von/PWA operation, public OAuth, production deployment, or
+removal of the original Gboard toast. The source conversation had no accessible
+locator; the canonical task and supplied files sufficed without inferring its
+contents.

docs/engineering/otter_archive_and_conversation_images.md

@@ -38,6 +38,18 @@ text. Consumers reassembling originals must check the complete upstream hash.
 
 ## Reusable image handling
 
+In the main conversation composer, open **More actions** (the **+** on a phone),
+then choose **Paste image** to read a copied screenshot through the browser's
+clipboard permission prompt. **Attach image** opens the native image picker if
+clipboard access is unavailable, denied, or contains no image. Both preserve the
+caption and use the existing upload/preview path; review the preview before
+sending. Ordinary text paste still belongs to the message field.
+
+Android keyboards can refuse their own image-tile insertion before Von receives
+a paste event. The explicit action is a recovery for that browser limitation;
+it does not enable Gboard's image tile. See the
+[native observations and diagnosis](mobile_screenshot_paste_2026-09-14.md).
+
 The existing drop, paste and picker routes feed one image pipeline. Up to eight still PNG, JPEG or WebP
 images may accompany a message (8 MiB and 25 million pixels per image). Media is decoded rather than
 trusted by filename. The composer shows progress, previews, removal and failures; unresolved preparation

src/frontend/web/von_interface/static/js/chatTab.js

@@ -14,7 +14,7 @@ import { CONVERSATION_LAYOUT_KEY, CONVERSATION_LAYOUT_KEYS, CONVERSATION_NARROW_
 import { createVoiceConversation } from './voiceConversation.js';
 import { getClientContext } from './clientContext.js';
 import { createDictationController } from './dictation.js';
-import { renderImageAttachments, renderImageComposer, uploadConversationImage, imagesBlocked, imageItems, takeImages, restoreImages, descriptorsForIds } from "./utils/conversationImages.js";
+import { initialiseImagePicker, renderImageAttachments, renderImageComposer, uploadConversationImage, imagesBlocked, imageItems, takeImages, restoreImages, descriptorsForIds } from "./utils/conversationImages.js";
 // Chat Tab Module
 import { annotateTurn, fetchWithTimeout, getJsonDetailed, getUserContext, getWindowSessionId, postJson, WINDOW_SESSION_HEADER } from './apiService.js';
 import {
@@ -6667,7 +6667,7 @@ function updateExternalConversationUi() {
                 : promptInput.dataset.nativePlaceholder
         );
     }
-    [resetButton, uploadButton, dictateButton].forEach((button) => {
+    [resetButton, uploadButton, dictateButton, document.getElementById('attachImageButton'), document.getElementById('pasteImageButton')].forEach((button) => {
         if (button) {
             button.disabled = readOnly;
         }
@@ -34020,6 +34020,8 @@ export function initializeChatTab() {
         });
     }
 
+    initialiseImagePicker(uploadFilesToVon, renderUploadStatus);
+
     if (uploadFileButton && uploadFileInput) {
         uploadFileButton.addEventListener('click', () => {
             try {

src/frontend/web/von_interface/static/js/test/conversationImages.test.js

@@ -24,7 +24,7 @@ test('failed preparation remains visible and prevents text-only submission until
     expect(panel.querySelector('[role=alert]').textContent).toMatch(/still PNG/);
     expect(imagesBlocked('failed')).toBe(true);
     expect(() => takeImages('failed')).toThrow(/preparation failed/);
-    panel.querySelector('button').click();
+    panel.querySelector('button[aria-label="Remove bad.svg"]').click();
     expect(imagesBlocked('failed')).toBe(false);
 });
 

src/frontend/web/von_interface/static/js/test/imagePicker.test.js

@@ -0,0 +1,37 @@
+/** @jest-environment jsdom */
+import { initialiseImagePicker } from '../utils/conversationImages.js';
+let upload, status;
+beforeEach(() => {
+    document.body.innerHTML = '<button id="attachImageButton"></button><input id="attachImageInput" type="file"><button id="pasteImageButton"></button>';
+    upload = jest.fn(); status = jest.fn();
+    Object.defineProperty(navigator, 'clipboard', {configurable:true, value:undefined});
+    initialiseImagePicker(upload, status);
+});
+test('picker cancellation preserves draft; selection uses shared upload path', () => {
+    const input = document.querySelector('input');
+    const click = jest.spyOn(input, 'click');
+    document.querySelector('#attachImageButton').click();
+    expect(click).toHaveBeenCalled();
+    input.dispatchEvent(new Event('cancel'));
+    expect(upload).not.toHaveBeenCalled();
+    expect(status).toHaveBeenCalledWith(expect.stringContaining('unchanged'));
+    const file = new File(['image'], 'photo.png', {type:'image/png'});
+    Object.defineProperty(input, 'files', {value:[file]});
+    input.dispatchEvent(new Event('change'));
+    expect(upload).toHaveBeenCalledWith([file]);
+    expect(input.value).toBe('');
+});
+test('missing clipboard API offers the picker', () => {
+    document.querySelector('#pasteImageButton').click();
+    expect(status).toHaveBeenCalledWith(expect.stringContaining('Use Attach image'), 'error');
+});
+test.each(['denied', 'empty', 'image'])('clipboard %s', async mode => {
+    Object.defineProperty(navigator, 'clipboard', {value:{read:jest.fn(async () => {
+        if (mode === 'denied') throw new Error('NotAllowedError');
+        return mode === 'empty' ? [] : [{types:['image/png'], getType:async () => new Blob(['image'], {type:'image/png'})}];
+    })}});
+    document.querySelector('#pasteImageButton').click();
+    await new Promise(resolve => setTimeout(resolve, 0));
+    if (mode === 'image') expect(upload.mock.calls[0][0][0].type).toBe('image/png');
+    else expect(status).toHaveBeenCalledWith(expect.stringContaining('Attach image'), 'error');
+});

src/frontend/web/von_interface/static/js/utils/conversationImages.js

@@ -142,6 +142,39 @@ async function uploadItem(item, file, sessionId, headers, changed) {
     changed();
     return Boolean(item.descriptor);
 }
+// Both controls use the existing session-bound upload path supplied by chatTab.
+export function initialiseImagePicker(upload, status) {
+    const input = document.getElementById('attachImageInput');
+    const button = document.getElementById('attachImageButton');
+    const paste = document.getElementById('pasteImageButton');
+    if (button && input) {
+        button.addEventListener('click', () => input.click());
+        input.addEventListener('change', () => {
+            const files = Array.from(input.files || []);
+            input.value = '';
+            if (files.length) void upload(files);
+        });
+        input.addEventListener('cancel', () => status('Image selection cancelled. Your draft is unchanged.'));
+    }
+    paste?.addEventListener('click', async () => {
+        if (!navigator.clipboard?.read) {
+            status('Clipboard images are unavailable here. Use Attach image, or paste into the message field.', 'error');
+            return;
+        }
+        try {
+            const items = await navigator.clipboard.read();
+            const files = [];
+            for (const item of items) {
+                const type = item.types.find(value => value.startsWith('image/'));
+                if (type) files.push(new File([await item.getType(type)], `pasted-image-${files.length + 1}.${type.split('/')[1]}`, {type}));
+            }
+            if (files.length) await upload(files);
+            else status('No image found on the clipboard. Use Attach image to choose a photo.', 'error');
+        } catch (_) {
+            status('Clipboard access was unavailable or cancelled. Use Attach image, or paste into the message field.', 'error');
+        }
+    });
+}
 export function takeImages(sessionId) {
     if (imagesBlocked(sessionId)) throw new Error('Attachment preparation failed or is still running. Remove or finish the upload before sending.');
     const ids = imageItems(sessionId).map(x => x.descriptor.concept_id);

src/frontend/web/von_interface/static/styles.css

@@ -18566,7 +18566,7 @@ body.settings-body {
         gap: 8px; margin: 4px 0; padding: 8px 0;
         border-top: 1px solid #e2e8f0;
     }
-    .compact-speech-help { display: block; flex-basis: 100%; font-size: 12px; margin: 0; }
+    .compact-speech-help, #imageAttachmentHelp { display: block; flex-basis: 100%; font-size: 12px; margin: 0; }
     .chat-composer-more-actions .button-row > button,
     .chat-composer-more-actions .button-row > select { min-height: 44px; margin: 0; }
     .chat-composer-more-actions #dictateButton { width: auto; flex-basis: auto; padding: 0 12px; border-radius: 6px; gap: 6px; }

src/frontend/web/von_interface/templates/chat_tab.html

@@ -256,6 +256,10 @@ <h3 id="conversationReferenceInspectorTitle" class="chat-reference-inspector-tit
               title="End this conversation (resets Von's context while keeping the conversation view)">End
               conversation</button>
 
+            <p id="imageAttachmentHelp">If your keyboard cannot paste a screenshot, use Paste image and allow clipboard access, or choose Attach image. Your caption stays in the draft.</p>
+            <button id="pasteImageButton" class="btn btn-secondary" type="button" aria-describedby="imageAttachmentHelp">Paste image</button>
+            <button id="attachImageButton" class="btn btn-secondary" type="button" aria-describedby="imageAttachmentHelp">Attach image</button>
+            <input id="attachImageInput" class="visually-hidden" type="file" tabindex="-1" accept="image/png,image/jpeg,image/webp" multiple aria-label="Choose images" />
             <button id="uploadFileButton" class="btn btn-secondary" type="button"
               title="Upload a file, drag and drop onto the conversation, or paste an image">Upload File</button>
             <input id="uploadFileInput" class="visually-hidden" type="file" multiple aria-label="Upload file" />