Skip to content

feat(video): expose native rendering mode transitions on Darwin - #41

Open
sibogentleman wants to merge 3 commits into
Predidit:mainfrom
sibogentleman:codex/ios-rendering-api
Open

sibogentleman wants to merge 3 commits into
Predidit:mainfrom
sibogentleman:codex/ios-rendering-api

Conversation

@sibogentleman

@sibogentleman sibogentleman commented Oct 11, 2026 •

Copy link
Copy Markdown

iOS sample-buffer Picture in Picture needs to switch an existing player's output between GPU rendering and software rendering while keeping its media, audio, position, and decoder settings. This adds a typed native API published through Flutter's plugin registry so integrations can request that transition without intercepting the plugin's method channel or replaying create/dispose messages.

The plugin owns output configuration, dimensions, replacement readiness, rollback, and disposal. The caller suspends and restores the player's video track and serializes that transaction. Results distinguish changed, unchanged, temporarily not ready, and failed requests. iOS also gates GPU rendering when the application resigns active. The README documents the API and its lifecycle requirements.

Validation:

  • 14 deterministic native lifecycle/thread tests pass, using the real manager/worker with controlled texture doubles.
  • Kazumi integration: 261 Flutter tests pass; iOS release and signed-device builds succeed. The contributor confirmed testing build 2.3.8 (20310) on iPhone 16 Pro / iOS 27.0.1.
  • 16 integration XCTest cases compile; execution remains blocked by the local CoreSimulator runtime error.
  • macOS shares initialization/error handling changes, but a macOS app build and device regression have not been performed. Real GPU transitions and performance are not established by the deterministic harness.

Companion integration: Predidit/Kazumi#2646. This draft is for review of the proposed public API before the integration switches back from the pinned fork commit to an upstream commit.

AI assistance: implementation and automated validation were assisted by Codex; device testing was performed by the contributor.

以下可直接用作中文 PR 说明,验证结果按[当前 PR #41](#41) 的记录整理。

标题: feat(video): 为 Darwin 平台提供原生渲染模式切换 API

为 iOS 基于 Sample Buffer 的画中画接入提供底层支持,允许现有播放器在 GPU 渲染与软件渲染之间切换,同时保留媒体、音频、播放位置和解码器配置。

通过 Flutter 插件注册表公开类型化的原生 API,接入方无需拦截插件的 Method Channel,也无需模拟创建与销毁消息。主要改动包括:

  • 支持切换现有视频输出的渲染模式,以及为后续创建的输出设置软件渲染默认值;恢复时遵循原有硬件渲染配置。
  • 插件统一管理输出配置、尺寸、纹理替换、初始化失败回退和资源释放;调用方负责按播放器串行执行视频轨暂停与恢复。
  • 区分切换成功、无需切换、暂未就绪和失败四种结果。切换完成表示上下文初始化与纹理注册完成,不代表首帧已渲染。
  • 在 iOS 应用失去活跃状态时限制 GPU 渲染操作,并处理恢复 GPU 期间进入后台的情况。
  • 补充原生 API 文档、生命周期约束和测试。软件渲染期间不应用 GPU Shader 效果,解码器兼容性由接入方保证。

验证情况:

  • 14 项原生生命周期与线程确定性测试通过。
  • Kazumi 集成侧 261 项 Flutter 测试通过,iOS Release 构建和真机签名构建成功;贡献者已在 iPhone 16 Pro / iOS 27.0.1 上测试版本 2.3.8(20310)。
  • 16 项集成 XCTest 用例已编译,执行仍受本地 CoreSimulator 运行时错误阻塞。
  • macOS 应用构建与设备回归尚未完成;确定性测试不覆盖真实 GPU 切换与性能验证。

配套画中画接入见 [Predidit/Kazumi#2646](https://github.com/Predidit/Kazumi/pull/2646)。本 PR 保持草稿状态,用于先评审公开 API;上游合入后,接入方再将固定的 fork 提交依赖切换至上游提交。

实现与自动化验证由 Codex 辅助完成,真机测试由贡献者执行。

@sibogentleman

Copy link
Copy Markdown
Author

@Predidit 为了解决 hack 接口的更改

@sibogentleman
sibogentleman marked this pull request as ready for review October 12, 2026 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant