Make Your RmlUi Interfaces Accessible to Users of Assistive Technologies!

rmlui-accessibility is an accessibility plugin for RmlUi. It translates RmlUi documents and elements into an AccessKit tree which can be read by screen readers and other assistive technologies, while preserving RmlUi's visual rendering, input handling and application-defined widget behavior. The aim is to create a plugin that can be integrated into any application using RmlUi, automatically expose RmlUi elements using the platform's native accessibility APIs and allow developers to customise how their documents and elements are presented to assistive technologies.

The plugin currently exposes:

• roles, names, descriptions, states, values, common ARIA type relations and focus for all of RmlUi's core elements;
• actions for buttons, form controls, selections and editable text;
• single-line and multiline text editing, selection and caret metadata;
• application and browsable-document presentation modes;
• incremental live-region and direct announcements;
• semantic markup elements such as images, links, lists, tables, landmarks, application-defined disclosures, form controls, progress indicators and tab sets; and
• multiple documents and multiple RmlUi contexts/windows.

The plugin connects to AccessKit's native adapters on Linux and the BSDs, Windows, macOS, iOS, tvOS, visionOS, watchOS and Android.

AI Assisted Development Notice

This project was developed with substantial assistance from OpenAI Codex. The AI was used to write a significant amount of the implementation and documentation. In the name of transparency, this is intentionally not hidden from the project's history.

I designed the architecture and major implementation details myself, often (but not always) through deep discussion and iteration with the model, and made the final decisions about what was implemented. I did not give an agent a high-level specification and accept whatever it produced. In several cases the resulting design differs substantially from the approaches initially proposed by the AI.

I treated the coding agent roughly as a very fast implementation and pair-programming tool: I reviewed proposed changes; I iteratively refined designs before implementation; and I instructed the AI to conduct regular reviews of the codebase, refactor bad code and fix any found problems.

Fairly early on, I also got the AI to build an end-to-end automated test suite, which covers both its AT-SPI accessibility tree representation and real interactions with the Orca screen reader inside an isolated environment. Since I develop mostly on Linux, this test suite makes use of Linux-specific technologies and has received little manual code review. It likely needs a major cleanup and integration tests should be added for other platforms. If you'd like to help with this, I'd love to hear from you!. I also manually tested functionality as features were added.

Before the initial public release, I manually reviewed the entire codebase. I read the implementation in full, recorded issues I found, had them corrected and verified the resulting changes myself.

I therefore take responsibility for the code in this repository. AI assistance should not be taken to mean that the implementation is unreviewed or that I am unable to explain, maintain or modify it.

Integration

1. Initialize the Plugin

Link the RmlUi::Accessibility CMake target and construct Rml::AccessibilityBridge after Rml::Initialise, but before creating any contexts. The bridge registers itself as an RmlUi plugin and installs the accessible instancers required for documents and native controls.

Rml::Initialise();

Rml::AccessibilityBridge accessibility_bridge(
    // Provide a callback to wake the host app's event loop; this will be called from an AccessKit thread
    [](Rml::AccessibilityContext* accessibility) {
        WakeHostEventLoop(accessibility);
    });
// If you want standard semantic markup elements such as headings and lists to expose their normal accessibility semantics under their regular tag names (h1, ul, li, etc), call this convenience 
// function. Otherwise, you can register accessible element instancers manually.
accessibility_bridge.RegisterStandardElementSemantics();

Rml::Context* context = Rml::CreateContext("main", dimensions);
// Each RmlUi Context is associated with a plugin provided AccessibilityContext
Rml::AccessibilityContext* accessibility = accessibility_bridge.GetAccessibilityContext(*context);

Rml::ElementDocument* document = 
    context->LoadDocument("interface.rml");
document->Show();

AccessibilityBridge::RegisterStandardElementSemantics opts conventional tags such as <a>, <ul>, <table> and landmarks into their HTML-like accessibility semantics. Applications may instead register only the exact tag names they use, or derive custom widgets from the corresponding accessible element class.

Each AccessibilityContext owns an independent AccessKit adapter, window root and action queue, as well as internal accessibility state, but The wake callback is bridge-global. Retrieve a handle to an AccessibilityContext with AccessibilityBridge::GetAccessibilityContext, forward its physical window's focus changes through AccessibilityContext::SetWindowFocus, and keep the bridge alive until Rml::Shutdown has delivered plugin shutdown notifications.

2. Connect The Accessibility Tree to Its Window/View

On all platforms except Unix (Linux/BSD), you must attach each accessibility context to the native window or view supplied by the host system, so that AccessKit can associate the exposed tree of elements with the right window. This API uses opaque pointers so we don't have to include platform-specific headers in the plugin's public interface.

// For example, on Windows. Attach before showing or focusing the HWND.
accessibility->AttachPlatformHost(hwnd);

Windows expects an HWND and attachment on its owning thread. macOS expects an NSWindow* and iOS-family targets expect a UIWindow*; attach Apple hosts on the platform main thread after the macOS content view or iOS root view controller is installed. Android expects the View as a JNI jobject and a temporary JNIEnv* as the second argument, which is not retained. The native host must outlive its attachment, so call AccessibilityContext::DetachPlatformHost before the window or view is destroyed.

3. Integrate With Your Event Loop

AccessKit may request that actions performed by an assistive technology be handled from a worker thread. The bridge queues those requests; you must provide a wake callback, which notifies the host event loop and must not call RmlUi or dereference its accessibility-context argument. The pointer to AccessibilityContext is provided so you can route the wake-up to the correct window/view, if necessary.

On the RmlUi thread, process each context in this order:

1. Dispatch native input to RmlUi, such as keyboard, mouse and touch input.
2. Call Context::Update.
3. Call AccessibilityContext::ProcessPendingActions.
4. If it returns true, then actions were handled, so call Context::Update again.
5. Call AccessibilityContext::Update.
6. Render normally.

The second context update is intentional. Accessibility actions can mutate controls, while processing them before the first update can allow stale native state to overwrite caret or selection changes.

Note on Element and Document Subclassing

Documents must be instances of Rml::AccessibleElementDocument. If you do no manual document subclassing, the bridge's default body instancer supplies this automatically. Otherwise, a custom document instancer must derive its document from that class. Inaccessible documents are reported and ignored so you can migrate document subclasses incrementally.

Likewise, to provide accessibility information, each native RmlUi element is subclassed with the help of AccessibleElementAdapter. The bridge automatically installs instancers for the core RmlUi elements to provide accessible subclasses. All accessible elements must derive from a template specialisation of this class.

Authoring Documents

All standard RmlUi elements should work out of the box, as long as you did not override their instancers. Set data-a11y-presentation="application" on <body> for a more native application style of navigation. Use "document" instead to allow assistive technologies to browse the text of a document like a webpage. Changing the attribute at runtime is supported, though at present, the plugin is unable to provide full browse-mode semantics for document presentation (it only exposes the raw text).

The bridge recognizes common ARIA attributes like labels and descriptions, roles, states such as hidden/required/read-only/invalid/expanded/selected, relations (aria-labelledby etc), live regions, popup metadata, aria-level and numeric values. Native RmlUi state such as the value of an <input> remains authoritative where a control already supplies it.

Additional Documentation

See Accessible semantics for the supported tags and attributes, and Architecture for ownership, invalidation, threading, and multi-document design details. Known upstream limitations and proposed fixes are recorded under future patches, but these were written with AI and need verifying and rewriting before I submit them upstream.

If you'd like to help to document this project, I'd love a PR!

Build

cmake --preset debug
cmake --build --preset debug
./build/debug/rmlui_accessibility_example

Use the release preset for an optimized build. CPM.cmake prefers compatible system packages and otherwise downloads pinned versions of RmlUi and AccessKit if you do not already have them.

When this repository is the top-level project, it also builds the SDL3/SDL_gpu example by default, downloading SDL3, SDL3_image and FreeType as needed. Set RMLUI_ACCESSIBILITY_BUILD_EXAMPLE=OFF to build only the host-independent plugin. The example backend is available as RmlUi::AccessibilitySDLBackend; it is not part of the plugin

For Android source builds, the bundled AccessKit dependency embeds its delegate DEX so the plugin can attach to an arbitrary View. A separately installed AccessKit C package must have been built with equivalent ACCESSKIT_ANDROID_EMBEDDED_DEX support.

Image accessibility currently depends on RmlUi's internal ElementImage. Consequently, AccessibleElementImage.h is usable in source builds but is excluded from installation until RmlUi exposes that class publicly. See the future RmlUi patch notes about this issue.

Tests

On Linux:

ctest --preset debug

The integration harness starts private Xvfb, D-Bus and AT-SPI services so it does not interfere with your desktop session. The AT-SPI suite checks the observable tree, actions, focus, relations, values, text, selection, document lifecycle, visibility and stacking. The separate Orca suite validates speech and keyboard interaction when Orca and its Python modules are available. The AT-SPI suite also creates a second context, routes actions and announcements through its adapter, destroys it and verifies the first context remains live.

License

This project is licensed under the MIT license.