Add ScanFilter.withServiceData to match advertising Service Data (#57)

AGENTS.md

@@ -47,6 +47,18 @@ When modifying any device ID handling or API methods taking or returning device
 
 ---
 
+## Advertisement Field Normalization
+
+Scan results normalise their UUID-bearing fields in the Dart layer, so the same device compares equal whichever platform reported it. Do not re-implement this per platform.
+
+- **`BleDevice.services`** — the advertised service UUIDs (advertising data types `0x02`/`0x03`, `0x04`/`0x05`, `0x06`/`0x07`). Emitted as full 128-bit lower-case UUIDs on every platform. Platforms report their own form (Apple, for instance, reports a 16-bit UUID as `180D`), so `BleDevice` normalises them via `BleUuidParser`.
+- **`BleDevice.serviceData` keys** — the *Service Data* data types (`0x16`/`0x20`/`0x21`), normalised the same way. The values are the raw payload bytes and are never transformed.
+- Entries that are not valid UUIDs are kept verbatim rather than dropped; matching simply skips them, so malformed advertisement data cannot fail a scan.
+- **Inputs** accept any case and short (`181A`), `0x`-prefixed, or full 128-bit form, canonicalised with `BleUuidParser`/`toValidUUIDList()` at the boundary. Native code and the filter matchers only ever see the full form.
+- When changing anything that emits or compares these fields, test cross-form equivalence (short, full, `0x`-prefixed, mixed case). When scan filter semantics change, update the shared conformance fixture `test/fixtures/scan_filter_cases.tsv`, which drives the Dart, Kotlin and C++ filter implementations.
+
+---
+
 ## Architecture & Code Conventions
 
 1. **Platform Interface Pattern:**

CHANGELOG.md

@@ -1,3 +1,10 @@
+## 3.1.0
+* **Breaking:** `BleDevice.services` is now normalised to full 128-bit lower-case UUIDs on every platform, matching the keys of `BleDevice.serviceData`. Apple was the outlier: it reported 16-bit advertised service UUIDs in their short form (`180D`), so the same field had a different shape per platform and stored keys or comparisons did not transfer between them. Compare against, or store, the 128-bit form; short (`180D`) and `0x`-prefixed forms are still accepted as *input*. Entries that are not valid UUIDs are kept verbatim rather than dropped.
+* Add `ScanFilter.withServiceData` to discover devices that carry a service UUID inside their advertising Service Data (the `0x16`/`0x20`/`0x21` advertising data types) — beacons such as Eddystone (`0xFEAA`) or the ATC_MiThermometer (`0x181A`) — which `withServices` cannot match, because it only looks at the advertised service UUID list (#57). The matching payloads are reported in `BleDevice.serviceData`. On Android and Apple this switches the scan to the software filter path (the platform cannot filter on Service Data), so more advertisements reach the app process; on Web it throws `notSupported`.
+* Apple: compare service UUIDs in their 128-bit form when the custom filter path is active (`withNamePrefix`/`withManufacturerData`), instead of relying on `CBUUID` equality, which does not match a 16-bit advertised service UUID against the full 128-bit form the API sends. `ScanFilter.withServices` can now filter 16-bit services on that path.
+* Linux: re-evaluate the scan filter against the device's current properties instead of the snapshot taken when BlueZ added it, and keep listening to a device that did not match at that point. A peripheral that only advertises its Service Data (or UUIDs) in a later advertisement was previously dropped permanently by any filter.
+* Add `BleDevice.serviceData` documentation to the README (the API existed but was undocumented).
+
 ## 3.0.2
 * Fix the broken "Try it online" demo URL in the README.
 

README.md

@@ -135,6 +135,7 @@ UniversalBle.startScan();
 UniversalBle.startScan(
   scanFilter: ScanFilter(
     withServices: ["SERVICE_UUID"],
+    withServiceData: ["SERVICE_DATA_UUID"],
     withManufacturerData: [ManufacturerDataFilter(companyIdentifier: 0x004c)],
     withNamePrefix: ["NAME_PREFIX"],
   )
@@ -147,6 +148,8 @@ UniversalBle.stopScan();
 UniversalBle.isScanning();
 ```
 
+Each scan result is a `BleDevice`. Besides `deviceId`, `name` and `rssi` it carries `services` (the advertised service UUIDs, normalised to full 128-bit lower-case UUIDs on every platform), `manufacturerDataList`, `serviceData` (the advertising Service Data payloads, keyed by service UUID), `connectable` and the reception timestamps.
+
 #### Skipping duplicate scan results
 
 By default every advertisement packet is reported, so the same device can show up many times per second — each one waking your app and drawing battery. Pass `skipDuplicates: true` to report each device only once per scan session: later sightings are suppressed until the scan is stopped (`UniversalBle.stopScan`) or the next `startScan`, including sightings whose advertisement payload changed.
@@ -198,7 +201,7 @@ You still need to explicitly [connect](#connecting) to them before being able to
 
 #### Scan Filter
 
-You can optionally set a filter when scanning. A filter can have multiple conditions (services, manufacturerData, namePrefix) and all conditions are in `OR` relation, returning results that match any of the given conditions.
+You can optionally set a filter when scanning. A filter can have multiple conditions (services, serviceData, manufacturerData, namePrefix) and all conditions are in `OR` relation, returning results that match any of the given conditions.
 
 ##### With Services
 
@@ -210,6 +213,35 @@ List<String> withServices;
 
 Note: On web **you have to** specify services before you are able to use them. See the [web](#web) section for more details.
 
+##### With ServiceData
+
+When setting this parameter, the scan results will only include devices that advertise **Service Data** for any of the specified services.
+
+Service Data is the advertising data type that pairs a service UUID with a payload (`0x16`, `0x20` and `0x21` in the Bluetooth specification). It is a *different* field from the advertised service UUID list matched by `withServices`, and beacons commonly carry their UUID only there — Eddystone (`0xFEAA`), the ATC_MiThermometer (`0x181A`), BLE Mesh provisioning (`0x1827`) — which makes them invisible to a `withServices` filter.
+
+```dart
+List<String> withServiceData;
+```
+
+```dart
+UniversalBle.startScan(
+  scanFilter: ScanFilter(withServiceData: ['181A']),
+);
+
+UniversalBle.onScanResult = (device) {
+  // The payload of the device's 181A Service Data field.
+  final payload = device.serviceData['0000181a-0000-1000-8000-00805f9b34fb'];
+};
+```
+
+UUIDs are accepted in any case and in short (`181A`), `0x`-prefixed, or full 128-bit form. Like the other conditions, an empty list is ignored and the conditions are in `OR` relation, so combining `withServiceData` with `withServices` returns devices matching either.
+
+Platform notes:
+
+- On `Android` and `Apple` this moves the scan to the software filter path: the platform cannot filter on Service Data (a controller-level scan filter, or `scanForPeripherals(withServices:)`, would discard the device before the plugin is notified), so more advertisements reach the app process at a battery cost. Set it only when you need it.
+- On `Web` it is not supported and `startScan` / `requestDevice` throw `notSupported`: the browser's device chooser cannot filter on Service Data. Scan without it and read `BleDevice.serviceData` from the advertisement events instead.
+- On `Android`, a peripheral using legacy BLE 4.x advertisements additionally needs `AndroidOptions(legacy: true)`.
+
 ##### With ManufacturerData
 
 Use the `withManufacturerData` parameter to filter devices by manufacturer data. When you pass a list of `ManufacturerDataFilter` objects to this parameter, the scan results will only include devices that contain any of the specified manufacturer data.

android/build.gradle

@@ -36,6 +36,10 @@ android {
     sourceSets {
         main.java.srcDirs += 'src/main/kotlin'
         test.java.srcDirs += 'src/test/kotlin'
+        // The scan filter conformance fixture is shared with the Dart and C++
+        // tests (test/fixtures/scan_filter_cases.tsv), so it lives outside this
+        // module and is exposed to the JVM tests as a resource.
+        test.resources.srcDirs += '../test/fixtures'
     }
 
     defaultConfig {

android/src/main/kotlin/com/navideck/universal_ble/UniversalBle.g.kt

@@ -905,22 +905,30 @@ data class UniversalScanConfig (
 data class UniversalScanFilter (
   val withServices: List<String>,
   val withNamePrefix: List<String>,
-  val withManufacturerData: List<ManufacturerDataFilter>
+  val withManufacturerData: List<ManufacturerDataFilter>,
+  /**
+   * Service data UUIDs to match. Appended last on purpose: pigeon encodes
+   * fields positionally, so inserting this anywhere else would shift the
+   * indices of the fields above and break against a stale native binary.
+   */
+  val withServiceData: List<String>
 )
  {
   companion object {
     fun fromList(pigeonVar_list: List<Any?>): UniversalScanFilter {
       val withServices = pigeonVar_list[0] as List<String>
       val withNamePrefix = pigeonVar_list[1] as List<String>
       val withManufacturerData = pigeonVar_list[2] as List<ManufacturerDataFilter>
-      return UniversalScanFilter(withServices, withNamePrefix, withManufacturerData)
+      val withServiceData = pigeonVar_list[3] as List<String>
+      return UniversalScanFilter(withServices, withNamePrefix, withManufacturerData, withServiceData)
     }
   }
   fun toList(): List<Any?> {
     return listOf(
       withServices,
       withNamePrefix,
       withManufacturerData,
+      withServiceData,
     )
   }
   override fun equals(other: Any?): Boolean {
@@ -931,14 +939,15 @@ data class UniversalScanFilter (
       return true
     }
     val other = other as UniversalScanFilter
-    return UniversalBlePigeonUtils.deepEquals(this.withServices, other.withServices) && UniversalBlePigeonUtils.deepEquals(this.withNamePrefix, other.withNamePrefix) && UniversalBlePigeonUtils.deepEquals(this.withManufacturerData, other.withManufacturerData)
+    return UniversalBlePigeonUtils.deepEquals(this.withServices, other.withServices) && UniversalBlePigeonUtils.deepEquals(this.withNamePrefix, other.withNamePrefix) && UniversalBlePigeonUtils.deepEquals(this.withManufacturerData, other.withManufacturerData) && UniversalBlePigeonUtils.deepEquals(this.withServiceData, other.withServiceData)
   }
 
   override fun hashCode(): Int {
     var result = javaClass.hashCode()
     result = 31 * result + UniversalBlePigeonUtils.deepHash(this.withServices)
     result = 31 * result + UniversalBlePigeonUtils.deepHash(this.withNamePrefix)
     result = 31 * result + UniversalBlePigeonUtils.deepHash(this.withManufacturerData)
+    result = 31 * result + UniversalBlePigeonUtils.deepHash(this.withServiceData)
     return result
   }
 }

android/src/main/kotlin/com/navideck/universal_ble/UniversalBleFilterUtil.kt

@@ -11,71 +11,86 @@ private const val TAG = "UniversalBlePlugin"
 
 @SuppressLint("MissingPermission")
 class UniversalBleFilterUtil {
-    var scanFilter: UniversalScanFilter? = null
-    var serviceFilterUUIDS: List<UUID> = emptyList()
+    private var scanFilter: UniversalScanFilter? = null
+    private var serviceFilters: List<UUID> = emptyList()
+    private var serviceDataFilters: List<UUID> = emptyList()
+
+    /// Single entry point for the filter state.
+    ///
+    /// Pass `null` when the platform applies the filter itself (native
+    /// `ScanFilter`), which makes [filterDevice] accept every device. Deriving the
+    /// UUID lists here, instead of letting callers assign them alongside
+    /// [scanFilter], keeps the three from drifting apart.
+    fun configure(filter: UniversalScanFilter?) {
+        scanFilter = filter
+        serviceFilters = filter?.withServices?.toUUIDList().orEmpty()
+        serviceDataFilters = filter?.withServiceData?.toUUIDList().orEmpty()
+    }
 
     fun filterDevice(
         name: String?,
         manufacturerDataList: List<UniversalManufacturerData>,
         serviceUuids: Array<UUID>,
+        serviceDataUuids: List<UUID>,
     ): Boolean {
         val filter = scanFilter ?: return true
 
-        val hasNamePrefixFilter = filter.withNamePrefix.isNotEmpty()
-        val hasServiceFilter = filter.withServices.isNotEmpty()
-        val hasManufacturerDataFilter = filter.withManufacturerData.isNotEmpty()
-
-        // If there is no filter at all, then allow device
-        if (!hasNamePrefixFilter &&
-            !hasServiceFilter &&
-            !hasManufacturerDataFilter
-        ) {
+        // The conditions are in OR relation. Each check below is a strict
+        // predicate answering "does this dimension match?", so an unfiltered
+        // dimension contributes nothing and needs no guard of its own.
+        val hasAnyFilter =
+            filter.withNamePrefix.isNotEmpty() ||
+                    serviceFilters.isNotEmpty() ||
+                    serviceDataFilters.isNotEmpty() ||
+                    filter.withManufacturerData.isNotEmpty()
+        if (!hasAnyFilter) {
             return true
         }
 
-        // For not, we only have DeviceName filter
-        return hasNamePrefixFilter && isNameMatchingFilters(filter, name) ||
-                hasServiceFilter && isServicesMatchingFilters(serviceUuids) ||
-                hasManufacturerDataFilter && isManufacturerDataMatchingFilters(
-            filter,
-            manufacturerDataList
-        )
+        return isNameMatchingFilters(filter, name) ||
+                isServicesMatchingFilters(serviceUuids) ||
+                isServiceDataMatchingFilters(serviceDataUuids) ||
+                isManufacturerDataMatchingFilters(
+                    filter,
+                    manufacturerDataList
+                )
     }
 
     private fun isNameMatchingFilters(scanFilter: UniversalScanFilter, name: String?): Boolean {
-        val namePrefixFilter = scanFilter.withNamePrefix
-        if (namePrefixFilter.isEmpty()) {
-            return true
-        }
         if (name.isNullOrEmpty()) {
             return false
         }
-        return namePrefixFilter.any { name.startsWith(it) }
+        return scanFilter.withNamePrefix.any { name.startsWith(it) }
     }
 
     private fun isServicesMatchingFilters(
         serviceUuids: Array<UUID>,
     ): Boolean {
-        if (serviceFilterUUIDS.isEmpty()) {
-            return true
-        }
-        if (serviceUuids.isEmpty()) {
-            return false
-        }
-        return serviceFilterUUIDS.any {
+        return serviceFilters.any {
             serviceUuids.contains(it)
         }
     }
 
+    /// Matches a device by the service UUIDs it carries in its advertising
+    /// Service Data (the `0x16`/`0x20`/`0x21` data types). This is a different
+    /// set from the advertised service UUID list matched by
+    /// [isServicesMatchingFilters]: beacons such as Eddystone (`0xFEAA`) or the
+    /// ATC_MiThermometer (`0x181A`) only appear here (#57).
+    private fun isServiceDataMatchingFilters(
+        serviceDataUuids: List<UUID>,
+    ): Boolean {
+        return serviceDataFilters.any {
+            serviceDataUuids.contains(it)
+        }
+    }
+
     private fun isManufacturerDataMatchingFilters(
         scanFilter: UniversalScanFilter,
         manufacturerDataList: List<UniversalManufacturerData>,
     ): Boolean {
-        val filters = scanFilter.withManufacturerData
-        if (filters.isEmpty()) return true
         if (manufacturerDataList.isEmpty()) return false
         return manufacturerDataList.any { msd ->
-            filters.any { filter ->
+            scanFilter.withManufacturerData.any { filter ->
                 msd.companyIdentifier == filter.companyIdentifier &&
                         isDataMatching(filter.payloadPrefix, msd.data, filter.payloadMask)
             }
@@ -100,8 +115,13 @@ class UniversalBleFilterUtil {
 }
 
 fun UniversalScanFilter.usesCustomFilters(): Boolean {
-    // NamePrefix Filtering is not supported in native filters
-    return withNamePrefix.isNotEmpty()
+    // NamePrefix filtering is not supported in native filters. Service Data
+    // matching cannot be expressed as a native ScanFilter either: the controller
+    // would drop a device that carries the UUID only inside its Service Data
+    // before the app is ever notified, so both have to be applied in the scan
+    // callback instead. Callers that opt in pay for it in battery (every
+    // advertisement reaches the app process before being filtered).
+    return withNamePrefix.isNotEmpty() || withServiceData.isNotEmpty()
 }
 
 // Convert UniversalScanFilter to ScanFilter

android/src/main/kotlin/com/navideck/universal_ble/UniversalBleHelper.kt

@@ -157,6 +157,14 @@ val ScanResult.serviceData: Map<String, ByteArray>
         return scanRecord?.serviceData?.mapKeys { it.key.uuid.toString() } ?: emptyMap()
     }
 
+/// Service UUIDs the packet carries in its advertising Service Data
+/// (`0x16`/`0x20`/`0x21` data types), as opposed to [serviceUuids] which are the
+/// advertised service UUID lists.
+val ScanResult.serviceDataUuids: List<UUID>
+    get() {
+        return scanRecord?.serviceData?.keys?.map { it.uuid } ?: emptyList()
+    }
+
 fun <T> SparseArray<T>.toList(): List<Pair<Int, T>> {
     return (0 until size).map { index ->
         keyAt(index) to valueAt(index)

android/src/main/kotlin/com/navideck/universal_ble/UniversalBlePlugin.kt

@@ -277,10 +277,9 @@ class UniversalBlePlugin : UniversalBlePlatformChannel, BluetoothGattCallback(),
             // Set custom scan filter only if required
             if (usesCustomFilters) {
                 UniversalBleLogger.logError("Using Custom Filters")
-                universalBleFilterUtil.scanFilter = filter
-                universalBleFilterUtil.serviceFilterUUIDS = filterServices
+                universalBleFilterUtil.configure(filter)
             } else {
-                universalBleFilterUtil.scanFilter = null
+                universalBleFilterUtil.configure(null)
                 scanFilters = filter?.toScanFilters(filterServices) ?: emptyList()
             }
 
@@ -1674,7 +1673,8 @@ class UniversalBlePlugin : UniversalBlePlatformChannel, BluetoothGattCallback(),
             if (!universalBleFilterUtil.filterDevice(
                     name,
                     manufacturerDataList,
-                    serviceUuids
+                    serviceUuids,
+                    result.serviceDataUuids
                 )
             ) return