diff --git a/.changeset/d8a-purchase-example.md b/.changeset/d8a-purchase-example.md new file mode 100644 index 000000000..ad7cfb321 --- /dev/null +++ b/.changeset/d8a-purchase-example.md @@ -0,0 +1,8 @@ +--- +'@walkeros/web-destination-d8a': patch +--- + +The purchase example's item condition works as published, so the example no +longer throws when a tool evaluates it from the package metadata. The +`send_page_view` description says d8a sends a page view when the tracker starts +unless it is set to false. diff --git a/.changeset/fingerprint-ip-not-an-address.md b/.changeset/fingerprint-ip-not-an-address.md new file mode 100644 index 000000000..e3e340f22 --- /dev/null +++ b/.changeset/fingerprint-ip-not-an-address.md @@ -0,0 +1,9 @@ +--- +'@walkeros/server-transformer-fingerprint': patch +'@walkeros/cli': patch +--- + +An `ip` input that is not one address, such as a forwarded chain, is now +reported as an error and left out of the hash, instead of a warning that said it +was empty. The event is still delivered. The complete example flow reads the +visitor IP from a header the proxy sets. diff --git a/.changeset/linkedin-conversion-description.md b/.changeset/linkedin-conversion-description.md new file mode 100644 index 000000000..aedb23e93 --- /dev/null +++ b/.changeset/linkedin-conversion-description.md @@ -0,0 +1,8 @@ +--- +'@walkeros/web-destination-linkedin': patch +--- + +The `conversion` mapping description now says what happens when the conversion +resolves to no id: no `lintrk` call is made and the push fails with +`missing-input`. It no longer says such events are ignored, and it names the +server LinkedIn destination as the deduplication partner of `eventId`. diff --git a/.changeset/optimizely-user-id-description.md b/.changeset/optimizely-user-id-description.md new file mode 100644 index 000000000..8e1cfcba8 --- /dev/null +++ b/.changeset/optimizely-user-id-description.md @@ -0,0 +1,8 @@ +--- +'@walkeros/web-destination-optimizely': patch +--- + +The `userId` description shows a working fallback, `["user.id", "user.device"]`, +instead of a `{ key, value }` form that sent the literal text `user.device` as +the user id of every anonymous visitor. The `attributes` description says the +attributes apply when the user context is created. diff --git a/.changeset/pinterest-identify-examples.md b/.changeset/pinterest-identify-examples.md new file mode 100644 index 000000000..ae49d6e15 --- /dev/null +++ b/.changeset/pinterest-identify-examples.md @@ -0,0 +1,10 @@ +--- +'@walkeros/web-destination-pinterest': patch +--- + +The `identify` description matches the destination: init calls +`pintrk('load', tagId)` without an identity, which is sent with +`pintrk('set', ...)` when it changes, and the tag takes a plain or hashed email. +The checkout example's condition works as published, and the commerce examples +send product fields inside `line_items`, with `product_name` taken from the +product name. diff --git a/.changeset/snowplow-mapping-schema.md b/.changeset/snowplow-mapping-schema.md new file mode 100644 index 000000000..e28ec072d --- /dev/null +++ b/.changeset/snowplow-mapping-schema.md @@ -0,0 +1,8 @@ +--- +'@walkeros/web-destination-snowplow': patch +--- + +The mapping schema lists the rule settings `data` (the self-describing event +data) and `struct` (a structured event), which the destination reads, so they +appear in the generated documentation. The `contexts` setting accepts `browser`, +the browser context entity the Snowplow tracker supports. diff --git a/AGENT.md b/AGENT.md index 0fff2b736..19c061d7b 100644 --- a/AGENT.md +++ b/AGENT.md @@ -83,9 +83,9 @@ apps/ - **No `any`:** Never in production code - **XP principles:** DRY, KISS, YAGNI, TDD - **Test first:** Watch it fail before implementing -- **Verify:** Use the verification tiers from `/workspaces/developer/AGENT.md` - rule 11. Default L1 (touched package) per task. L2 at plan completion. Never - L4 unless the plan touched shared infra or the user asked. +- **Verify:** Default L1 per task (`npm run verify:touched -- `, the + touched package). L2 at plan completion (`npm run verify:affected`). Never L4 + (the full suite) unless the plan touched shared infra or the user asked. ## Creating/Maintaining Skills diff --git a/apps/demos/flows/README.md b/apps/demos/flows/README.md index 62a896684..149463a03 100644 --- a/apps/demos/flows/README.md +++ b/apps/demos/flows/README.md @@ -12,7 +12,8 @@ production-ready live demo for data ownership and sovereignty. ## Setup ```bash -cd /workspaces/developer/walkerOS/apps/demos/flows +# from the repository root +cd apps/demos/flows export PROJECT_ID=YOUR_PROJECTID export DATASET_ID=YOUR_DATASET diff --git a/packages/cli/CHANGELOG.md b/packages/cli/CHANGELOG.md index 82be216d1..899708286 100644 --- a/packages/cli/CHANGELOG.md +++ b/packages/cli/CHANGELOG.md @@ -1029,9 +1029,6 @@ and `event.context` remain required. - Each source self-registers via TypeScript module augmentation of `SourceMap` in `@walkeros/core`. - - App-side coordination (`/workspaces/developer/app`) is a follow-up plan, not - part of this release. Telemetry from v4 CLI/MCP will not validate against - the existing app schema until that follow-up ships. - `Mapping.Rule.skip` is renamed to `Mapping.Rule.silent`. Customer flow.json configs using `skip: true` in mapping rules must rename to `silent: true`. Hard cut: no legacy alias, the field is gone. diff --git a/packages/cli/examples/flow-complete.json b/packages/cli/examples/flow-complete.json index d38259348..b09b4cb6a 100644 --- a/packages/cli/examples/flow-complete.json +++ b/packages/cli/examples/flow-complete.json @@ -1882,7 +1882,7 @@ }, "ingest": { "map": { - "ip": "ip", + "ip": "headers.x-client-ip", "path": "path", "method": "method", "url": "url", diff --git a/packages/collector/CHANGELOG.md b/packages/collector/CHANGELOG.md index 2db0f382c..4bc503813 100644 --- a/packages/collector/CHANGELOG.md +++ b/packages/collector/CHANGELOG.md @@ -711,9 +711,6 @@ and `event.context` remain required. - Each source self-registers via TypeScript module augmentation of `SourceMap` in `@walkeros/core`. - - App-side coordination (`/workspaces/developer/app`) is a follow-up plan, not - part of this release. Telemetry from v4 CLI/MCP will not validate against - the existing app schema until that follow-up ships. - `Mapping.Rule.skip` is renamed to `Mapping.Rule.silent`. Customer flow.json configs using `skip: true` in mapping rules must rename to `silent: true`. Hard cut: no legacy alias, the field is gone. diff --git a/packages/collector/src/store-cache-wrapper.ts b/packages/collector/src/store-cache-wrapper.ts index 2b4dde4ec..c24a1f568 100644 --- a/packages/collector/src/store-cache-wrapper.ts +++ b/packages/collector/src/store-cache-wrapper.ts @@ -110,9 +110,7 @@ export interface WrappedStoreInstance extends Store.Instance { * number of tiers (negligible for the typical 2-3 tier chain). Each tier owns * its own `exp` from its own rule, so the staleness bound is per tier. * - * Write semantics (`set` / `delete`) follow the "cache is advisory" policy - * documented in `docs/plans/2026-05-13-store-cache-design.md` (Write path and - * error policy): + * Write semantics (`set` / `delete`) follow the "cache is advisory" policy: * - Backing first. If `backing.set`/`backing.delete` throws, the wrapper * throws and the cache layer is never touched. The backing is the source * of truth and its failures must surface to the caller. diff --git a/packages/server/destinations/rudderstack/src/__tests__/retry.test.ts b/packages/server/destinations/rudderstack/src/__tests__/retry.test.ts index ccd9e0d68..ee3849377 100644 --- a/packages/server/destinations/rudderstack/src/__tests__/retry.test.ts +++ b/packages/server/destinations/rudderstack/src/__tests__/retry.test.ts @@ -56,12 +56,13 @@ describe('rudderstack retry within config.timeout', () => { const plane = await dataPlane([503, 503, 200]); const collector = await flowWith(plane.url); - const started = Date.now(); const result = await collector.push(createEvent({ name: 'product view' })); + // The collector races the push against config.timeout, so ok means the + // SDK's retries landed inside it; a wall clock here would also time the + // collector and the test runner. expect(result.ok).toBe(true); expect(plane.requests()).toBe(3); - expect(Date.now() - started).toBeLessThan(TIMEOUT); await collector.command('shutdown'); await plane.close(); }); diff --git a/packages/server/sources/express/src/__tests__/client-ip.test.ts b/packages/server/sources/express/src/__tests__/client-ip.test.ts new file mode 100644 index 000000000..f686f5153 --- /dev/null +++ b/packages/server/sources/express/src/__tests__/client-ip.test.ts @@ -0,0 +1,107 @@ +import { startFlow } from '@walkeros/collector'; +import { isObject, isString, Source } from '@walkeros/core'; +import type { + Destination, + Ingest, + Mapping, + Source as SourceNS, +} from '@walkeros/core'; +import { sourceExpress } from '../index'; +import type { ExpressSource, Types } from '../types'; + +function serverOf( + source: SourceNS.Instance, +): ExpressSource['server'] | undefined { + const candidate: { type: string; server?: ExpressSource['server'] } = source; + return candidate.server; +} + +// The documented one-liner for a proxy that appends `, `: +// the entry one hop in from the right of X-Forwarded-For. +const secondFromRight: Mapping.Fn = (scope) => { + if (!isObject(scope) || !isObject(scope.headers)) return undefined; + const chain = scope.headers['x-forwarded-for']; + return isString(chain) ? chain.split(',').at(-2)?.trim() : undefined; +}; + +/** + * One request to a started Express source on a real collector, with + * `config.ingest.map.ip` set to the given mapping; returns the ingest the + * destination receives. + */ +async function ingestFor( + ip: Mapping.Value, + headers: Record, +): Promise { + const received: Ingest[] = []; + const recorder: Destination.Instance = { + type: 'recorder', + config: {}, + push: (_event, context) => { + received.push(context.ingest); + }, + }; + const { collector } = await startFlow({ + sources: { + express: { + code: sourceExpress, + config: { + async: false, + settings: { port: 0, paths: ['/collect'] }, + ingest: { map: { ip } }, + }, + }, + }, + destinations: { recorder: { code: recorder } }, + }); + const server = serverOf(Source.getSource(collector, 'express')); + try { + const address = server?.address(); + if (!address || typeof address === 'string') { + throw new Error('Server did not bind'); + } + const response = await fetch(`http://127.0.0.1:${address.port}/collect`, { + method: 'POST', + headers: { 'content-type': 'application/json', ...headers }, + body: JSON.stringify({ name: 'page view' }), + }); + expect(response.status).toBe(200); + } finally { + await new Promise((resolve) => { + if (!server) return resolve(); + server.close(() => resolve()); + }); + } + expect(received).toHaveLength(1); + return received[0]; +} + +describe('client IP behind a proxy, as the Express docs map it', () => { + it.each([ + [ + 'a single-address header the proxy sets', + { key: 'headers.x-client-ip' }, + { 'x-client-ip': '203.0.113.7' }, + '203.0.113.7', + ], + [ + 'the second entry from the right of X-Forwarded-For', + { fn: secondFromRight }, + { 'x-forwarded-for': '198.51.100.9, 203.0.113.7, 10.0.0.1' }, + '203.0.113.7', + ], + ])('reads %s into ingest.ip', async (_, ip, headers, expected) => { + const ingest = await ingestFor(ip, headers); + + expect(ingest.ip).toBe(expected); + }); + + it('leaves ingest.ip absent when X-Forwarded-For has a single entry', async () => { + const ingest = await ingestFor( + { fn: secondFromRight }, + { 'x-forwarded-for': '10.0.0.1' }, + ); + + expect(ingest).not.toHaveProperty('ip'); + }); +}); diff --git a/packages/server/transformers/bot/src/data/crawlers.ts b/packages/server/transformers/bot/src/data/crawlers.ts index 8766e599f..fd02fcf07 100644 --- a/packages/server/transformers/bot/src/data/crawlers.ts +++ b/packages/server/transformers/bot/src/data/crawlers.ts @@ -25,9 +25,8 @@ * * A UA match is a claim, not proof: Screaming Frog ships Googlebot and Bingbot * presets, and any client can send any UA. Vendor IP ranges and reverse DNS - * suffixes for the entries that publish them are listed in - * `docs/research/2026-08-20-search-crawler-list.md` and are consumed once - * identity verification exists. + * suffixes, for the entries that publish them, are in the vendor docs below + * and will be used once identity verification exists. * * Vendor docs of record (verified 2026-08): * Google: https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers diff --git a/packages/server/transformers/fingerprint/src/__tests__/ip-not-an-address.test.ts b/packages/server/transformers/fingerprint/src/__tests__/ip-not-an-address.test.ts new file mode 100644 index 000000000..b33bcfbea --- /dev/null +++ b/packages/server/transformers/fingerprint/src/__tests__/ip-not-an-address.test.ts @@ -0,0 +1,132 @@ +import { startFlow } from '@walkeros/collector'; +import type { Destination, Logger, Mapping, WalkerOS } from '@walkeros/core'; +import { Level } from '@walkeros/core'; +import { transformerFingerprint } from '../transformer'; + +const chrome124 = + 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.6367.60 Safari/537.36'; + +interface Line { + level: Logger.Level; + message: string; + context: Logger.LogContext; +} + +/** + * The fingerprint transformer in a destination's before chain of a real + * collector: two events pushed, every log line from WARN up recorded, and + * what the destination receives. + */ +async function deliver( + ip: Mapping.Value, + userAgent = chrome124, + data: WalkerOS.Properties = {}, +) { + const lines: Line[] = []; + const received: WalkerOS.Event[] = []; + const recorder: Destination.Instance = { + type: 'recorder', + config: {}, + push: (event) => { + received.push(event); + }, + }; + const { elb } = await startFlow({ + logger: { + level: 'WARN', + handler: (level, message, context) => { + lines.push({ level, message, context }); + }, + }, + transformers: { + fingerprint: { + code: transformerFingerprint, + config: { + settings: { + salt: 'test-salt', + ip, + userAgent: { value: userAgent }, + }, + }, + }, + }, + destinations: { recorder: { code: recorder, before: 'fingerprint' } }, + }); + await elb({ name: 'page view', data }); + await elb({ name: 'page view', data }); + + const errors = lines.filter((line) => line.level === Level.ERROR); + const ipEmptyWarnings = lines.filter( + (line) => + line.level === Level.WARN && + line.message.includes('Fingerprint input "ip" resolved empty'), + ); + const hashes = received.map((event) => typeof event.user.hash); + return { errors, ipEmptyWarnings, hashes }; +} + +describe('an ip input that is not one address, through a real collector', () => { + it.each([ + ['a forwarded chain', '203.0.113.7, 172.70.1.2', ['203.0.113', '172.70']], + ['a word', 'unknown', ['unknown']], + ])( + 'reports %s once and still delivers both events with a hash', + async (_, ip, fragments) => { + const { errors, ipEmptyWarnings, hashes } = await deliver({ value: ip }); + + expect(hashes).toEqual(['string', 'string']); + expect(errors).toHaveLength(1); + const [error] = errors; + expect(error.message).toContain('"ip"'); + for (const fragment of fragments) { + expect(error.message).not.toContain(fragment); + expect(JSON.stringify(error.context)).not.toContain(fragment); + } + expect(error.context).toEqual({ + step: 'transformer.fingerprint', + code: 'invalid', + }); + expect(ipEmptyWarnings).toHaveLength(0); + }, + ); + + it.each([ + ['IPv4', '203.0.113.7'], + ['IPv6', '2001:db8:abcd:12::1'], + ['IPv4-mapped IPv6', '::ffff:203.0.113.7'], + ])('reports nothing for an %s address', async (_, ip) => { + const { errors, ipEmptyWarnings, hashes } = await deliver({ value: ip }); + + expect(hashes).toEqual(['string', 'string']); + expect(errors).toHaveLength(0); + expect(ipEmptyWarnings).toHaveLength(0); + }); + + it('keeps the one warning for an empty ip and reports no error', async () => { + const { errors, ipEmptyWarnings, hashes } = await deliver({ value: '' }); + + expect(hashes).toEqual(['string', 'string']); + expect(errors).toHaveLength(0); + expect(ipEmptyWarnings).toHaveLength(1); + }); + + it('reports nothing for an unknown user agent', async () => { + const { errors, hashes } = await deliver( + { value: '203.0.113.7' }, + 'curl/8.5.0', + ); + + expect(hashes).toEqual(['string', 'string']); + expect(errors).toHaveLength(0); + }); + + it('names the configured path of the ip input, never its value', async () => { + const { errors } = await deliver('event.data.ip', chrome124, { + ip: '203.0.113.7, 172.70.1.2', + }); + + expect(errors).toHaveLength(1); + expect(errors[0].message).toMatch(/ \(event\.data\.ip\)$/); + expect(errors[0].message).not.toContain('203.0.113'); + }); +}); diff --git a/packages/server/transformers/fingerprint/src/hints.ts b/packages/server/transformers/fingerprint/src/hints.ts index 29312688a..c9874ca04 100644 --- a/packages/server/transformers/fingerprint/src/hints.ts +++ b/packages/server/transformers/fingerprint/src/hints.ts @@ -2,7 +2,7 @@ import type { Hint } from '@walkeros/core'; export const hints: Hint.Hints = { 'ingest-prerequisite': { - text: 'The default inputs ip and userAgent read ingest.ip and ingest.userAgent, so the server source must have config.ingest configured. config.ingest MUST use the map operator with direct request field paths (no req. prefix); a bare object like { ip: "req.ip" } is silently inert and leaves ingest empty. An input that resolves empty logs one warning ("Fingerprint input ... resolved empty") and the hash is still generated, but less distinct. Behind a load balancer the request ip is the balancer; read the client from headers.x-forwarded-for instead.', + text: 'The default inputs ip and userAgent read ingest.ip and ingest.userAgent, so the server source must have config.ingest configured. config.ingest MUST use the map operator with direct request field paths (no req. prefix); a bare object like { ip: "req.ip" } is silently inert and leaves ingest empty. An input that resolves empty logs one warning ("Fingerprint input ... resolved empty") and the hash is still generated, but less distinct. Behind a proxy the request ip is the address of the proxy: map the single-address header the proxy sets, or follow the Express source docs, section "Client IP behind a proxy". An ip that is not one address is reported as an error and left out of the hash.', code: [ { lang: 'json', @@ -16,7 +16,7 @@ export const hints: Hint.Hints = { settings: { port: 8080 }, ingest: { map: { - ip: { key: 'ip' }, + ip: { key: 'headers.x-client-ip' }, userAgent: { key: 'headers.user-agent' }, }, }, diff --git a/packages/server/transformers/fingerprint/src/transformer.ts b/packages/server/transformers/fingerprint/src/transformer.ts index 94a1e1fc6..15a94cdf0 100644 --- a/packages/server/transformers/fingerprint/src/transformer.ts +++ b/packages/server/transformers/fingerprint/src/transformer.ts @@ -6,8 +6,10 @@ import { getBrowserVersion, getMappingValue, getOS, + reportCause, setByPath, StepError, + stepId, } from '@walkeros/core'; import { getHashServer } from '@walkeros/server-core'; import type { FingerprintSettings } from './types'; @@ -108,17 +110,17 @@ export const transformerFingerprint: Transformer.Init< config: { ...config, settings: { ...settings, salt } }, async push(event, context) { - const { ingest, collector } = context; + const { ingest, collector, id } = context; const source = createMappingRoot(ingest, event); const resolve = async (value: Mapping.Value) => String((await getMappingValue(source, value, { collector })) ?? ''); - const inputValues = await Promise.all( - inputs.map(async ({ name, value }) => { - const resolved = await resolve(value); - return resolved ? reduceInput[name](resolved) : ''; - }), + const resolvedValues = await Promise.all( + inputs.map(({ value }) => resolve(value)), + ); + const inputValues = resolvedValues.map((resolved, i) => + resolved ? reduceInput[inputs[i].name](resolved) : '', ); const fieldValues = await Promise.all(fields.map(resolve)); @@ -135,8 +137,25 @@ export const transformerFingerprint: Transformer.Init< { code: 'missing-input' }, ); - inputs.forEach(({ name }, i) => { - if (inputValues[i] || warnedEmpty.has(name)) return; + inputs.forEach(({ name, value }, i) => { + if (inputValues[i]) return; + + // Only the ip reducer turns text into nothing: the value is not one + // address (a forwarded chain, a word). The event is still delivered, + // so this is reported, never thrown, and never carries the value. + if (resolvedValues[i]) { + const path = typeof value === 'string' ? ` (${value})` : ''; + reportCause( + collector, + logger, + { step: stepId('transformer', id), code: 'invalid' }, + `Fingerprint input "${name}" is not one IP address, so the hash is made without it${path}`, + {}, + ); + return; + } + + if (warnedEmpty.has(name)) return; warnedEmpty.add(name); logger.warn( `Fingerprint input "${name}" resolved empty, so hashes are less distinct. Check the source's config.ingest or set settings.${name}.`, diff --git a/packages/web/destinations/d8a/src/__tests__/schemas.test.ts b/packages/web/destinations/d8a/src/__tests__/schemas.test.ts new file mode 100644 index 000000000..934051a33 --- /dev/null +++ b/packages/web/destinations/d8a/src/__tests__/schemas.test.ts @@ -0,0 +1,14 @@ +import { isObject } from '@walkeros/core'; +import { settings } from '../schemas'; + +describe('schemas', () => { + it('send_page_view text says d8a sends a page_view by default', () => { + const properties = settings.properties; + const sendPageView = + isObject(properties) && isObject(properties.send_page_view) + ? properties.send_page_view.description + : undefined; + + expect(sendPageView).toMatch(/Default true/); + }); +}); diff --git a/packages/web/destinations/d8a/src/examples/step.ts b/packages/web/destinations/d8a/src/examples/step.ts index f7e5086a5..e42683ebc 100644 --- a/packages/web/destinations/d8a/src/examples/step.ts +++ b/packages/web/destinations/d8a/src/examples/step.ts @@ -1,5 +1,5 @@ import type { Flow } from '@walkeros/core'; -import { getEvent, isObject } from '@walkeros/core'; +import { getEvent } from '@walkeros/core'; const INIT_DATE_MS = 1700000000000; const INIT_DATE = new Date(INIT_DATE_MS); @@ -102,10 +102,11 @@ export const purchase: Flow.StepExample = { loop: [ 'nested', { - condition: (entity: unknown) => { - const candidate = entity as { entity?: unknown }; - return isObject(entity) && candidate.entity === 'product'; - }, + condition: (entity: unknown) => + typeof entity === 'object' && + entity !== null && + 'entity' in entity && + entity.entity === 'product', map: { item_id: 'data.id', item_name: 'data.name', diff --git a/packages/web/destinations/d8a/src/index.test.ts b/packages/web/destinations/d8a/src/index.test.ts index 15f13d2c6..97b971837 100644 --- a/packages/web/destinations/d8a/src/index.test.ts +++ b/packages/web/destinations/d8a/src/index.test.ts @@ -1,6 +1,6 @@ // The real @d8a-tech/wt package is ESM and can't be parsed by Jest's // default CommonJS transformer. Tests always wire their own mock via -// `env.installD8a`, so the real module is never invoked — we stub the +// `env.installD8a`, so the real module is never invoked; we stub the // import with a no-op installer to let Jest load the destination file. jest.mock( '@d8a-tech/wt', @@ -27,9 +27,10 @@ jest.mock( { virtual: true }, ); -import type { Destination, WalkerOS } from '@walkeros/core'; +import type { Destination, Mapping, WalkerOS } from '@walkeros/core'; import type { D8aFn, InstallD8aOptions, InstallD8aResult } from '@d8a-tech/wt'; -import { createMockLogger } from '@walkeros/core'; +import { runInNewContext } from 'vm'; +import { createMockLogger, isObject } from '@walkeros/core'; import { startFlow } from '@walkeros/collector'; import { examples } from './dev'; import { resetConsentState } from './index'; @@ -86,6 +87,21 @@ function createInstallD8a(calls: CallRecord[]): Env['installD8a'] { }; } +// Functions in examples are published as `{ $code: fn.toString() }` in +// walkerOS.json and evaluated without the module scope they were written in. +function asPublished(rule: unknown): Mapping.Rule | undefined { + if (!isObject(rule)) return undefined; + return JSON.parse( + JSON.stringify(rule, (_, value) => + typeof value === 'function' ? { $code: value.toString() } : value, + ), + (_, value) => + isObject(value) && typeof value.$code === 'string' + ? runInNewContext(`(${value.$code})`) + : value, + ); +} + function makeTestEnv(): { env: Env; calls: CallRecord[] } { const calls: CallRecord[] = []; return { @@ -152,4 +168,27 @@ describe('d8a web destination -- step examples', () => { const actual = calls.slice(initOut.length); expect(actual).toEqual(example.out); }); + + it.each(stepEntries.filter(([, example]) => !example.command))( + '%s as published', + async (_name, example) => { + const { env, calls } = makeTestEnv(); + const dest = jest.requireActual('./').default; + const { elb } = await startFlow(); + + const event = example.in as WalkerOS.Event; + const rule = asPublished(example.mapping); + const mapping = rule + ? { [event.entity]: { [event.action]: rule } } + : undefined; + + elb('walker destination', { + code: { ...dest, env }, + config: { ...initIn, mapping }, + }); + await elb(event); + + expect(calls.slice(initOut.length)).toEqual(example.out); + }, + ); }); diff --git a/packages/web/destinations/d8a/src/schemas/primitives.ts b/packages/web/destinations/d8a/src/schemas/primitives.ts index 6d58a26c6..352fd78e1 100644 --- a/packages/web/destinations/d8a/src/schemas/primitives.ts +++ b/packages/web/destinations/d8a/src/schemas/primitives.ts @@ -28,7 +28,9 @@ export const SettingsSchema = z .optional(), send_page_view: z .boolean() - .describe('Enable automatic pageview tracking') + .describe( + 'd8a sends a page_view when the tracker starts. Default true; set false when the flow sends its own page view events', + ) .optional(), snakeCase: z .boolean() diff --git a/packages/web/destinations/linkedin/README.md b/packages/web/destinations/linkedin/README.md index bb3ff61b1..7be485faf 100644 --- a/packages/web/destinations/linkedin/README.md +++ b/packages/web/destinations/linkedin/README.md @@ -32,7 +32,10 @@ npm install @walkeros/web-destination-linkedin "destinations": { "linkedin": { "package": "@walkeros/web-destination-linkedin", - "config": { "settings": { "apiKey": "YOUR_PARTNER_ID" } } + "config": { + "loadScript": true, + "settings": { "apiKey": "123456" } + } } } } @@ -40,6 +43,8 @@ npm install @walkeros/web-destination-linkedin } ``` +LinkedIn tracks only the events whose rule sets a `conversion`, see the docs. + ## Documentation Full configuration, mapping, and examples live in the docs: diff --git a/packages/web/destinations/linkedin/src/__tests__/schemas.test.ts b/packages/web/destinations/linkedin/src/__tests__/schemas.test.ts new file mode 100644 index 000000000..d6f6c7010 --- /dev/null +++ b/packages/web/destinations/linkedin/src/__tests__/schemas.test.ts @@ -0,0 +1,15 @@ +import { isObject } from '@walkeros/core'; +import { mapping } from '../schemas'; + +describe('schemas', () => { + it('conversion text says a missing id fails the push', () => { + const properties = mapping.properties; + const conversion = + isObject(properties) && isObject(properties.conversion) + ? properties.conversion.description + : undefined; + + expect(conversion).toMatch(/missing-input/); + expect(conversion).not.toMatch(/silently/); + }); +}); diff --git a/packages/web/destinations/linkedin/src/index.ts b/packages/web/destinations/linkedin/src/index.ts index 9a1f0108a..7ec7baecc 100644 --- a/packages/web/destinations/linkedin/src/index.ts +++ b/packages/web/destinations/linkedin/src/index.ts @@ -19,8 +19,6 @@ export * as DestinationLinkedIn from './types'; * Loads the LinkedIn Insight Tag script from LinkedIn's CDN and forwards * explicit conversions via `window.lintrk('track', ...)`. Opt-in: events * without `mapping.settings.conversion` produce zero calls. - * - * See `/workspaces/developer/docs/research/linkedin-destination-guide.md`. */ const LINKEDIN_SCRIPT_SRC = diff --git a/packages/web/destinations/linkedin/src/schemas/mapping.ts b/packages/web/destinations/linkedin/src/schemas/mapping.ts index 82bb14aa7..23c556f5f 100644 --- a/packages/web/destinations/linkedin/src/schemas/mapping.ts +++ b/packages/web/destinations/linkedin/src/schemas/mapping.ts @@ -5,14 +5,14 @@ import { z } from '@walkeros/core/dev'; * * Every field on `conversion` is a walkerOS mapping value (static literal, * string path, or `{ map, value, key, fn, ... }` object). The schema uses - * `z.unknown()` for each so users can supply any valid mapping shape — the + * `z.unknown()` for each so users can supply any valid mapping shape; the * destination resolves it via `getMappingValue()` at push time. */ export const MappingSchema = z.object({ conversion: z .unknown() .describe( - 'Resolves to a LinkedIn conversion object with short keys: { id (required number — the Campaign Manager conversion_id), value? (number), currency? (ISO code), eventId? (string for deduplication with a future server/Conversions API destination) }. Events without a resolved id are silently ignored.', + 'Resolves to a LinkedIn conversion object with short keys: { id (required number, the Campaign Manager conversion_id), value? (number), currency? (ISO code), eventId? (string, the same event id the server LinkedIn destination sends, for deduplication) }. A rule whose conversion resolves to no id makes no lintrk call and fails the push with missing-input.', ) .optional(), }); diff --git a/packages/web/destinations/linkedin/src/types/index.ts b/packages/web/destinations/linkedin/src/types/index.ts index 5ddbce0bf..9f897d54f 100644 --- a/packages/web/destinations/linkedin/src/types/index.ts +++ b/packages/web/destinations/linkedin/src/types/index.ts @@ -7,7 +7,7 @@ import type { DestinationWeb } from '@walkeros/web-core'; /** * LinkedIn Insight Tag runtime surface. * - * The Insight Tag script installs `window.lintrk` — a single function that + * The Insight Tag script installs `window.lintrk`, a single function that * accepts exactly one tracked action: `lintrk('track', data)`. * * Before the script loads, the destination installs a queue-backed shim so @@ -39,13 +39,8 @@ declare global { /** * Destination-level settings. * - * apiKey — the LinkedIn Partner ID (numeric string, e.g. "123456"), assigned - * to `window._linkedin_partner_id` before the script loads. - * include — event sections made available for mapping resolution. Present for - * consistency with other destinations; has no effect at call time - * because `lintrk()` only accepts four fixed fields. Kept so users - * can reference section-prefixed keys in future custom mapping - * strategies without a breaking change. + * apiKey: the LinkedIn Partner ID (numeric string, e.g. "123456"), assigned + * to `window._linkedin_partner_id` before the script loads. */ export interface Settings { apiKey: string; @@ -56,21 +51,22 @@ export type InitSettings = Partial; /** * Per-rule mapping settings. * - * conversion — mapping value resolving to an object with short keys: + * conversion: mapping value resolving to an object with short keys: * { id (required), value?, currency?, eventId? } * * `id` becomes `conversion_id` (number, Campaign Manager conversion rule ID). * Other keys translate 1:1 to the vendor parameter names. * - * Events without a resolved `conversion.id` are silently ignored — LinkedIn - * is an opt-in conversion model. + * A rule without `conversion` makes no call: LinkedIn is an opt-in + * conversion model. A `conversion` that resolves to no `id` fails the push + * with `missing-input`. */ export interface Mapping { conversion?: WalkerOSMapping.Value; } /** - * Env — mock surface for tests and dev. The destination mutates + * Env: mock surface for tests and dev. The destination mutates * `window._linkedin_partner_id` / `window._linkedin_data_partner_ids` and * installs `window.lintrk` in init; tests can pre-seed `window.lintrk` with * a spy to skip the script injection path. diff --git a/packages/web/destinations/optimizely/README.md b/packages/web/destinations/optimizely/README.md index adbc04941..4d53c71ce 100644 --- a/packages/web/destinations/optimizely/README.md +++ b/packages/web/destinations/optimizely/README.md @@ -32,7 +32,9 @@ npm install @walkeros/web-destination-optimizely "destinations": { "optimizely": { "package": "@walkeros/web-destination-optimizely", - "config": { "settings": { "sdkKey": "YOUR_SDK_KEY" } } + "config": { + "settings": { "sdkKey": "YOUR_SDK_KEY", "userId": "user.id" } + } } } } @@ -40,6 +42,9 @@ npm install @walkeros/web-destination-optimizely } ``` +An event whose `user.id` is not set fails with `missing-input`. The docs show a +fallback for anonymous visitors. + ## Documentation Full configuration, mapping, and examples live in the docs: diff --git a/packages/web/destinations/optimizely/src/__tests__/schemas.test.ts b/packages/web/destinations/optimizely/src/__tests__/schemas.test.ts new file mode 100644 index 000000000..2509716b3 --- /dev/null +++ b/packages/web/destinations/optimizely/src/__tests__/schemas.test.ts @@ -0,0 +1,21 @@ +import { isObject } from '@walkeros/core'; +import { settings } from '../schemas'; + +function description(key: string): unknown { + const properties = settings.properties; + return isObject(properties) && isObject(properties[key]) + ? properties[key].description + : undefined; +} + +describe('schemas', () => { + it('userId text shows a fallback chain, not a static value', () => { + const userId = description('userId'); + expect(userId).toMatch(/\["user\.id", "user\.device"\]/); + expect(userId).not.toMatch(/key: "user\.id"/); + }); + + it('attributes text says when attributes are applied', () => { + expect(description('attributes')).toMatch(/user context is created/); + }); +}); diff --git a/packages/web/destinations/optimizely/src/schemas/settings.ts b/packages/web/destinations/optimizely/src/schemas/settings.ts index 0074dc090..c7906e81b 100644 --- a/packages/web/destinations/optimizely/src/schemas/settings.ts +++ b/packages/web/destinations/optimizely/src/schemas/settings.ts @@ -11,13 +11,13 @@ export const SettingsSchema = z userId: z .unknown() .describe( - 'walkerOS mapping value to resolve userId for experiment bucketing. Example: "user.id" or { key: "user.id", value: "user.device" } for fallback.', + 'walkerOS mapping value to resolve userId for experiment bucketing. Example: "user.id", or the fallback chain ["user.id", "user.device"]. An event where it resolves to no string fails with missing-input.', ) .optional(), attributes: z .unknown() .describe( - 'User attributes for audience targeting. Resolves to Record. Applied to every event via createUserContext().', + 'User attributes for audience targeting. Resolves to Record. Applied through createUserContext() when the user context is created, on the first event of each userId; a later change for the same userId is not applied.', ) .optional(), updateInterval: z diff --git a/packages/web/destinations/pinterest/README.md b/packages/web/destinations/pinterest/README.md index 4c6d4264c..8586dc74e 100644 --- a/packages/web/destinations/pinterest/README.md +++ b/packages/web/destinations/pinterest/README.md @@ -33,7 +33,7 @@ npm install @walkeros/web-destination-pinterest "destinations": { "pinterest": { "package": "@walkeros/web-destination-pinterest", - "config": { "settings": { "apiKey": "YOUR_TAG_ID" } } + "config": { "settings": { "apiKey": "2612345678901" } } } } } diff --git a/packages/web/destinations/pinterest/src/__tests__/schemas.test.ts b/packages/web/destinations/pinterest/src/__tests__/schemas.test.ts new file mode 100644 index 000000000..616753fc8 --- /dev/null +++ b/packages/web/destinations/pinterest/src/__tests__/schemas.test.ts @@ -0,0 +1,15 @@ +import { isObject } from '@walkeros/core'; +import { settings } from '../schemas'; + +describe('schemas', () => { + it('identify text says init loads without an identity', () => { + const properties = settings.properties; + const identify = + isObject(properties) && isObject(properties.identify) + ? properties.identify.description + : undefined; + + expect(identify).not.toMatch(/tagId, data/); + expect(identify).toMatch(/hashed/); + }); +}); diff --git a/packages/web/destinations/pinterest/src/__tests__/stepExamples.test.ts b/packages/web/destinations/pinterest/src/__tests__/stepExamples.test.ts index 832735aca..72c4c45c9 100644 --- a/packages/web/destinations/pinterest/src/__tests__/stepExamples.test.ts +++ b/packages/web/destinations/pinterest/src/__tests__/stepExamples.test.ts @@ -1,5 +1,5 @@ /** - * Pinterest destination — step examples runner. + * Pinterest destination: step examples runner. * * Every fixture in `examples.step` is a test case. The runner installs a * spy as `env.window.pintrk`, pushes the example event through a real @@ -8,6 +8,7 @@ */ import type { WalkerOS, Mapping as WalkerOSMapping } from '@walkeros/core'; +import { runInNewContext } from 'vm'; import { startFlow } from '@walkeros/collector'; import type { Logger } from '@walkeros/core'; import { clone, createEvent, isObject, isString, Level } from '@walkeros/core'; @@ -53,7 +54,24 @@ function spyEnv(env: Env): { env: Env; collected: () => CallRecord[] } { return { env, collected: () => calls }; } -describe('pinterest destination — step examples', () => { +// Functions in examples are published as `{ $code: fn.toString() }` in +// walkerOS.json and evaluated without the module scope they were written in. +function asPublished( + rule: WalkerOSMapping.Rule | undefined, +): WalkerOSMapping.Rule | undefined { + if (rule === undefined) return rule; + return JSON.parse( + JSON.stringify(rule, (_, value) => + typeof value === 'function' ? { $code: value.toString() } : value, + ), + (_, value) => + isObject(value) && typeof value.$code === 'string' + ? runInNewContext(`(${value.$code})`) + : value, + ); +} + +describe('pinterest destination: step examples', () => { it.each(Object.entries(examples.step))('%s', async (name, example) => { const env = clone(examples.env.push); const { env: spiedEnv, collected } = spyEnv(env); @@ -100,6 +118,57 @@ describe('pinterest destination — step examples', () => { expect(actual).toEqual([...(example.out ?? [])]); }); + it.each( + Object.entries(examples.step).filter( + ([, example]) => example.command === undefined && isEvent(example.in), + ), + )('%s as published', async (_name, example) => { + const env = clone(examples.env.push); + const { env: spiedEnv, collected } = spyEnv(env); + const dest = jest.requireActual('../').default; + const { elb } = await startFlow(); + + if (!isEvent(example.in)) throw new Error('no event input'); + const event = example.in; + const mapping = asPublished( + isRule(example.mapping) ? example.mapping : undefined, + ); + + await elb('walker destination', { + code: { ...dest, env: spiedEnv }, + config: { + settings: { apiKey: '2612345678901' }, + mapping: mapping + ? { [event.entity]: { [event.action]: mapping } } + : undefined, + }, + }); + await elb(event); + + const actual = collected().filter( + (call) => call[1] !== 'load' && call[1] !== 'page', + ); + expect(actual).toEqual([...(example.out ?? [])]); + }); + + it('sends product fields inside line_items', () => { + const topLevel = Object.entries(examples.step).filter(([, example]) => { + const rule = isRule(example.mapping) ? example.mapping : undefined; + const map = isObject(rule?.data) ? rule.data.map : undefined; + return isObject(map) && ('product_id' in map || 'product_name' in map); + }); + expect(topLevel.map(([name]) => name)).toEqual([]); + }); + + it('maps product_name from the product name', () => { + const fromColor = Object.entries(examples.step).filter(([, example]) => + JSON.stringify(example.mapping ?? {}).includes( + '"product_name":"data.color"', + ), + ); + expect(fromColor.map(([name]) => name)).toEqual([]); + }); + it('writes one DEBUG line naming the event and the mapping key for a declared skip', async () => { const lines: Array<[Level, string, unknown]> = []; const handler: Logger.Handler = (level, message, context) => { diff --git a/packages/web/destinations/pinterest/src/examples/step.ts b/packages/web/destinations/pinterest/src/examples/step.ts index 0af2aeb23..95301e3b7 100644 --- a/packages/web/destinations/pinterest/src/examples/step.ts +++ b/packages/web/destinations/pinterest/src/examples/step.ts @@ -1,5 +1,5 @@ import type { Flow } from '@walkeros/core'; -import { createEvent, getEvent, isObject } from '@walkeros/core'; +import { createEvent, getEvent } from '@walkeros/core'; /** * Output convention: @@ -88,7 +88,7 @@ export const siteSearch: Flow.StepExample = { export const productViewContent: Flow.StepExample = { title: 'View content', description: - 'A product view fires Pinterest viewcontent with value, currency, product_id, and product_name.', + 'A product view fires Pinterest viewcontent with value, currency, and a single-item line_items array.', in: getEvent('product view', { timestamp: 1700000104, id: 'ev-1700000104', @@ -99,8 +99,17 @@ export const productViewContent: Flow.StepExample = { map: { value: 'data.price', currency: { key: 'data.currency', value: 'EUR' }, - product_id: 'data.id', - product_name: 'data.color', + line_items: { + set: [ + { + map: { + product_id: 'data.id', + product_name: 'data.name', + product_price: 'data.price', + }, + }, + ], + }, }, }, }, @@ -112,8 +121,13 @@ export const productViewContent: Flow.StepExample = { { value: 420, currency: 'EUR', - product_id: 'ers', - product_name: 'black', + line_items: [ + { + product_id: 'ers', + product_name: 'Everyday Ruck Snack', + product_price: 420, + }, + ], event_id: 'ev-1700000104', }, ], @@ -143,7 +157,7 @@ export const productAddToCart: Flow.StepExample = { { map: { product_id: 'data.id', - product_name: 'data.color', + product_name: 'data.name', product_price: 'data.price', product_quantity: { value: 1 }, }, @@ -165,7 +179,7 @@ export const productAddToCart: Flow.StepExample = { line_items: [ { product_id: 'ers', - product_name: 'black', + product_name: 'Everyday Ruck Snack', product_price: 420, product_quantity: 1, }, @@ -199,12 +213,16 @@ export const orderCompleteCheckout: Flow.StepExample = { 'nested', { condition: (value: unknown) => - isObject(value) && - isObject(value.data) && + typeof value === 'object' && + value !== null && + 'data' in value && + typeof value.data === 'object' && + value.data !== null && + 'price' in value.data && typeof value.data.price === 'number', map: { product_id: 'data.id', - product_name: 'data.color', + product_name: 'data.name', product_price: 'data.price', product_quantity: { value: 1 }, }, @@ -226,12 +244,13 @@ export const orderCompleteCheckout: Flow.StepExample = { line_items: [ { product_id: 'ers', - product_name: 'black', + product_name: 'Everyday Ruck Snack', product_price: 420, product_quantity: 1, }, { product_id: 'cc', + product_name: 'Cool Cap', product_price: 42, product_quantity: 1, }, diff --git a/packages/web/destinations/pinterest/src/schemas/settings.ts b/packages/web/destinations/pinterest/src/schemas/settings.ts index e4209b61c..83d670187 100644 --- a/packages/web/destinations/pinterest/src/schemas/settings.ts +++ b/packages/web/destinations/pinterest/src/schemas/settings.ts @@ -12,13 +12,13 @@ export const SettingsSchema = z pageview: z .boolean() .describe( - 'Fire pintrk("page") once in init after core.js loads. Default true (matches Pinterest\'s base code convention). Set false when walkerOS sources already emit page view events and you would otherwise get a duplicate initial fire.', + 'Call pintrk("page") once in init, as Pinterest\'s base code does. Default true. Set false to leave the call out.', ) .optional(), identify: z .unknown() .describe( - 'walkerOS mapping value resolving to enhanced matching fields: { em?: string, external_id?: string }. em is auto-hashed by the Pinterest JS tag — do not hash before passing. Passed to pintrk("load", tagId, data) on init and pintrk("set", data) when the resolved value changes.', + 'walkerOS mapping value resolving to enhanced matching fields: { em?: string, external_id?: string }. The Pinterest tag takes em plain (it hashes it with SHA-256 in the browser) or already hashed. Resolved on each event and sent with pintrk("set", data) when it changes; init calls pintrk("load", tagId) without it.', ) .optional(), }) diff --git a/packages/web/destinations/snowplow/README.md b/packages/web/destinations/snowplow/README.md index 1ab2618dc..e6424bf67 100644 --- a/packages/web/destinations/snowplow/README.md +++ b/packages/web/destinations/snowplow/README.md @@ -33,7 +33,11 @@ npm install @walkeros/web-destination-snowplow "snowplow": { "package": "@walkeros/web-destination-snowplow", "config": { - "settings": { "collectorUrl": "https://collector.example.com" } + "loadScript": true, + "settings": { + "collectorUrl": "https://collector.example.com", + "pageViewEvent": "page view" + } } } } @@ -42,6 +46,9 @@ npm install @walkeros/web-destination-snowplow } ``` +This sends page views. Every other event needs a rule with a `name` (the action +type) or a `struct`; an event without one fails with `missing-input`. + ## Plugins URL plugins in `settings.plugins` load with sp.js. With `settings.tracker` (your diff --git a/packages/web/destinations/snowplow/src/__tests__/schemas.test.ts b/packages/web/destinations/snowplow/src/__tests__/schemas.test.ts new file mode 100644 index 000000000..0c531e8b1 --- /dev/null +++ b/packages/web/destinations/snowplow/src/__tests__/schemas.test.ts @@ -0,0 +1,19 @@ +import { isObject } from '@walkeros/core'; +import { mapping, settings } from '../schemas'; + +describe('schemas', () => { + it('describes the rule settings the destination reads', () => { + const properties = isObject(mapping.properties) ? mapping.properties : {}; + expect(Object.keys(properties).sort()).toEqual( + ['context', 'data', 'snowplow', 'struct'].sort(), + ); + }); + + it('lists the browser context', () => { + const contexts = + isObject(settings.properties) && isObject(settings.properties.contexts) + ? settings.properties.contexts.properties + : undefined; + expect(isObject(contexts) && Object.keys(contexts)).toContain('browser'); + }); +}); diff --git a/packages/web/destinations/snowplow/src/schemas/mapping.ts b/packages/web/destinations/snowplow/src/schemas/mapping.ts index d29f3e240..a314769d5 100644 --- a/packages/web/destinations/snowplow/src/schemas/mapping.ts +++ b/packages/web/destinations/snowplow/src/schemas/mapping.ts @@ -22,6 +22,24 @@ export const SnowplowMappingSettingsSchema = z.object({ .describe('Override action schema for this specific event'), }); +/** + * Structured event mapping schema (trackStructEvent instead of a + * self-describing event) + */ +export const StructuredEventMappingSchema = z.object({ + category: z.unknown().describe('Mapping value resolving to the category'), + action: z.unknown().describe('Mapping value resolving to the action'), + label: z + .unknown() + .optional() + .describe('Mapping value resolving to the label'), + property: z + .unknown() + .optional() + .describe('Mapping value resolving to the property'), + value: z.unknown().optional().describe('Mapping value resolving to a number'), +}); + /** * Custom mapping parameters schema for Snowplow events * @@ -36,6 +54,15 @@ export const MappingSchema = z.object({ snowplow: SnowplowMappingSettingsSchema.optional().describe( 'Snowplow-specific settings override', ), + data: z + .unknown() + .optional() + .describe( + 'Mapping value with a map resolving to the self-describing event data (like { map: { percentProgress: "data.progress" } })', + ), + struct: StructuredEventMappingSchema.optional().describe( + 'Send a structured event with trackStructEvent instead of a self-describing event; category and action must resolve to strings', + ), }); export type ContextEntity = z.infer; diff --git a/packages/web/destinations/snowplow/src/schemas/settings.ts b/packages/web/destinations/snowplow/src/schemas/settings.ts index ae1cebe0c..4e7b12dee 100644 --- a/packages/web/destinations/snowplow/src/schemas/settings.ts +++ b/packages/web/destinations/snowplow/src/schemas/settings.ts @@ -37,6 +37,7 @@ export const TrackerContextsSchema = z.object({ .boolean() .optional() .describe('Client session context - enables client_session schema'), + browser: z.boolean().optional().describe('Browser context'), performanceTiming: z .boolean() .optional() diff --git a/skills/walkeros-create-cmp-source/SKILL.md b/skills/walkeros-create-cmp-source/SKILL.md index 623446196..ed4285051 100644 --- a/skills/walkeros-create-cmp-source/SKILL.md +++ b/skills/walkeros-create-cmp-source/SKILL.md @@ -806,15 +806,15 @@ capture these variations so they are identified early. ## Reference files -| What | Where | -| ------------------ | ---------------------------------------------- | -| Skill examples | [examples/](examples/) | -| Skill templates | [templates/cmp/](templates/cmp/) | -| Canonical template | `packages/web/sources/cmps/cookiefirst/` | -| Source types | `packages/core/src/types/source.ts` | -| Consent guide | `website/docs/guides/consent/` | -| Usercentrics plan | `docs/plans/2026-02-15-usercentrics-source.md` | -| CookiePro plan | `docs/plans/2026-02-15-cookiepro-source.md` | +| What | Where | +| ------------------- | ----------------------------------------- | +| Skill examples | [examples/](examples/) | +| Skill templates | [templates/cmp/](templates/cmp/) | +| Canonical template | `packages/web/sources/cmps/cookiefirst/` | +| Source types | `packages/core/src/types/source.ts` | +| Consent guide | `website/docs/guides/consent/` | +| Usercentrics source | `packages/web/sources/cmps/usercentrics/` | +| CookiePro source | `packages/web/sources/cmps/cookiepro/` | ## Related skills diff --git a/skills/walkeros-testing-strategy/SKILL.md b/skills/walkeros-testing-strategy/SKILL.md index 2ad7c12fb..b1c602f5f 100644 --- a/skills/walkeros-testing-strategy/SKILL.md +++ b/skills/walkeros-testing-strategy/SKILL.md @@ -416,11 +416,10 @@ approach destinations use — see ## Commands -Verification tier (per `/workspaces/developer/AGENT.md` rule 11): +Verification tiers (run from the repository root): ```bash # L1, the default during a task: typecheck, lint, test for the touched package -cd /workspaces/developer/walkerOS npm run verify:touched -- core npm run verify:touched -- web-destination-gtag diff --git a/skills/walkeros-understanding-development/SKILL.md b/skills/walkeros-understanding-development/SKILL.md index 906a599f4..ad6621785 100644 --- a/skills/walkeros-understanding-development/SKILL.md +++ b/skills/walkeros-understanding-development/SKILL.md @@ -31,8 +31,6 @@ complete. - At plan completion (L2, affected since `origin/main`): `npm run verify:affected` -See `/workspaces/developer/AGENT.md` rule 11 for the full doctrine. - ## XP Principles (Non-Negotiable) | Principle | In Practice | diff --git a/turbo.json b/turbo.json index 928af1370..ac834da91 100644 --- a/turbo.json +++ b/turbo.json @@ -1,6 +1,6 @@ { "$schema": "https://turbo.build/schema.json", - "//": "concurrency: 4 with per-task memory bounds (jest workerIdleMemoryLimit) instead of a global throttle; keep the heavy website/docusaurus build out of inner-loop filters. Do not exceed 4 without re-running the memory probe (docs/plans/2026-05-22-test-perf-optimization.md). dependsOn: ['^build'] is on typecheck ONLY (kept): it preserves strict per-package typecheck (each package checks against upstream compiled .d.ts under its own lib/target), which a single-program source-based typecheck would lose. lint does NOT depend on ^build: eslint here is purely syntactic (@typescript-eslint/parser, no parserOptions.project, no type-aware rules), so it needs no built types. Cold-run speed comes from the remote cache + concurrency, not from weakening typecheck. See /workspaces/developer/AGENT.md rule 11.", + "//": "concurrency: 4 with per-task memory bounds (jest workerIdleMemoryLimit) instead of a global throttle; keep the heavy website/docusaurus build out of inner-loop filters. Do not exceed 4 without measuring peak memory at the new concurrency first. dependsOn: ['^build'] is on typecheck ONLY (kept): it preserves strict per-package typecheck (each package checks against upstream compiled .d.ts under its own lib/target), which a single-program source-based typecheck would lose. lint does NOT depend on ^build: eslint here is purely syntactic (@typescript-eslint/parser, no parserOptions.project, no type-aware rules), so it needs no built types. Cold-run speed comes from the remote cache + concurrency, not from weakening typecheck.", "globalDependencies": [ "**/.env.*local", "website/.env", diff --git a/website/docs/destinations/server/piwikpro.mdx b/website/docs/destinations/server/piwikpro.mdx index 214e345b7..5ac9075cf 100644 --- a/website/docs/destinations/server/piwikpro.mdx +++ b/website/docs/destinations/server/piwikpro.mdx @@ -97,7 +97,7 @@ In the browser, the Piwik PRO tracker sends the visitor's IP address, user agent "config": { "ingest": { "map": { - "ip": { "key": "ip" }, + "ip": { "key": "headers.x-client-ip" }, "userAgent": { "key": "headers.user-agent" }, "language": { "key": "headers.accept-language" } } @@ -108,7 +108,7 @@ In the browser, the Piwik PRO tracker sends the visitor's IP address, user agent language="json" /> -Without it, Piwik PRO receives no client IP, user agent or language, and locates every visitor at your server. Behind a load balancer or a CDN, `ip` is the address of the last proxy; see [client IP behind a proxy](/docs/sources/server/express#client-ip-behind-a-proxy) for the header to map instead. The other server sources take the same map, see the [request scope](/docs/sources/scope). +Without it, Piwik PRO receives no client IP, user agent or language, and locates every visitor at your server. Replace `x-client-ip` with the single-address header your proxy sets or overwrites with the visitor's address, see [Client IP behind a proxy](/docs/sources/server/express#client-ip-behind-a-proxy). The other server sources take the same map, see the [request scope](/docs/sources/scope). ## What each setting sends diff --git a/website/docs/destinations/web/d8a.mdx b/website/docs/destinations/web/d8a.mdx index b6e6951ee..44a9b05ff 100644 --- a/website/docs/destinations/web/d8a.mdx +++ b/website/docs/destinations/web/d8a.mdx @@ -122,6 +122,13 @@ This produces: language="typescript" /> +## Page views + +d8a sends a `page_view` itself when the tracker starts. A walkerOS +`page view` event (the browser source sends one by default) arrives as a +second `page_view`, so each page counts twice. Set `send_page_view: false`, or +ignore the walkerOS event with a rule (`page: { view: { ignore: true } }`). + ## Consent Mode The destination supports d8a's gtag-compatible consent mode. By default, @@ -132,7 +139,11 @@ walkerOS consent keys map to d8a consent fields: | `marketing` | `ad_storage`, `ad_user_data`, `ad_personalization` | | `functional` | `analytics_storage` | -Disable consent mode with `como: false`, or provide a custom mapping via the -`como` setting. +The first consent command sets every mapped field to `denied` before it +applies the update. A walkerOS consent key the mapping does not name changes +nothing: if your consent source reports analytics consent as `analytics`, +`analytics_storage` stays denied until you map it, like +`como: { analytics: 'analytics_storage', marketing: ['ad_storage', 'ad_user_data', 'ad_personalization'] }`. +Disable consent mode with `como: false`. diff --git a/website/docs/destinations/web/linkedin.mdx b/website/docs/destinations/web/linkedin.mdx index 852d40b31..ff54474dc 100644 --- a/website/docs/destinations/web/linkedin.mdx +++ b/website/docs/destinations/web/linkedin.mdx @@ -112,21 +112,28 @@ Add to your `flow.json` destinations: ## Opt-in conversion model -LinkedIn is an **opt-in conversion platform**. Every tracked event must +LinkedIn is an opt-in conversion platform. Every tracked event must reference a pre-created Conversion Rule in Campaign Manager via its numeric `conversion_id`. Events without an explicit `mapping.settings.conversion` produce zero `lintrk` calls. This is the opposite of analytics destinations (Amplitude, Clarity, PostHog) which forward every event by default. Map only the events that matter for attribution. +Events that no rule sends still count as delivered. To report them as skipped +instead, add a wildcard rule with `ignore: true` (`'*': { '*': { ignore: true } }`) +and map the events you want, see [ignore vs silent](/docs/mapping/rule#ignore-vs-skip). + ## Falsy guards The destination protects against misconfigured mappings: -- If `id` resolves to falsy or `0` the entire `lintrk` call is skipped (no - silent error or partial conversion). -- If `value` / `currency` / `eventId` resolve to falsy, those individual - fields are dropped from the call. `id` is always required. +- If `id` resolves to nothing, `0` or a non-numeric string, no `lintrk` call + is made and the push fails with `missing-input`: the push result lists the + destination under `failed` and the collector writes an ERROR line. A numeric + string such as `"67890"` is sent as a number. +- `value` (a number or numeric string), `currency` and `eventId` (non-empty + strings) are left out of the call when they resolve to anything else. A + `value` of `0` is sent. ## Consent @@ -142,29 +149,26 @@ LinkedIn is an advertising platform. The walkerOS consent key is **`marketing`** language="typescript" /> -The walkerOS `config.consent` gate blocks unconsented events from reaching -the destination in the first place. The Insight Tag itself has no vendor -opt-out API. When consent is later revoked, walkerOS stops sending events, -but the loaded tag stays in the page. - -Events that no rule sends still count as delivered. To report them as skipped -instead, add a wildcard rule with `ignore: true` (`'*': { '*': { ignore: true } }`) -and map the events you want. +Until `marketing` is granted, the collector holds the destination's events in +a queue and sends them once consent is granted, see +[consent management](/docs/guides/consent). The Insight Tag itself has no +vendor opt-out API: after a revocation, walkerOS holds new events again, but +the loaded tag stays in the page. ### Deferred script load -If you want the Insight Tag to load **only** after consent is granted, set -`loadScript: true` and leave `config.consent.marketing` un-granted at init -time. The destination will not load the script during init and injects it -the first time an `on('consent')` event grants `marketing`. The -handler is idempotent. If a `