Feature Description
aws_policy_equal(a, b) and json_equal(a, b) should return NULL when either argument is SQL NULL, instead of raising Invalid policy strings / Invalid JSON strings. Parse failures on non-NULL input should stay errors.
This would make them consistent with every other function in public/sqlfuncs. split_part and all four regexp functions already return NULL for NULL input, and their golden vectors pin that. It also matches the SQLite convention noted in public/sqlfuncs/CLAUDE.md ("SQLite convention is NULL in -> NULL out").
Example(s)
Cloud Control reads return SQL NULL for optional properties that have never been set. On an S3 bucket with no versioning configuration and no tags:
SELECT
versioning_configuration IS NULL AS versioning_is_null, -- 1
tags IS NULL AS tags_is_null -- 1
FROM awscc.s3.buckets
WHERE region = 'us-east-1'
AND Identifier = 'my-bucket';
Comparing either of these to a desired state fails the whole query:
SELECT AWS_POLICY_EQUAL(versioning_configuration, '{"Status":"Suspended"}') AS test_versioning
FROM awscc.s3.buckets
WHERE region = 'us-east-1'
AND Identifier = 'my-bucket';
-- SQL logic error: Invalid policy strings (1)
The main problem is that this fails the whole query, not just the row. Suppose you compare a property across many resources, for example AWS_POLICY_EQUAL(tags, '[...]') across a literal IN (...) list of buckets. A single untagged bucket makes the query error out, when that row should just come back as not equal.
For stackql-deploy, every statecheck on an optional property needs a guard today:
SELECT COUNT(*) AS count
FROM (
SELECT
AWS_POLICY_EQUAL(COALESCE(ownership_controls, '{}'), '{{ ownership_controls }}') AS test_ownership_controls,
AWS_POLICY_EQUAL(COALESCE(public_access_block_configuration, '{}'), '{{ public_access_block_configuration }}') AS test_public_access_block_configuration,
AWS_POLICY_EQUAL(COALESCE(versioning_configuration, '{}'), '{{ versioning_configuration }}') AS test_versioning_configuration,
AWS_POLICY_EQUAL(COALESCE(tags, '[]'), '{{ tags }}') AS test_tags
FROM awscc.s3.buckets
WHERE region = '{{ region }}'
AND Identifier = '{{ bucket_name }}'
) t
WHERE test_ownership_controls = 1
AND test_public_access_block_configuration = 1
AND test_versioning_configuration = 1
AND test_tags = 1
With NULL in -> NULL out, the guards go away. NULL = 1 is not true, so the row is filtered out, count is 0 and the statecheck fails cleanly, which then triggers the update. The deploy outcome is the same as today (stackql-deploy already treats a query error as a failed check), without an error on every fresh or drifted resource.
Possible Approaches or Libraries to Consider
- In
AWSPolicyEqual (awspolicyequal.go) and JSONEqual (jsonequal.go), return nil, nil when valueText reports a NULL argument, instead of errInvalidPolicyArgs / errInvalidJSONArgs. Remove the two error vars if nothing else uses them.
- Update the four golden vectors that pin the current behaviour:
cref_null_lhs and cref_null_rhs in testdata/aws_policy_equal.json and testdata/json_equal.json. Change "error": true to "result": null, and keep a note saying the retired C implementation raised an error.
- Add a numbered divergence to
DIVERGENCES.md under "JSON parsing". Also update the two "Preserved C behaviors" bullets that currently say NULL arguments are errors.
- Keep the byte-identical shortcut in
aws_policy_equal as is. It runs after the NULL check, so a pair of NULLs returns NULL, not 1.
Alternatives considered:
- NULL vs NULL -> 1. Rejected. It breaks SQL three-valued logic, and callers who want that can use
IS or COALESCE explicitly.
- Treat SQL NULL as the JSON literal
null, returning 0 against any non-null document. This also removes the guards and keeps the projection non-NULL. But it would be the only function in the package that doesn't propagate NULL.
Additional context
Compatibility: queries that succeed today return the same results. Only queries that currently fail with Invalid policy strings / Invalid JSON strings change behaviour, and they now return NULL for the affected rows.
Found while moving a stackql-deploy stack's S3 buckets from aws.s3.buckets to awscc.s3.buckets with explicit ownership, encryption, public access block, versioning and tags controls.
Feature Description
aws_policy_equal(a, b)andjson_equal(a, b)should returnNULLwhen either argument is SQLNULL, instead of raisingInvalid policy strings/Invalid JSON strings. Parse failures on non-NULL input should stay errors.This would make them consistent with every other function in
public/sqlfuncs.split_partand all four regexp functions already return NULL for NULL input, and their golden vectors pin that. It also matches the SQLite convention noted inpublic/sqlfuncs/CLAUDE.md("SQLite convention is NULL in -> NULL out").Example(s)
Cloud Control reads return SQL
NULLfor optional properties that have never been set. On an S3 bucket with no versioning configuration and no tags:Comparing either of these to a desired state fails the whole query:
The main problem is that this fails the whole query, not just the row. Suppose you compare a property across many resources, for example
AWS_POLICY_EQUAL(tags, '[...]')across a literalIN (...)list of buckets. A single untagged bucket makes the query error out, when that row should just come back as not equal.For
stackql-deploy, every statecheck on an optional property needs a guard today:With NULL in -> NULL out, the guards go away.
NULL = 1is not true, so the row is filtered out,countis 0 and the statecheck fails cleanly, which then triggers the update. The deploy outcome is the same as today (stackql-deploy already treats a query error as a failed check), without an error on every fresh or drifted resource.Possible Approaches or Libraries to Consider
AWSPolicyEqual(awspolicyequal.go) andJSONEqual(jsonequal.go), returnnil, nilwhenvalueTextreports a NULL argument, instead oferrInvalidPolicyArgs/errInvalidJSONArgs. Remove the two error vars if nothing else uses them.cref_null_lhsandcref_null_rhsintestdata/aws_policy_equal.jsonandtestdata/json_equal.json. Change"error": trueto"result": null, and keep a note saying the retired C implementation raised an error.DIVERGENCES.mdunder "JSON parsing". Also update the two "Preserved C behaviors" bullets that currently say NULL arguments are errors.aws_policy_equalas is. It runs after the NULL check, so a pair of NULLs returns NULL, not 1.Alternatives considered:
ISorCOALESCEexplicitly.null, returning 0 against any non-null document. This also removes the guards and keeps the projection non-NULL. But it would be the only function in the package that doesn't propagate NULL.Additional context
Compatibility: queries that succeed today return the same results. Only queries that currently fail with
Invalid policy strings/Invalid JSON stringschange behaviour, and they now return NULL for the affected rows.Found while moving a
stackql-deploystack's S3 buckets fromaws.s3.bucketstoawscc.s3.bucketswith explicit ownership, encryption, public access block, versioning and tags controls.