Docs: Add custom SQL examples, validatorClass values, and failure thresholds to test API pages

v1.13.x/api-reference/data-quality/test-cases/create.mdx

@@ -18,7 +18,7 @@ The server associates the test case with the basic suite of the table in `entity
 </ParamField>
 
 <ParamField body="testDefinition" type="string" required>
-  Fully qualified name (FQN) of the test definition to use. For example, `columnValueMaxToBeBetween`.
+  Fully qualified name (FQN) of the test definition to use. For example, `columnValueMaxToBeBetween`. For a custom SQL test (the **Custom SQL Query** test type in the UI), use `tableCustomSQLQuery`. For a test case on a custom test definition you created, use that definition's name, such as `columnValuesMaxLength`. For more information, see [Custom SQL Test Cases](#custom-sql-test-cases).
 </ParamField>
 
 <ParamField body="entityLink" type="string" required>
@@ -140,6 +140,39 @@ curl -X POST "{base_url}/api/v1/dataQuality/testCases" \
     "description": "Validates shop_id max value stays between 1 and 100"
   }'
 ```
+
+```bash POST /v1/dataQuality/testCases (custom SQL)
+curl -X POST "{base_url}/api/v1/dataQuality/testCases" \
+  -H "Authorization: Bearer {access_token}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "name": "orders_no_negative_amounts",
+    "testDefinition": "tableCustomSQLQuery",
+    "entityLink": "<#E::table::sample_data.ecommerce_db.shopify.orders>",
+    "parameterValues": [
+      {"name": "sqlExpression", "value": "SELECT * FROM orders WHERE amount < 0"},
+      {"name": "strategy", "value": "ROWS"},
+      {"name": "operator", "value": "=="},
+      {"name": "threshold", "value": "0"}
+    ],
+    "computePassedFailedRowCount": true
+  }'
+```
+
+```bash POST /v1/dataQuality/testCases (custom test definition)
+curl -X POST "{base_url}/api/v1/dataQuality/testCases" \
+  -H "Authorization: Bearer {access_token}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "name": "email_max_length",
+    "testDefinition": "columnValuesMaxLength",
+    "entityLink": "<#E::table::sample_data.ecommerce_db.shopify.dim_address::columns::email>",
+    "parameterValues": [
+      {"name": "maxLength", "value": "50"}
+    ],
+    "computePassedFailedRowCount": true
+  }'
+```
 </RequestExample>
 
 <ResponseExample>
@@ -294,8 +327,15 @@ curl -X PUT "{base_url}/api/v1/dataQuality/testCases" \
 
 ## Custom SQL Test Cases
 
+OpenMetadata has two ways to run SQL in a test case:
+
+- **Custom SQL test**: the **Custom SQL Query** test type in the UI. The test case uses the built-in `tableCustomSQLQuery` test definition and carries the SQL itself in `parameterValues`. You don't create a test definition.
+- **Custom test definition**: a reusable SQL template you create once, with `{{ paramName }}` placeholders. Test cases then reference it by name. For more information, see [Test Cases for a Custom Test Definition](#test-cases-for-a-custom-test-definition).
+
 A test case passes values to a SQL-based test definition through `parameterValues`. The server rejects any parameter that the test definition doesn't declare in `parameterDefinition`, and requires every parameter marked `required`.
 
+### Custom SQL Test
+
 The built-in `tableCustomSQLQuery` test definition runs a SQL query against a table. It accepts these parameters:
 
 | Parameter | Required | Description |
@@ -316,13 +356,37 @@ curl -X POST "{base_url}/api/v1/dataQuality/testCases" \
     "entityLink": "<#E::table::sample_data.ecommerce_db.shopify.orders>",
     "parameterValues": [
       {"name": "sqlExpression", "value": "SELECT * FROM orders WHERE amount < 0"},
-      {"name": "strategy", "value": "ROWS"}
+      {"name": "strategy", "value": "ROWS"},
+      {"name": "operator", "value": "=="},
+      {"name": "threshold", "value": "0"}
+    ],
+    "computePassedFailedRowCount": true
+  }'
+```
+
+### Test Cases for a Custom Test Definition
+
+A test case for a custom SQL test definition sets `testDefinition` to the definition's name and passes one `parameterValues` entry per `{{ paramName }}` placeholder. The following request uses a custom `columnValuesMaxLength` definition, which declares `maxLength`:
+
+```bash
+curl -X POST "{base_url}/api/v1/dataQuality/testCases" \
+  -H "Authorization: Bearer {access_token}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "name": "email_max_length",
+    "testDefinition": "columnValuesMaxLength",
+    "entityLink": "<#E::table::sample_data.ecommerce_db.shopify.dim_address::columns::email>",
+    "parameterValues": [
+      {"name": "maxLength", "value": "50"}
     ],
     "computePassedFailedRowCount": true
   }'
 ```
 
-For a custom test definition that uses `{{ paramName }}` placeholders, pass one `parameterValues` entry per placeholder. For more information, see [Create a Test Definition](/v1.13.x/api-reference/data-quality/test-definitions/create#custom-sql-test-definitions).
+- **Pass condition**: The test case passes only when the number of rows the SQL returns is 0.
+- **Row counts**: Set `computePassedFailedRowCount` to `true` to report passed and failed row counts.
+
+For more information, see [Create a Test Definition](/v1.13.x/api-reference/data-quality/test-definitions/create#custom-sql-test-definitions).
 
 ## Error Handling
 

v1.13.x/api-reference/data-quality/test-definitions/create.mdx

@@ -68,7 +68,12 @@ Create a custom test definition to define organization-specific data quality val
 </ParamField>
 
 <ParamField body="validatorClass" type="string">
-  Name of the validator that runs the test. For custom SQL-based tests, defaults to `ruleLibrarySqlExpressionValidator` when omitted.
+  Class name of the validator that runs the test. For a custom SQL test definition, set one of the following values. The OpenMetadata UI sets the same values when you create a custom SQL test definition:
+
+  - `ColumnRuleLibrarySqlExpressionValidator`: use when `entityType` is `COLUMN`.
+  - `TableRuleLibrarySqlExpressionValidator`: use when `entityType` is `TABLE`.
+
+  Built-in test definitions use their own validators, which you can't select for a custom definition. The server accepts any string for this field, so always set it explicitly for SQL-based definitions.
 </ParamField>
 
 <ParamField body="dataQualityDimension" type="string">
@@ -113,24 +118,21 @@ configure(
 )
 
 request = CreateTestDefinitionRequest(
-    name="columnValuesToMatchBusinessRule",
-    displayName="Column Values To Match Business Rule",
-    description="Validates that column values conform to a custom business rule regex pattern",
+    name="columnValuesMaxLength",
+    displayName="Column Values Max Length",
+    description="Fails when a value is longer than the allowed length",
     entityType="COLUMN",
     testPlatforms=["OpenMetadata"],
+    dataQualityDimension="SQL",
     supportedDataTypes=["STRING", "VARCHAR", "TEXT"],
+    sqlExpression="SELECT * FROM {{ table_name }} WHERE LENGTH({{ column_name }}) > {{ maxLength }}",
+    validatorClass="ColumnRuleLibrarySqlExpressionValidator",
     parameterDefinition=[
         {
-            "name": "regexPattern",
-            "dataType": "STRING",
-            "required": True,
-            "description": "The regex pattern to match against"
-        },
-        {
-            "name": "threshold",
+            "name": "maxLength",
             "dataType": "INT",
-            "required": False,
-            "description": "Maximum number of non-matching values allowed"
+            "required": True,
+            "description": "Maximum allowed length"
         }
     ]
 )
@@ -177,25 +179,17 @@ curl -X POST "{base_url}/api/v1/dataQuality/testDefinitions" \
   -H "Authorization: Bearer {access_token}" \
   -H "Content-Type: application/json" \
   -d '{
-    "name": "columnValuesToMatchBusinessRule",
-    "displayName": "Column Values To Match Business Rule",
-    "description": "Validates that column values conform to a custom business rule regex pattern",
+    "name": "columnValuesMaxLength",
+    "displayName": "Column Values Max Length",
+    "description": "Fails when a value is longer than the allowed length",
     "entityType": "COLUMN",
     "testPlatforms": ["OpenMetadata"],
+    "dataQualityDimension": "SQL",
     "supportedDataTypes": ["STRING", "VARCHAR", "TEXT"],
+    "sqlExpression": "SELECT * FROM {{ table_name }} WHERE LENGTH({{ column_name }}) > {{ maxLength }}",
+    "validatorClass": "ColumnRuleLibrarySqlExpressionValidator",
     "parameterDefinition": [
-      {
-        "name": "regexPattern",
-        "dataType": "STRING",
-        "required": true,
-        "description": "The regex pattern to match against"
-      },
-      {
-        "name": "threshold",
-        "dataType": "INT",
-        "required": false,
-        "description": "Maximum number of non-matching values allowed"
-      }
+      {"name": "maxLength", "dataType": "INT", "required": true, "description": "Maximum allowed length"}
     ]
   }'
 ```
@@ -321,6 +315,8 @@ Returns the created test definition object with all specified properties and sys
 
 ## Custom SQL Test Definitions
 
+A custom test definition is a reusable SQL template. To run a one-off SQL query on a single table, such as the **Custom SQL Query** test type in the UI, create a test case with the built-in `tableCustomSQLQuery` definition instead. For more information, see [Custom SQL Test Cases](/v1.13.x/api-reference/data-quality/test-cases/create#custom-sql-test-cases).
+
 To create a test definition that runs your own SQL, set `sqlExpression` and declare every `{{ paramName }}` placeholder in `parameterDefinition`. The following request creates a column-level test that fails when any value in the column is longer than a limit:
 
 ```bash
@@ -336,13 +332,9 @@ curl -X POST "{base_url}/api/v1/dataQuality/testDefinitions" \
     "dataQualityDimension": "SQL",
     "supportedDataTypes": ["STRING", "VARCHAR", "TEXT"],
     "sqlExpression": "SELECT * FROM {{ table_name }} WHERE LENGTH({{ column_name }}) > {{ maxLength }}",
+    "validatorClass": "ColumnRuleLibrarySqlExpressionValidator",
     "parameterDefinition": [
-      {
-        "name": "maxLength",
-        "dataType": "INT",
-        "required": true,
-        "description": "Maximum allowed length"
-      }
+      {"name": "maxLength", "dataType": "INT", "required": true, "description": "Maximum allowed length"}
     ]
   }'
 ```

v2.0.x/api-reference/data-quality/test-cases/create.mdx

@@ -18,7 +18,7 @@ The server associates the test case with the basic suite of the table in `entity
 </ParamField>
 
 <ParamField body="testDefinition" type="string" required>
-  Fully qualified name (FQN) of the test definition to use. For example, `columnValueMaxToBeBetween`.
+  Fully qualified name (FQN) of the test definition to use. For example, `columnValueMaxToBeBetween`. For a custom SQL test (the **Custom SQL Query** test type in the UI), use `tableCustomSQLQuery`. For a test case on a custom test definition you created, use that definition's name, such as `columnValuesMaxLength`. For more information, see [Custom SQL Test Cases](#custom-sql-test-cases).
 </ParamField>
 
 <ParamField body="entityLink" type="string" required>
@@ -140,6 +140,39 @@ curl -X POST "{base_url}/api/v1/dataQuality/testCases" \
     "description": "Validates shop_id max value stays between 1 and 100"
   }'
 ```
+
+```bash POST /v1/dataQuality/testCases (custom SQL)
+curl -X POST "{base_url}/api/v1/dataQuality/testCases" \
+  -H "Authorization: Bearer {access_token}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "name": "orders_no_negative_amounts",
+    "testDefinition": "tableCustomSQLQuery",
+    "entityLink": "<#E::table::sample_data.ecommerce_db.shopify.orders>",
+    "parameterValues": [
+      {"name": "sqlExpression", "value": "SELECT * FROM orders WHERE amount < 0"},
+      {"name": "strategy", "value": "ROWS"},
+      {"name": "operator", "value": "=="},
+      {"name": "threshold", "value": "0"}
+    ],
+    "computePassedFailedRowCount": true
+  }'
+```
+
+```bash POST /v1/dataQuality/testCases (custom test definition)
+curl -X POST "{base_url}/api/v1/dataQuality/testCases" \
+  -H "Authorization: Bearer {access_token}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "name": "email_max_length",
+    "testDefinition": "columnValuesMaxLength",
+    "entityLink": "<#E::table::sample_data.ecommerce_db.shopify.dim_address::columns::email>",
+    "parameterValues": [
+      {"name": "maxLength", "value": "50"}
+    ],
+    "computePassedFailedRowCount": true
+  }'
+```
 </RequestExample>
 
 <ResponseExample>
@@ -294,8 +327,15 @@ curl -X PUT "{base_url}/api/v1/dataQuality/testCases" \
 
 ## Custom SQL Test Cases
 
+OpenMetadata has two ways to run SQL in a test case:
+
+- **Custom SQL test**: the **Custom SQL Query** test type in the UI. The test case uses the built-in `tableCustomSQLQuery` test definition and carries the SQL itself in `parameterValues`. You don't create a test definition.
+- **Custom test definition**: a reusable SQL template you create once, with `{{ paramName }}` placeholders. Test cases then reference it by name. For more information, see [Test Cases for a Custom Test Definition](#test-cases-for-a-custom-test-definition).
+
 A test case passes values to a SQL-based test definition through `parameterValues`. The server rejects any parameter that the test definition doesn't declare in `parameterDefinition`, and requires every parameter marked `required`.
 
+### Custom SQL Test
+
 The built-in `tableCustomSQLQuery` test definition runs a SQL query against a table. It accepts these parameters:
 
 | Parameter | Required | Description |
@@ -316,13 +356,37 @@ curl -X POST "{base_url}/api/v1/dataQuality/testCases" \
     "entityLink": "<#E::table::sample_data.ecommerce_db.shopify.orders>",
     "parameterValues": [
       {"name": "sqlExpression", "value": "SELECT * FROM orders WHERE amount < 0"},
-      {"name": "strategy", "value": "ROWS"}
+      {"name": "strategy", "value": "ROWS"},
+      {"name": "operator", "value": "=="},
+      {"name": "threshold", "value": "0"}
+    ],
+    "computePassedFailedRowCount": true
+  }'
+```
+
+### Test Cases for a Custom Test Definition
+
+A test case for a custom SQL test definition sets `testDefinition` to the definition's name and passes one `parameterValues` entry per `{{ paramName }}` placeholder. The following request uses a custom `columnValuesMaxLength` definition, which declares `maxLength`:
+
+```bash
+curl -X POST "{base_url}/api/v1/dataQuality/testCases" \
+  -H "Authorization: Bearer {access_token}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "name": "email_max_length",
+    "testDefinition": "columnValuesMaxLength",
+    "entityLink": "<#E::table::sample_data.ecommerce_db.shopify.dim_address::columns::email>",
+    "parameterValues": [
+      {"name": "maxLength", "value": "50"}
     ],
     "computePassedFailedRowCount": true
   }'
 ```
 
-For a custom test definition that uses `{{ paramName }}` placeholders, pass one `parameterValues` entry per placeholder. For more information, see [Create a Test Definition](/v2.0.x/api-reference/data-quality/test-definitions/create#custom-sql-test-definitions).
+- **Pass condition**: The test case passes only when the number of rows the SQL returns is 0.
+- **Row counts**: Set `computePassedFailedRowCount` to `true` to report passed and failed row counts.
+
+For more information, see [Create a Test Definition](/v2.0.x/api-reference/data-quality/test-definitions/create#custom-sql-test-definitions).
 
 ## Error Handling