diff --git a/docs/source/how-to/configuration/use-schema-registry.ipynb b/docs/source/how-to/configuration/use-schema-registry.ipynb new file mode 100644 index 0000000..57d7ff5 --- /dev/null +++ b/docs/source/how-to/configuration/use-schema-registry.ipynb @@ -0,0 +1,1166 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "74c407c2", + "metadata": {}, + "source": [ + "# Use the Schema Registry\n", + "\n", + "This guide shows how to register configuration schemas and use the `SchemaRegistry` to validate configuration and generate JSON Schemas." + ] + }, + { + "cell_type": "markdown", + "id": "d75b9a62", + "metadata": {}, + "source": [ + "## Create a Registry\n", + "\n", + "When creating a new schema registry it will be empty. Schemas can be registered manually or automatically by using the discovery functionality which will be explained later in this guide." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "ac5d5210", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "SchemaRegistry({})\n" + ] + } + ], + "source": [ + "from pyaml.validation import SchemaRegistry\n", + "\n", + "registry = SchemaRegistry()\n", + "print(registry)" + ] + }, + { + "cell_type": "markdown", + "id": "210f6040", + "metadata": {}, + "source": [ + "The registry is implemented as a singleton which means the same object will be returned every time you create a new instance of it during the same session. Schemas therefore only have to be registered once per session." + ] + }, + { + "cell_type": "markdown", + "id": "1066cc68", + "metadata": {}, + "source": [ + "## Register a Schema\n", + "\n", + "There are two ways to register a schema:\n", + "\n", + "1. Use the `register` method\n", + "2. Use the `register_schema` decorator to automatically register schemas when a module is loaded\n", + "\n", + " The decorator can be used in two ways: dynamically generating the schema from the class constructor or by explicitly declaring a schema too use for the class.\n", + "\n", + " Note: a decorator only runs when a module is imported. If you use the `register_schema` decorator in a module that is not imported the schema will not be registered.\n", + "\n", + "The different ways to use the decorator is explained below." + ] + }, + { + "cell_type": "markdown", + "id": "0433de5b", + "metadata": {}, + "source": [ + "### Register a Dynamic Schema\n", + "\n", + "The decorator generates a `ConfigurationSchema` and registers it in the schema registry using the class's fully qualified path." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "2d0cf526", + "metadata": {}, + "outputs": [], + "source": [ + "from pyaml.validation import register_schema\n", + "\n", + "@register_schema\n", + "class Magnet:\n", + " def __init__(self, length: float):\n", + " self.length = length" + ] + }, + { + "cell_type": "markdown", + "id": "88e76217", + "metadata": {}, + "source": [ + "The schema is now visible in the registry. In this case the schema will get the path `__main__.Magnet` since the class was declared directly in the script. For other classes it will be of the form `package.module.Class`.\n", + "\n", + "You can use `describe` to get pretty output of the fields in the schema. `ConfigurationSchema` inherits from Pydantic `BaseModel` so the functionality of that class is also available. See Pydantic's documentation for details." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "fa6e5c5f", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "SchemaRegistry(\n", + " '__main__.Magnet': __main__.MagnetConfigurationSchema,\n", + ")\n", + "MagnetConfigurationSchema(\n", + " class_path: str — Fully qualified class path.\n", + " length: float\n", + ")\n" + ] + } + ], + "source": [ + "# See the content of the registry\n", + "print(registry)\n", + "\n", + "# See the fields in the model\n", + "print(registry[\"__main__.Magnet\"].describe())" + ] + }, + { + "cell_type": "markdown", + "id": "2de37063", + "metadata": {}, + "source": [ + "### Register an Explicit Schema\n", + "\n", + "Define a `ConfigurationSchema` explicitly when you need more control over the configuration fields or validation rules. The functionality of Pydantic is available when defining the schema.\n", + "\n", + "The explicit schema must inherit from `ConfigurationSchema` since that defined the minimum required fields for all items in the registry." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "cd900de7", + "metadata": {}, + "outputs": [], + "source": [ + "from pyaml.validation import ConfigurationSchema, register_schema\n", + "\n", + "class DifferentMagnetSchema(ConfigurationSchema):\n", + " length: float\n", + "\n", + "@register_schema(DifferentMagnetSchema)\n", + "class DifferentMagnet:\n", + " def __init__(self, length: float):\n", + " self.length = length" + ] + }, + { + "cell_type": "markdown", + "id": "d66a055c", + "metadata": {}, + "source": [ + "Bot schemas are now available in the registry." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "dbf52aa4", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "SchemaRegistry(\n", + " '__main__.DifferentMagnet': __main__.DifferentMagnetSchema,\n", + " '__main__.Magnet': __main__.MagnetConfigurationSchema,\n", + ")\n" + ] + } + ], + "source": [ + "print(registry)" + ] + }, + { + "cell_type": "markdown", + "id": "a2a1596b", + "metadata": {}, + "source": [ + "## Automatically Discover Schemas\n", + "\n", + "Since registration only happens when the modules containing decorated classes are imported, a `discover` method is available to automatically scan packages and register the schemas in the package." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "99523615", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "SchemaRegistry(\n", + " 'abc.ABC': abc.ABCConfigurationSchema,\n", + " 'pyaml.arrays.array.ArrayConfig': pyaml.arrays.array.ArrayConfigConfigurationSchema,\n", + " 'pyaml.arrays.bpm.BPM': pyaml.arrays.bpm.BPMConfigurationSchema,\n", + " 'pyaml.arrays.cfm_magnet.CombinedFunctionMagnet': pyaml.arrays.cfm_magnet.CombinedFunctionMagnetConfigurationSchema,\n", + " 'pyaml.arrays.element.Element': pyaml.arrays.element.ElementConfigurationSchema,\n", + " 'pyaml.arrays.magnet.Magnet': pyaml.arrays.magnet.MagnetConfigurationSchema,\n", + " 'pyaml.arrays.serialized_magnet.SerializedMagnets': pyaml.arrays.serialized_magnet.SerializedMagnetsConfigurationSchema,\n", + " 'pyaml.bpm.bpm.BPM': pyaml.bpm.bpm.BPMConfigurationSchema,\n", + " 'pyaml.common.element.Element': pyaml.common.element.ElementConfigurationSchema,\n", + " 'pyaml.common.holders.element_holder.ElementHolder': pyaml.common.holders.element_holder.ElementHolderConfigurationSchema,\n", + " 'pyaml.diagnostics.atune_monitor.ABetatronTuneMonitor': pyaml.diagnostics.atune_monitor.ABetatronTuneMonitorConfigurationSchema,\n", + " 'pyaml.diagnostics.tune_monitor.BetatronTuneMonitor': pyaml.diagnostics.tune_monitor.BetatronTuneMonitorConfigurationSchema,\n", + " 'pyaml.lattice.attribute_linker.PyAtAttributeElementsLinker': pyaml.lattice.attribute_linker.PyAtAttributeElementsLinkerConfigurationSchema,\n", + " 'pyaml.lattice.lattice_elements_linker.LatticeElementsLinker': pyaml.lattice.lattice_elements_linker.LatticeElementsLinkerConfigurationSchema,\n", + " 'pyaml.lattice.lattice_elements_linker.LinkerConfigModel': pyaml.lattice.lattice_elements_linker.LinkerConfigModelConfigurationSchema,\n", + " 'pyaml.lattice.simulator.Simulator': pyaml.lattice.simulator.SimulatorConfigurationSchema,\n", + " 'pyaml.magnet.cfm_magnet.CombinedFunctionMagnet': pyaml.magnet.cfm_magnet.CombinedFunctionMagnetConfigurationSchema,\n", + " 'pyaml.magnet.csvcurve.CSVCurve': pyaml.magnet.csvcurve.CSVCurveConfigurationSchema,\n", + " 'pyaml.magnet.csvmatrix.CSVMatrix': pyaml.magnet.csvmatrix.CSVMatrixConfigurationSchema,\n", + " 'pyaml.magnet.curve.Curve': pyaml.magnet.curve.CurveConfigurationSchema,\n", + " 'pyaml.magnet.hcorrector.HCorrector': pyaml.magnet.hcorrector.HCorrectorConfigurationSchema,\n", + " 'pyaml.magnet.identity_cfm_model.IdentityCFMagnetModel': pyaml.magnet.identity_cfm_model.IdentityCFMagnetModelConfigurationSchema,\n", + " 'pyaml.magnet.identity_model.IdentityMagnetModel': pyaml.magnet.identity_model.IdentityMagnetModelConfigurationSchema,\n", + " 'pyaml.magnet.inline_curve.InlineCurve': pyaml.magnet.inline_curve.InlineCurveConfigurationSchema,\n", + " 'pyaml.magnet.inline_matrix.InlineMatrix': pyaml.magnet.inline_matrix.InlineMatrixConfigurationSchema,\n", + " 'pyaml.magnet.linear_cfm_model.LinearCFMagnetModel': pyaml.magnet.linear_cfm_model.LinearCFMagnetModelConfigurationSchema,\n", + " 'pyaml.magnet.linear_model.LinearMagnetModel': pyaml.magnet.linear_model.LinearMagnetModelConfigurationSchema,\n", + " 'pyaml.magnet.linear_serialized_model.LinearSerializedMagnetModel': pyaml.magnet.linear_serialized_model.LinearSerializedMagnetModelConfigurationSchema,\n", + " 'pyaml.magnet.magnet.Magnet': pyaml.magnet.magnet.MagnetConfigurationSchema,\n", + " 'pyaml.magnet.matrix.Matrix': pyaml.magnet.matrix.MatrixConfigurationSchema,\n", + " 'pyaml.magnet.model.MagnetModel': pyaml.magnet.model.MagnetModelConfigurationSchema,\n", + " 'pyaml.magnet.octupole.Octupole': pyaml.magnet.octupole.OctupoleConfigurationSchema,\n", + " 'pyaml.magnet.quadrupole.Quadrupole': pyaml.magnet.quadrupole.QuadrupoleConfigurationSchema,\n", + " 'pyaml.magnet.serialized_magnet.SerializedMagnets': pyaml.magnet.serialized_magnet.SerializedMagnetsConfigurationSchema,\n", + " 'pyaml.magnet.sextupole.Sextupole': pyaml.magnet.sextupole.SextupoleConfigurationSchema,\n", + " 'pyaml.magnet.skewoctu.SkewOctu': pyaml.magnet.skewoctu.SkewOctuConfigurationSchema,\n", + " 'pyaml.magnet.skewquad.SkewQuad': pyaml.magnet.skewquad.SkewQuadConfigurationSchema,\n", + " 'pyaml.magnet.skewsext.SkewSext': pyaml.magnet.skewsext.SkewSextConfigurationSchema,\n", + " 'pyaml.magnet.spline_model.SplineMagnetModel': pyaml.magnet.spline_model.SplineMagnetModelConfigurationSchema,\n", + " 'pyaml.magnet.vcorrector.VCorrector': pyaml.magnet.vcorrector.VCorrectorConfigurationSchema,\n", + " 'pyaml.rf.rf_plant.RFPlant': pyaml.rf.rf_plant.RFPlantConfigurationSchema,\n", + " 'pyaml.rf.rf_transmitter.RFTransmitter': pyaml.rf.rf_transmitter.RFTransmitterConfigurationSchema,\n", + " 'pyaml.tuning_tools.bba.BBA': pyaml.tuning_tools.bba.BBAConfigurationSchema,\n", + " 'pyaml.tuning_tools.bba2.BBA2': pyaml.tuning_tools.bba2.BBA2ConfigurationSchema,\n", + " 'pyaml.tuning_tools.chromaticity.Chromaticity': pyaml.tuning_tools.chromaticity.ChromaticityConfigurationSchema,\n", + " 'pyaml.tuning_tools.chromaticity_monitor.ChromaticityMonitor': pyaml.tuning_tools.chromaticity_monitor.ChromaticityMonitorConfigurationSchema,\n", + " 'pyaml.tuning_tools.chromaticity_response_matrix.ChromaticityResponseMatrix': pyaml.tuning_tools.chromaticity_response_matrix.ChromaticityResponseMatrixConfigurationSchema,\n", + " 'pyaml.tuning_tools.dispersion.Dispersion': pyaml.tuning_tools.dispersion.DispersionConfigurationSchema,\n", + " 'pyaml.tuning_tools.measurement_tool.MeasurementTool': pyaml.tuning_tools.measurement_tool.MeasurementToolConfigurationSchema,\n", + " 'pyaml.tuning_tools.orbit.Orbit': pyaml.tuning_tools.orbit.OrbitConfigurationSchema,\n", + " 'pyaml.tuning_tools.orbit_response_matrix.OrbitResponseMatrix': pyaml.tuning_tools.orbit_response_matrix.OrbitResponseMatrixConfigurationSchema,\n", + " 'pyaml.tuning_tools.orbit_response_matrix_data.OrbitResponseMatrixData': pyaml.tuning_tools.orbit_response_matrix_data.OrbitResponseMatrixDataConfigurationSchema,\n", + " 'pyaml.tuning_tools.response_matrix_data.ResponseMatrixData': pyaml.tuning_tools.response_matrix_data.ResponseMatrixDataConfigurationSchema,\n", + " 'pyaml.tuning_tools.tune.Tune': pyaml.tuning_tools.tune.TuneConfigurationSchema,\n", + " 'pyaml.tuning_tools.tune_response_matrix.TuneResponseMatrix': pyaml.tuning_tools.tune_response_matrix.TuneResponseMatrixConfigurationSchema,\n", + " 'pyaml.tuning_tools.tuning_tool.TuningTool': pyaml.tuning_tools.tuning_tool.TuningToolConfigurationSchema,\n", + " 'pyaml.validation.validation_models.DynamicValidation': pyaml.validation.validation_models.DynamicValidationConfigurationSchema,\n", + " 'typing.Any': typing.AnyConfigurationSchema,\n", + ")\n" + ] + } + ], + "source": [ + "from pyaml.validation import SchemaRegistry\n", + "\n", + "registry = SchemaRegistry()\n", + "\n", + "# Clear the registry to remove the schemas that were manually registered\n", + "registry.clear()\n", + "\n", + "# Automatically discover and register all schemas\n", + "registry.discover()\n", + "\n", + "# Print the content of the registry\n", + "print(registry)" + ] + }, + { + "cell_type": "markdown", + "id": "fcf398c6", + "metadata": {}, + "source": [ + "## Browse the Registry\n", + "\n", + "The registry also supports common mapping operations. Class paths are the keys and registered schema classes are the values. See the API documentation for all available methods." + ] + }, + { + "cell_type": "markdown", + "id": "2b6bc8fe", + "metadata": {}, + "source": [ + "Use `get()` to retrieve the schema for a class path. It returns `None` when the path is not registered." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "69a98196", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "QuadrupoleConfigurationSchema(\n", + " class_path: str — Fully qualified class path.\n", + " name: str\n", + " model: pyaml.magnet.model.MagnetModelConfigurationSchema | None\n", + " lattice_names: str | None\n", + " description: str | None\n", + ")\n" + ] + } + ], + "source": [ + "schema = registry.get(\"pyaml.magnet.quadrupole.Quadrupole\")\n", + "print(schema.describe())" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "browse-registry-operations-code", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "True\n" + ] + } + ], + "source": [ + "class_path = \"pyaml.magnet.quadrupole.Quadrupole\"\n", + "\n", + "# Check if the class is in the registry\n", + "print(class_path in registry)" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "c8e02805", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "Registered class paths:\n", + "- pyaml.bpm.bpm.BPM\n", + "- pyaml.common.element.Element\n", + "- pyaml.validation.validation_models.DynamicValidation\n", + "- pyaml.magnet.model.MagnetModel\n", + "- pyaml.magnet.hcorrector.HCorrector\n", + "- pyaml.magnet.magnet.Magnet\n", + "- pyaml.magnet.octupole.Octupole\n", + "- pyaml.magnet.quadrupole.Quadrupole\n", + "- pyaml.magnet.sextupole.Sextupole\n", + "- pyaml.magnet.skewoctu.SkewOctu\n", + "- pyaml.magnet.skewquad.SkewQuad\n", + "- pyaml.magnet.skewsext.SkewSext\n", + "- pyaml.magnet.vcorrector.VCorrector\n", + "- typing.Any\n", + "- pyaml.magnet.cfm_magnet.CombinedFunctionMagnet\n", + "- pyaml.magnet.serialized_magnet.SerializedMagnets\n", + "- pyaml.diagnostics.tune_monitor.BetatronTuneMonitor\n", + "- pyaml.diagnostics.atune_monitor.ABetatronTuneMonitor\n", + "- pyaml.rf.rf_transmitter.RFTransmitter\n", + "- pyaml.rf.rf_plant.RFPlant\n", + "- pyaml.tuning_tools.chromaticity_monitor.ChromaticityMonitor\n", + "- pyaml.tuning_tools.measurement_tool.MeasurementTool\n", + "- pyaml.arrays.array.ArrayConfig\n", + "- pyaml.lattice.lattice_elements_linker.LinkerConfigModel\n", + "- abc.ABC\n", + "- pyaml.lattice.lattice_elements_linker.LatticeElementsLinker\n", + "- pyaml.lattice.simulator.Simulator\n", + "- pyaml.common.holders.element_holder.ElementHolder\n", + "- pyaml.arrays.bpm.BPM\n", + "- pyaml.arrays.cfm_magnet.CombinedFunctionMagnet\n", + "- pyaml.arrays.element.Element\n", + "- pyaml.arrays.magnet.Magnet\n", + "- pyaml.arrays.serialized_magnet.SerializedMagnets\n", + "- pyaml.lattice.attribute_linker.PyAtAttributeElementsLinker\n", + "- pyaml.magnet.csvcurve.CSVCurve\n", + "- pyaml.magnet.curve.Curve\n", + "- pyaml.magnet.csvmatrix.CSVMatrix\n", + "- pyaml.magnet.matrix.Matrix\n", + "- pyaml.magnet.identity_cfm_model.IdentityCFMagnetModel\n", + "- pyaml.magnet.identity_model.IdentityMagnetModel\n", + "- pyaml.magnet.inline_curve.InlineCurve\n", + "- pyaml.magnet.inline_matrix.InlineMatrix\n", + "- pyaml.magnet.linear_cfm_model.LinearCFMagnetModel\n", + "- pyaml.magnet.linear_model.LinearMagnetModel\n", + "- pyaml.magnet.linear_serialized_model.LinearSerializedMagnetModel\n", + "- pyaml.magnet.spline_model.SplineMagnetModel\n", + "- pyaml.tuning_tools.bba.BBA\n", + "- pyaml.tuning_tools.bba2.BBA2\n", + "- pyaml.tuning_tools.response_matrix_data.ResponseMatrixData\n", + "- pyaml.tuning_tools.chromaticity.Chromaticity\n", + "- pyaml.tuning_tools.tuning_tool.TuningTool\n", + "- pyaml.tuning_tools.chromaticity_response_matrix.ChromaticityResponseMatrix\n", + "- pyaml.tuning_tools.dispersion.Dispersion\n", + "- pyaml.tuning_tools.orbit_response_matrix_data.OrbitResponseMatrixData\n", + "- pyaml.tuning_tools.orbit.Orbit\n", + "- pyaml.tuning_tools.orbit_response_matrix.OrbitResponseMatrix\n", + "- pyaml.tuning_tools.tune.Tune\n", + "- pyaml.tuning_tools.tune_response_matrix.TuneResponseMatrix\n", + "\n", + " Registered schema classes:\n", + "- BPMConfigurationSchema\n", + "- ElementConfigurationSchema\n", + "- DynamicValidationConfigurationSchema\n", + "- MagnetModelConfigurationSchema\n", + "- HCorrectorConfigurationSchema\n", + "- MagnetConfigurationSchema\n", + "- OctupoleConfigurationSchema\n", + "- QuadrupoleConfigurationSchema\n", + "- SextupoleConfigurationSchema\n", + "- SkewOctuConfigurationSchema\n", + "- SkewQuadConfigurationSchema\n", + "- SkewSextConfigurationSchema\n", + "- VCorrectorConfigurationSchema\n", + "- AnyConfigurationSchema\n", + "- CombinedFunctionMagnetConfigurationSchema\n", + "- SerializedMagnetsConfigurationSchema\n", + "- BetatronTuneMonitorConfigurationSchema\n", + "- ABetatronTuneMonitorConfigurationSchema\n", + "- RFTransmitterConfigurationSchema\n", + "- RFPlantConfigurationSchema\n", + "- ChromaticityMonitorConfigurationSchema\n", + "- MeasurementToolConfigurationSchema\n", + "- ArrayConfigConfigurationSchema\n", + "- LinkerConfigModelConfigurationSchema\n", + "- ABCConfigurationSchema\n", + "- LatticeElementsLinkerConfigurationSchema\n", + "- SimulatorConfigurationSchema\n", + "- ElementHolderConfigurationSchema\n", + "- BPMConfigurationSchema\n", + "- CombinedFunctionMagnetConfigurationSchema\n", + "- ElementConfigurationSchema\n", + "- MagnetConfigurationSchema\n", + "- SerializedMagnetsConfigurationSchema\n", + "- PyAtAttributeElementsLinkerConfigurationSchema\n", + "- CSVCurveConfigurationSchema\n", + "- CurveConfigurationSchema\n", + "- CSVMatrixConfigurationSchema\n", + "- MatrixConfigurationSchema\n", + "- IdentityCFMagnetModelConfigurationSchema\n", + "- IdentityMagnetModelConfigurationSchema\n", + "- InlineCurveConfigurationSchema\n", + "- InlineMatrixConfigurationSchema\n", + "- LinearCFMagnetModelConfigurationSchema\n", + "- LinearMagnetModelConfigurationSchema\n", + "- LinearSerializedMagnetModelConfigurationSchema\n", + "- SplineMagnetModelConfigurationSchema\n", + "- BBAConfigurationSchema\n", + "- BBA2ConfigurationSchema\n", + "- ResponseMatrixDataConfigurationSchema\n", + "- ChromaticityConfigurationSchema\n", + "- TuningToolConfigurationSchema\n", + "- ChromaticityResponseMatrixConfigurationSchema\n", + "- DispersionConfigurationSchema\n", + "- OrbitResponseMatrixDataConfigurationSchema\n", + "- OrbitConfigurationSchema\n", + "- OrbitResponseMatrixConfigurationSchema\n", + "- TuneConfigurationSchema\n", + "- TuneResponseMatrixConfigurationSchema\n" + ] + } + ], + "source": [ + "# Iterate through the registered schemas\n", + "\n", + "print(\"\\nRegistered class paths:\")\n", + "for path in registry:\n", + " print(f\"- {path}\")\n", + "\n", + "print(\"\\n Registered schema classes:\")\n", + "for schema in registry.values():\n", + " print(f\"- {schema.__name__}\") " + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "9e4d7e7a", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "Number of registered schemas: 58\n" + ] + } + ], + "source": [ + "# Print the number of schemas in the registry\n", + "print(f\"\\nNumber of registered schemas: {len(registry)}\")" + ] + }, + { + "cell_type": "markdown", + "id": "f693c376", + "metadata": {}, + "source": [ + "## Validate Configuration\n", + "\n", + "Configuration data can be validated using the `SchemaValidator`. It makes use of the schema registry to extract which schema to validate against for a specific class.\n", + "\n", + "For validation to be possible the class must be registered in the schema registry. If the class is not registered, validation will be skipped, a warning given and the data kept unchanged. Beware that this can lead to unexpected errors." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "73114262", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "source": [ + "model_path = \"pyaml.magnet.identity_model.IdentityMagnetModel\"\n", + "print(registry.get(model_path))" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "b2523c34", + "metadata": {}, + "outputs": [], + "source": [ + "from pyaml.validation import SchemaRegistry, SchemaValidator\n", + "\n", + "registry = SchemaRegistry()\n", + "registry.discover()\n", + "\n", + "configuration = {\n", + " \"class_path\": \"pyaml.magnet.quadrupole.Quadrupole\",\n", + " \"name\": \"QF1\",\n", + " \"description\": \"This is the QF1 quadrupole magnet.\"\n", + "}\n", + "validated = SchemaValidator.validate(configuration)" + ] + }, + { + "cell_type": "markdown", + "id": "365329b2", + "metadata": {}, + "source": [ + "Validation also handles nested configuration data." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "2e5df599", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "class_path='pyaml.magnet.quadrupole.Quadrupole' name='QF1' model=IdentityMagnetModelConfigurationSchema(class_path='pyaml.magnet.identity_model.IdentityMagnetModel', powerconverter=None, physics='', unit='1/m') lattice_names=None description='This is the QF1 quadrupole magnet.'\n" + ] + } + ], + "source": [ + "configuration = {\n", + " \"class_path\": \"pyaml.magnet.quadrupole.Quadrupole\",\n", + " \"name\": \"QF1\",\n", + " \"model\": {\n", + " \"class_path\": \"pyaml.magnet.identity_model.IdentityMagnetModel\",\n", + " \"unit\": \"1/m\",\n", + " \"physics\": \"\"\n", + " },\n", + " \"description\": \"This is the QF1 quadrupole magnet.\"\n", + "}\n", + "\n", + "validated = SchemaValidator.validate(configuration)\n", + "print(validated)" + ] + }, + { + "cell_type": "markdown", + "id": "aa814612", + "metadata": {}, + "source": [ + "The validated result can also be returned as a dictionary." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "3308aba1", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'class_path': 'pyaml.magnet.quadrupole.Quadrupole',\n", + " 'description': 'This is the QF1 quadrupole magnet.',\n", + " 'lattice_names': None,\n", + " 'model': {'class_path': 'pyaml.magnet.identity_model.IdentityMagnetModel',\n", + " 'physics': '',\n", + " 'powerconverter': None,\n", + " 'unit': '1/m'},\n", + " 'name': 'QF1'}\n" + ] + } + ], + "source": [ + "from pprint import pprint\n", + "\n", + "validated_dict = SchemaValidator.validate_to_dict(configuration)\n", + "pprint(validated_dict )" + ] + }, + { + "cell_type": "markdown", + "id": "2d03ad15", + "metadata": {}, + "source": [ + "## Generate JSON Schema\n", + "\n", + "The registry can also be used together with the `SchemaGenerator` to generate JSON Schema to use with external tools.\n", + "\n", + "If a base schema has registered concrete or virtual subclasses, the generated JSON Schema includes those alternatives. This allows editors and other JSON Schema tools to offer the appropriate fields for each configuration type." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "dfb96bb7", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.quadrupole.Quadrupole',\n", + " 'description': 'Fully qualified class path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'description': {'default': None,\n", + " 'title': 'Description',\n", + " 'type': ['string', 'null']},\n", + " 'lattice_names': {'default': None,\n", + " 'title': 'Lattice Names',\n", + " 'type': ['string', 'null']},\n", + " 'model': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.identity_cfm_model.IdentityCFMagnetModel',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'multipoles': {'items': {'type': 'string'},\n", + " 'title': 'Multipoles',\n", + " 'type': 'array'},\n", + " 'physics': {'anyOf': [{'items': {'type': ['string',\n", + " 'null']},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Physics'},\n", + " 'powerconverters': {'anyOf': [{'items': {'type': ['string',\n", + " 'null']},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Powerconverters'},\n", + " 'units': {'anyOf': [{'items': {'type': 'string'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Units'}},\n", + " 'required': ['class',\n", + " 'multipoles'],\n", + " 'title': 'IdentityCFMagnetModelConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.identity_model.IdentityMagnetModel',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'physics': {'default': None,\n", + " 'title': 'Physics',\n", + " 'type': ['string',\n", + " 'null']},\n", + " 'powerconverter': {'default': None,\n", + " 'title': 'Powerconverter',\n", + " 'type': ['string',\n", + " 'null']},\n", + " 'unit': {'default': None,\n", + " 'title': 'Unit',\n", + " 'type': ['string',\n", + " 'null']}},\n", + " 'required': ['class'],\n", + " 'title': 'IdentityMagnetModelConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'calibration_factors': {'anyOf': [{'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Calibration '\n", + " 'Factors'},\n", + " 'calibration_offsets': {'anyOf': [{'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Calibration '\n", + " 'Offsets'},\n", + " 'class': {'const': 'pyaml.magnet.linear_cfm_model.LinearCFMagnetModel',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'curves': {'items': {'anyOf': [{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'file': {'title': 'File',\n", + " 'type': 'string'}},\n", + " 'required': ['class',\n", + " 'file'],\n", + " 'title': 'CSVCurveConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'mat': {'items': {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " 'title': 'Mat',\n", + " 'type': 'array'}},\n", + " 'required': ['class',\n", + " 'mat'],\n", + " 'title': 'InlineCurveConfigurationSchema',\n", + " 'type': 'object'}],\n", + " 'title': 'CurveConfigurationSchema'},\n", + " 'title': 'Curves',\n", + " 'type': 'array'},\n", + " 'hardware_units': {'items': {'type': 'string'},\n", + " 'title': 'Hardware '\n", + " 'Units',\n", + " 'type': 'array'},\n", + " 'matrix': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.csvmatrix.CSVMatrix',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'file': {'title': 'File',\n", + " 'type': 'string'}},\n", + " 'required': ['class',\n", + " 'file'],\n", + " 'title': 'CSVMatrixConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.inline_matrix.InlineMatrix',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'mat': {'items': {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " 'title': 'Mat',\n", + " 'type': 'array'}},\n", + " 'required': ['class',\n", + " 'mat'],\n", + " 'title': 'InlineMatrixConfigurationSchema',\n", + " 'type': 'object'}],\n", + " 'title': 'MatrixConfigurationSchema'},\n", + " {'type': 'null'}],\n", + " 'default': None},\n", + " 'multipoles': {'items': {'type': 'string'},\n", + " 'title': 'Multipoles',\n", + " 'type': 'array'},\n", + " 'powerconverters': {'items': {'type': ['string',\n", + " 'null']},\n", + " 'title': 'Powerconverters',\n", + " 'type': 'array'},\n", + " 'pseudo_factors': {'anyOf': [{'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Pseudo '\n", + " 'Factors'},\n", + " 'pseudo_offsets': {'anyOf': [{'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Pseudo '\n", + " 'Offsets'},\n", + " 'units': {'anyOf': [{'items': {'type': 'string'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Units'}},\n", + " 'required': ['class',\n", + " 'multipoles',\n", + " 'curves',\n", + " 'powerconverters',\n", + " 'hardware_units'],\n", + " 'title': 'LinearCFMagnetModelConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'calibration_factor': {'default': 1.0,\n", + " 'title': 'Calibration '\n", + " 'Factor',\n", + " 'type': 'number'},\n", + " 'calibration_offset': {'default': 0.0,\n", + " 'title': 'Calibration '\n", + " 'Offset',\n", + " 'type': 'number'},\n", + " 'class': {'const': 'pyaml.magnet.linear_model.LinearMagnetModel',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'crosstalk': {'default': 1.0,\n", + " 'title': 'Crosstalk',\n", + " 'type': 'number'},\n", + " 'curve': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'file': {'title': 'File',\n", + " 'type': 'string'}},\n", + " 'required': ['class',\n", + " 'file'],\n", + " 'title': 'CSVCurveConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'mat': {'items': {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " 'title': 'Mat',\n", + " 'type': 'array'}},\n", + " 'required': ['class',\n", + " 'mat'],\n", + " 'title': 'InlineCurveConfigurationSchema',\n", + " 'type': 'object'}],\n", + " 'title': 'CurveConfigurationSchema'},\n", + " {'type': 'null'}],\n", + " 'default': None},\n", + " 'hardware_unit': {'title': 'Hardware '\n", + " 'Unit',\n", + " 'type': 'string'},\n", + " 'powerconverter': {'default': None,\n", + " 'title': 'Powerconverter',\n", + " 'type': ['string',\n", + " 'null']},\n", + " 'unit': {'title': 'Unit',\n", + " 'type': 'string'}},\n", + " 'required': ['class',\n", + " 'unit',\n", + " 'hardware_unit'],\n", + " 'title': 'LinearMagnetModelConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'calibration_factors': {'anyOf': [{'type': 'number'},\n", + " {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Calibration '\n", + " 'Factors'},\n", + " 'calibration_offsets': {'anyOf': [{'type': 'number'},\n", + " {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Calibration '\n", + " 'Offsets'},\n", + " 'class': {'const': 'pyaml.magnet.linear_serialized_model.LinearSerializedMagnetModel',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'crosstalk': {'anyOf': [{'type': 'number'},\n", + " {'items': {'type': 'number'},\n", + " 'type': 'array'}],\n", + " 'default': 1.0,\n", + " 'title': 'Crosstalk'},\n", + " 'curves': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'file': {'title': 'File',\n", + " 'type': 'string'}},\n", + " 'required': ['class',\n", + " 'file'],\n", + " 'title': 'CSVCurveConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'mat': {'items': {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " 'title': 'Mat',\n", + " 'type': 'array'}},\n", + " 'required': ['class',\n", + " 'mat'],\n", + " 'title': 'InlineCurveConfigurationSchema',\n", + " 'type': 'object'}],\n", + " 'title': 'CurveConfigurationSchema'},\n", + " {'items': {'anyOf': [{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'file': {'title': 'File',\n", + " 'type': 'string'}},\n", + " 'required': ['class',\n", + " 'file'],\n", + " 'title': 'CSVCurveConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'mat': {'items': {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " 'title': 'Mat',\n", + " 'type': 'array'}},\n", + " 'required': ['class',\n", + " 'mat'],\n", + " 'title': 'InlineCurveConfigurationSchema',\n", + " 'type': 'object'}],\n", + " 'title': 'CurveConfigurationSchema'},\n", + " 'type': 'array'}],\n", + " 'title': 'Curves'},\n", + " 'hardware_unit': {'default': None,\n", + " 'title': 'Hardware '\n", + " 'Unit',\n", + " 'type': ['string',\n", + " 'null']},\n", + " 'powerconverter': {'default': None,\n", + " 'title': 'Powerconverter',\n", + " 'type': ['string',\n", + " 'null']},\n", + " 'unit': {'default': None,\n", + " 'title': 'Unit',\n", + " 'type': ['string',\n", + " 'null']}},\n", + " 'required': ['class', 'curves'],\n", + " 'title': 'LinearSerializedMagnetModelConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'alpha': {'default': 0.0,\n", + " 'title': 'Alpha',\n", + " 'type': 'number'},\n", + " 'calibration_factor': {'default': 1.0,\n", + " 'title': 'Calibration '\n", + " 'Factor',\n", + " 'type': 'number'},\n", + " 'calibration_offset': {'default': 0.0,\n", + " 'title': 'Calibration '\n", + " 'Offset',\n", + " 'type': 'number'},\n", + " 'class': {'const': 'pyaml.magnet.spline_model.SplineMagnetModel',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'crosstalk': {'default': 1.0,\n", + " 'title': 'Crosstalk',\n", + " 'type': 'number'},\n", + " 'curve': {'anyOf': [{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'file': {'title': 'File',\n", + " 'type': 'string'}},\n", + " 'required': ['class',\n", + " 'file'],\n", + " 'title': 'CSVCurveConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'mat': {'items': {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " 'title': 'Mat',\n", + " 'type': 'array'}},\n", + " 'required': ['class',\n", + " 'mat'],\n", + " 'title': 'InlineCurveConfigurationSchema',\n", + " 'type': 'object'}],\n", + " 'title': 'CurveConfigurationSchema'},\n", + " 'hardware_unit': {'default': None,\n", + " 'title': 'Hardware '\n", + " 'Unit',\n", + " 'type': ['string',\n", + " 'null']},\n", + " 'powerconverter': {'default': None,\n", + " 'title': 'Powerconverter',\n", + " 'type': ['string',\n", + " 'null']},\n", + " 'unit': {'default': None,\n", + " 'title': 'Unit',\n", + " 'type': ['string',\n", + " 'null']}},\n", + " 'required': ['class', 'curve'],\n", + " 'title': 'SplineMagnetModelConfigurationSchema',\n", + " 'type': 'object'}],\n", + " 'title': 'MagnetModelConfigurationSchema'},\n", + " {'type': 'null'}],\n", + " 'default': None},\n", + " 'name': {'title': 'Name', 'type': 'string'}},\n", + " 'required': ['class', 'name'],\n", + " 'title': 'QuadrupoleConfigurationSchema',\n", + " 'type': 'object'}\n" + ] + } + ], + "source": [ + "from pprint import pprint\n", + "\n", + "from pyaml.validation import SchemaGenerator\n", + "\n", + "json_schema = SchemaGenerator.generate(\"pyaml.magnet.quadrupole.Quadrupole\")\n", + "pprint(json_schema)" + ] + }, + { + "cell_type": "markdown", + "id": "d329d7e5", + "metadata": {}, + "source": [ + "The result can also be saved directly to a file." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "12610af9", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "PosixPath('quadrupole-schema.json')" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "SchemaGenerator.save(\"pyaml.magnet.quadrupole.Quadrupole\",\"quadrupole-schema.json\")" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "pyaml-documentation", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.16" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/how-to/index.md b/docs/source/how-to/index.md index 9de2541..3226780 100644 --- a/docs/source/how-to/index.md +++ b/docs/source/how-to/index.md @@ -22,6 +22,7 @@ installation/developer-installation :caption: Configuration configuration/create-configuration +configuration/use-schema-registry ``` @@ -32,12 +33,6 @@ configuration/create-configuration virtual-accelerator/apptainer ``` -```{toctree} -:maxdepth: 1 -:caption: Validation - -validation/use-schema-registry -``` ```{toctree} :maxdepth: 1