diff --git a/docs/requirements.txt b/docs/requirements.txt new file mode 100644 index 0000000..1b92caa --- /dev/null +++ b/docs/requirements.txt @@ -0,0 +1,4 @@ +pyaml-test-lattice +pyaml +pyaml-cs-oa[tango,epics] +tango-pyaml \ No newline at end of file diff --git a/docs/source/_static/metaconfigurator_frontpage.png b/docs/source/_static/metaconfigurator_frontpage.png new file mode 100644 index 0000000..2fb2008 Binary files /dev/null and b/docs/source/_static/metaconfigurator_frontpage.png differ diff --git a/docs/source/_static/metaconfigurator_gui_view.png b/docs/source/_static/metaconfigurator_gui_view.png new file mode 100644 index 0000000..2b6f8fc Binary files /dev/null and b/docs/source/_static/metaconfigurator_gui_view.png differ diff --git a/docs/source/_static/vscode-json-schema-json.png b/docs/source/_static/vscode-json-schema-json.png new file mode 100644 index 0000000..3a0815c Binary files /dev/null and b/docs/source/_static/vscode-json-schema-json.png differ diff --git a/docs/source/_static/vscode-json-schema-yaml.png b/docs/source/_static/vscode-json-schema-yaml.png new file mode 100644 index 0000000..a5e9bad Binary files /dev/null and b/docs/source/_static/vscode-json-schema-yaml.png differ diff --git a/docs/source/explanation/configuration.md b/docs/source/explanation/configuration.md new file mode 100644 index 0000000..839e256 --- /dev/null +++ b/docs/source/explanation/configuration.md @@ -0,0 +1,90 @@ +# Principles and Syntax of the Configuration + +By creating a configuration it is possible to have pyAML build devices and applications automatically for several control modes. + +The configuration is normally written and loaded on the level of an `Accelerator`. This allows to define parameters and metadata which are common for all objects in the accelerator in addition to control modes, arrays and devices. For example: + +```yaml +class: pyaml.accelerator.Accelerator +facility: pyAML_facility +machine: storage_ring +data_folder: '' +energy: 1e6 +controls: +simulators: +arrays: +devices: +``` + +The configuration is organised as a description of a nested Python object tree. It contains the information needed to construct the objects. + +The syntax has been chosen to allow configuration and construction of objects for both pyAML classes and third party classes. This is to allow integration of facility specific implementation in pyAML, but also to simplify future development where it might be desirable to replace old classes with newer versions without breaking compatibility. + +## Configuration Items + +Each configurable item is represented by a mapping which describes the attributes and values needed to construct one Python object. The field `class` or `class_path` identifies the type to construct. It should be written as a fully qualified Python class path, consisting of the module and class name. For example: + +```yaml +class: pyaml.magnet.quadrupole.Quadrupole +``` + +When pyAML reads the configuration, it uses this path to select the class and passes the remaining configuration fields to the constructor of the class. + +An object can contain other configurable objects. The nested objects follow the same principle: each has its own `class` field and the values needed to construct it. For example: + +```yaml +class: pyaml.magnet.quadrupole.Quadrupole +name: QF_001 +model: + class: pyaml.magnet.identity_model.IdentityMagnetModel + unit: 1/m + physics: AN01-AR/EM-QP/QF.01/magnetic_strength +``` + +## Separation between Configuration and Source Code + +The configuration describes what should be constructed; it does not contain executable Python code. This keeps configuration readable, reviewable, and usable by tools such as JSON Schema editors. + +The configuration is possible to maintain separately to pyAML, for example in a separate Git repository or in a database. + +## Supported Formats + +The configuration can be written and loaded in different formats: + +**File**: It can be written as a text file and loaded using `Accelerator.load()`. Both `YAML` and `JSON` are supported but `YAML` is considered the default option. + +**Dictionary**: It can also be written as a nested dictionary and loaded using `Accelerator.from_dict()`. + +## Configuration Root + +The configuration root is the directory used to resolve relative configuration paths. It applies to the file passed to `Accelerator.load()`, paths used by resolvers, and automatic file includes. Relative paths are resolved against this directory. + +By default, the root is the current working directory when pyAML is imported. It can be changed before loading a configuration with `ROOT.set()`: + +```python +from pyaml.configuration import ROOT + +ROOT.set("/path/to/configuration") +``` + +After setting the root, `devices/quadrupole.yaml` refers to `/path/to/configuration/devices/quadrupole.yaml`. Absolute paths are normalized and used directly. Setting the root makes it possible to keep a configuration and its included files in a portable directory tree while selecting that tree at runtime. + +## Resolvers + +Resolvers let a configuration value refer to information that is supplied when the configuration is loaded, such as an environment variable or another configuration file. A resolver expression has the following form: + +```yaml +${resolver:payload} +``` + +The part before the first colon selects the resolver whereas the part after is passed to that resolver. Expressions are expanded recursively in mappings and lists. + +The following built-in resolvers are available: + +| Resolver | Type | Description | +| --- | --- | --- | +| `env` | Environment variable | The value of the named environment variable. An error is raised if it is not set. | +| `path` | A file or directory path | The absolute, normalized path, resolved relative to pyAML's configuration root. The target is not loaded as part of loading the configuration. | +| `file` | A YAML, YML, or JSON file path | The path to a file which should be loaded and expanded into the configuration as part of loading the configuration. Relative paths use the configuration root. + +Configuration files can also be included without an explicit `file` expression. A string ending in `.yaml`, `.yml`, or `.json` is loaded automatically for convenience if one wishes to split the configuration into several files. diff --git a/docs/source/explanation/index.md b/docs/source/explanation/index.md index e46e5e7..ed5850c 100644 --- a/docs/source/explanation/index.md +++ b/docs/source/explanation/index.md @@ -5,7 +5,8 @@ Here you can find explanations of the concepts, design decisions, and underlying ```{toctree} :maxdepth: 1 -schema_and_validation +configuration +schema_and_validation catalog ``` diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 8ec5a1d..00de849 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -1,27 +1,24 @@ -# Create Configuration +# Create and Load Configuration -By creating a configuration it is possible to have pyAML create devices and applications automatically for several control modes. +The principles and syntax of the configuration are explained in more detail in [Principles and Syntax of the Configuration](../../explanation/configuration). This guide focuses on the different ways to create it. -There are different ways to create a configuration and different formats are supported. -These are explained in this guide. +There are several ways to create a configuration. It is recommended to test the different options and see which one you prefer: -## Principle +- [Use ConfigurationSchema](./use-configuration-schema.ipynb) objects and export as a dictionary or text file -The configuration is done on the level of an `Accelerator`. This allows not only to create devices and applications for different control modes but also to define parameters and metadata which are common for the accelerator. +- Use a JSON Schema in the [MetaConfigurator](./use-meta-configurator.md) -The syntax supports configuration of both pyAML classes and third party classes to allow the use of pyAML implementations as well as facility specific implementation in the same accelerator. This is done by for each item in the configuration define the field `class` or `class_path` to say which class to build an object of. +- Use a [JSON Schema in VS Code](./use-vscode-json-schema.md) -## Format Options +## Load the Configuration -A configuration can be created and loaded using different formats: +The configuration can be loaded into the `Accelerator` in several ways: -1. File +| Type| Command | Description | +| --- | --- | --- | +| File | `Accelerator.load()` | A text file in JSON on YAML format. +| Dictionary | `Accelerator.from_dict()` | A nested dictionary. - The configuration can be written as a text file and loaded using `Accelerator.load()`. Both `YAML` and `JSON` are supported but `YAML` is considered the default option. +## Validation -2. Dictionary - - The configuration can be written as a nested dictionary and loaded using `Accelerator.from_dict()`. - - -To be continued with details about the different tools to help write it... +The configuration is validated when loading it into the `Accelerator` but it can also be validated without having to load it. This is useful if you want to be able to maintain it separately from pyAML. See [Validate Configuration](./validate-configuration) for details. \ No newline at end of file diff --git a/docs/source/how-to/configuration/generate-json-schema.ipynb b/docs/source/how-to/configuration/generate-json-schema.ipynb new file mode 100644 index 0000000..3e74178 --- /dev/null +++ b/docs/source/how-to/configuration/generate-json-schema.ipynb @@ -0,0 +1,559 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "74c407c2", + "metadata": {}, + "source": [ + "# Generate JSON Schemas\n", + "\n", + "This guide shows how to use the `SchemaRegistry` to generate JSON Schemas." + ] + }, + { + "cell_type": "markdown", + "id": "0e66489e", + "metadata": {}, + "source": [ + "## Create the Registry\n", + "\n", + "First create the registry and register the classes you want included in the schema.\n", + "\n", + "Here the discovery function is used to register all classes in `pyaml` and other packages which define entry points." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "3ee0074c", + "metadata": {}, + "outputs": [], + "source": [ + "from pyaml.validation import SchemaRegistry\n", + "\n", + "registry = SchemaRegistry()\n", + "registry.discover()" + ] + }, + { + "cell_type": "markdown", + "id": "2d03ad15", + "metadata": {}, + "source": [ + "## Generate JSON Schema\n", + "\n", + "Use the `SchemaGenerator` to generate a JSON Schema to use with external tools.\n", + "\n", + "If a base schema has registered concrete or virtual subclasses, the generated JSON Schema includes those alternatives. This allows editors and other JSON Schema tools to offer the appropriate fields for each configuration type." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "dfb96bb7", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.quadrupole.Quadrupole',\n", + " 'description': 'Fully qualified class path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'description': {'default': None,\n", + " 'title': 'Description',\n", + " 'type': ['string', 'null']},\n", + " 'lattice_names': {'default': None,\n", + " 'title': 'Lattice Names',\n", + " 'type': ['string', 'null']},\n", + " 'model': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.identity_cfm_model.IdentityCFMagnetModel',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'multipoles': {'items': {'type': 'string'},\n", + " 'title': 'Multipoles',\n", + " 'type': 'array'},\n", + " 'physics': {'anyOf': [{'items': {'type': ['string',\n", + " 'null']},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Physics'},\n", + " 'powerconverters': {'anyOf': [{'items': {'type': ['string',\n", + " 'null']},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Powerconverters'},\n", + " 'units': {'anyOf': [{'items': {'type': 'string'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Units'}},\n", + " 'required': ['class',\n", + " 'multipoles'],\n", + " 'title': 'IdentityCFMagnetModelConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.identity_model.IdentityMagnetModel',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'physics': {'default': None,\n", + " 'title': 'Physics',\n", + " 'type': ['string',\n", + " 'null']},\n", + " 'powerconverter': {'default': None,\n", + " 'title': 'Powerconverter',\n", + " 'type': ['string',\n", + " 'null']},\n", + " 'unit': {'default': None,\n", + " 'title': 'Unit',\n", + " 'type': ['string',\n", + " 'null']}},\n", + " 'required': ['class'],\n", + " 'title': 'IdentityMagnetModelConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'calibration_factors': {'anyOf': [{'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Calibration '\n", + " 'Factors'},\n", + " 'calibration_offsets': {'anyOf': [{'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Calibration '\n", + " 'Offsets'},\n", + " 'class': {'const': 'pyaml.magnet.linear_cfm_model.LinearCFMagnetModel',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'curves': {'items': {'anyOf': [{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'file': {'title': 'File',\n", + " 'type': 'string'}},\n", + " 'required': ['class',\n", + " 'file'],\n", + " 'title': 'CSVCurveConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'mat': {'items': {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " 'title': 'Mat',\n", + " 'type': 'array'}},\n", + " 'required': ['class',\n", + " 'mat'],\n", + " 'title': 'InlineCurveConfigurationSchema',\n", + " 'type': 'object'}],\n", + " 'title': 'CurveConfigurationSchema'},\n", + " 'title': 'Curves',\n", + " 'type': 'array'},\n", + " 'hardware_units': {'items': {'type': 'string'},\n", + " 'title': 'Hardware '\n", + " 'Units',\n", + " 'type': 'array'},\n", + " 'matrix': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.csvmatrix.CSVMatrix',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'file': {'title': 'File',\n", + " 'type': 'string'}},\n", + " 'required': ['class',\n", + " 'file'],\n", + " 'title': 'CSVMatrixConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.inline_matrix.InlineMatrix',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'mat': {'items': {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " 'title': 'Mat',\n", + " 'type': 'array'}},\n", + " 'required': ['class',\n", + " 'mat'],\n", + " 'title': 'InlineMatrixConfigurationSchema',\n", + " 'type': 'object'}],\n", + " 'title': 'MatrixConfigurationSchema'},\n", + " {'type': 'null'}],\n", + " 'default': None},\n", + " 'multipoles': {'items': {'type': 'string'},\n", + " 'title': 'Multipoles',\n", + " 'type': 'array'},\n", + " 'powerconverters': {'items': {'type': ['string',\n", + " 'null']},\n", + " 'title': 'Powerconverters',\n", + " 'type': 'array'},\n", + " 'pseudo_factors': {'anyOf': [{'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Pseudo '\n", + " 'Factors'},\n", + " 'pseudo_offsets': {'anyOf': [{'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Pseudo '\n", + " 'Offsets'},\n", + " 'units': {'anyOf': [{'items': {'type': 'string'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Units'}},\n", + " 'required': ['class',\n", + " 'multipoles',\n", + " 'curves',\n", + " 'powerconverters',\n", + " 'hardware_units'],\n", + " 'title': 'LinearCFMagnetModelConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'calibration_factor': {'default': 1.0,\n", + " 'title': 'Calibration '\n", + " 'Factor',\n", + " 'type': 'number'},\n", + " 'calibration_offset': {'default': 0.0,\n", + " 'title': 'Calibration '\n", + " 'Offset',\n", + " 'type': 'number'},\n", + " 'class': {'const': 'pyaml.magnet.linear_model.LinearMagnetModel',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'crosstalk': {'default': 1.0,\n", + " 'title': 'Crosstalk',\n", + " 'type': 'number'},\n", + " 'curve': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'file': {'title': 'File',\n", + " 'type': 'string'}},\n", + " 'required': ['class',\n", + " 'file'],\n", + " 'title': 'CSVCurveConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'mat': {'items': {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " 'title': 'Mat',\n", + " 'type': 'array'}},\n", + " 'required': ['class',\n", + " 'mat'],\n", + " 'title': 'InlineCurveConfigurationSchema',\n", + " 'type': 'object'}],\n", + " 'title': 'CurveConfigurationSchema'},\n", + " {'type': 'null'}],\n", + " 'default': None},\n", + " 'hardware_unit': {'title': 'Hardware '\n", + " 'Unit',\n", + " 'type': 'string'},\n", + " 'powerconverter': {'default': None,\n", + " 'title': 'Powerconverter',\n", + " 'type': ['string',\n", + " 'null']},\n", + " 'unit': {'title': 'Unit',\n", + " 'type': 'string'}},\n", + " 'required': ['class',\n", + " 'unit',\n", + " 'hardware_unit'],\n", + " 'title': 'LinearMagnetModelConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'calibration_factors': {'anyOf': [{'type': 'number'},\n", + " {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Calibration '\n", + " 'Factors'},\n", + " 'calibration_offsets': {'anyOf': [{'type': 'number'},\n", + " {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " {'type': 'null'}],\n", + " 'default': None,\n", + " 'title': 'Calibration '\n", + " 'Offsets'},\n", + " 'class': {'const': 'pyaml.magnet.linear_serialized_model.LinearSerializedMagnetModel',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'crosstalk': {'anyOf': [{'type': 'number'},\n", + " {'items': {'type': 'number'},\n", + " 'type': 'array'}],\n", + " 'default': 1.0,\n", + " 'title': 'Crosstalk'},\n", + " 'curves': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'file': {'title': 'File',\n", + " 'type': 'string'}},\n", + " 'required': ['class',\n", + " 'file'],\n", + " 'title': 'CSVCurveConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'mat': {'items': {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " 'title': 'Mat',\n", + " 'type': 'array'}},\n", + " 'required': ['class',\n", + " 'mat'],\n", + " 'title': 'InlineCurveConfigurationSchema',\n", + " 'type': 'object'}],\n", + " 'title': 'CurveConfigurationSchema'},\n", + " {'items': {'anyOf': [{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'file': {'title': 'File',\n", + " 'type': 'string'}},\n", + " 'required': ['class',\n", + " 'file'],\n", + " 'title': 'CSVCurveConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'mat': {'items': {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " 'title': 'Mat',\n", + " 'type': 'array'}},\n", + " 'required': ['class',\n", + " 'mat'],\n", + " 'title': 'InlineCurveConfigurationSchema',\n", + " 'type': 'object'}],\n", + " 'title': 'CurveConfigurationSchema'},\n", + " 'type': 'array'}],\n", + " 'title': 'Curves'},\n", + " 'hardware_unit': {'default': None,\n", + " 'title': 'Hardware '\n", + " 'Unit',\n", + " 'type': ['string',\n", + " 'null']},\n", + " 'powerconverter': {'default': None,\n", + " 'title': 'Powerconverter',\n", + " 'type': ['string',\n", + " 'null']},\n", + " 'unit': {'default': None,\n", + " 'title': 'Unit',\n", + " 'type': ['string',\n", + " 'null']}},\n", + " 'required': ['class', 'curves'],\n", + " 'title': 'LinearSerializedMagnetModelConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'alpha': {'default': 0.0,\n", + " 'title': 'Alpha',\n", + " 'type': 'number'},\n", + " 'calibration_factor': {'default': 1.0,\n", + " 'title': 'Calibration '\n", + " 'Factor',\n", + " 'type': 'number'},\n", + " 'calibration_offset': {'default': 0.0,\n", + " 'title': 'Calibration '\n", + " 'Offset',\n", + " 'type': 'number'},\n", + " 'class': {'const': 'pyaml.magnet.spline_model.SplineMagnetModel',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'crosstalk': {'default': 1.0,\n", + " 'title': 'Crosstalk',\n", + " 'type': 'number'},\n", + " 'curve': {'anyOf': [{'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'file': {'title': 'File',\n", + " 'type': 'string'}},\n", + " 'required': ['class',\n", + " 'file'],\n", + " 'title': 'CSVCurveConfigurationSchema',\n", + " 'type': 'object'},\n", + " {'additionalProperties': False,\n", + " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", + " 'description': 'Fully '\n", + " 'qualified '\n", + " 'class '\n", + " 'path.',\n", + " 'title': 'Class',\n", + " 'type': 'string'},\n", + " 'mat': {'items': {'items': {'type': 'number'},\n", + " 'type': 'array'},\n", + " 'title': 'Mat',\n", + " 'type': 'array'}},\n", + " 'required': ['class',\n", + " 'mat'],\n", + " 'title': 'InlineCurveConfigurationSchema',\n", + " 'type': 'object'}],\n", + " 'title': 'CurveConfigurationSchema'},\n", + " 'hardware_unit': {'default': None,\n", + " 'title': 'Hardware '\n", + " 'Unit',\n", + " 'type': ['string',\n", + " 'null']},\n", + " 'powerconverter': {'default': None,\n", + " 'title': 'Powerconverter',\n", + " 'type': ['string',\n", + " 'null']},\n", + " 'unit': {'default': None,\n", + " 'title': 'Unit',\n", + " 'type': ['string',\n", + " 'null']}},\n", + " 'required': ['class', 'curve'],\n", + " 'title': 'SplineMagnetModelConfigurationSchema',\n", + " 'type': 'object'}],\n", + " 'title': 'MagnetModelConfigurationSchema'},\n", + " {'type': 'null'}],\n", + " 'default': None},\n", + " 'name': {'title': 'Name', 'type': 'string'}},\n", + " 'required': ['class', 'name'],\n", + " 'title': 'QuadrupoleConfigurationSchema',\n", + " 'type': 'object'}\n" + ] + } + ], + "source": [ + "from pprint import pprint\n", + "\n", + "from pyaml.validation import SchemaGenerator\n", + "\n", + "json_schema = SchemaGenerator.generate(\"pyaml.magnet.quadrupole.Quadrupole\")\n", + "pprint(json_schema)" + ] + }, + { + "cell_type": "markdown", + "id": "d329d7e5", + "metadata": {}, + "source": [ + "The result can also be saved directly to a file." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "12610af9", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "PosixPath('quadrupole-schema.json')" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "SchemaGenerator.save(\"pyaml.magnet.quadrupole.Quadrupole\",\"quadrupole-schema.json\")" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "pyaml-documentation", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.16" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/how-to/configuration/use-configuration-schema.ipynb b/docs/source/how-to/configuration/use-configuration-schema.ipynb new file mode 100644 index 0000000..277730a --- /dev/null +++ b/docs/source/how-to/configuration/use-configuration-schema.ipynb @@ -0,0 +1,413 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "33dda6ae", + "metadata": {}, + "source": [ + "# Generate Configuration Using Configuration Schemas\n", + "\n", + "This guide shows how to use `ConfigurationSchema` to create a configuration." + ] + }, + { + "cell_type": "markdown", + "id": "2eaa8240", + "metadata": { + "vscode": { + "languageId": "plaintext" + } + }, + "source": [ + "## Discover Schemas\n", + "\n", + "First create the registry and discover available schemas for the configuration." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "e37b2c3b", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "SchemaRegistry(\n", + " 'pyaml.accelerator.Accelerator': pyaml.accelerator.AcceleratorConfigurationSchema,\n", + " 'pyaml.arrays.array.ArrayConfig': pyaml.arrays.array.ArrayConfigConfigurationSchema,\n", + " 'pyaml.arrays.bpm.BPM': pyaml.arrays.bpm.BPMConfigurationSchema,\n", + " 'pyaml.arrays.cfm_magnet.CombinedFunctionMagnet': pyaml.arrays.cfm_magnet.CombinedFunctionMagnetConfigurationSchema,\n", + " 'pyaml.arrays.element.Element': pyaml.arrays.element.ElementConfigurationSchema,\n", + " 'pyaml.arrays.magnet.Magnet': pyaml.arrays.magnet.MagnetConfigurationSchema,\n", + " 'pyaml.arrays.serialized_magnet.SerializedMagnets': pyaml.arrays.serialized_magnet.SerializedMagnetsConfigurationSchema,\n", + " 'pyaml.bpm.bpm.BPM': pyaml.bpm.bpm.BPMConfigurationSchema,\n", + " 'pyaml.common.element.Element': pyaml.common.element.ElementConfigurationSchema,\n", + " 'pyaml.common.holders.element_holder.ElementHolder': pyaml.common.holders.element_holder.ElementHolderConfigurationSchema,\n", + " 'pyaml.control.controlsystem.ControlSystem': pyaml.control.controlsystem.ControlSystemConfigurationSchema,\n", + " 'pyaml.control.deviceaccess.DeviceAccess': pyaml.control.deviceaccess.DeviceAccessConfigurationSchema,\n", + " 'pyaml.control.deviceaccesslist.DeviceAccessList': pyaml.control.deviceaccesslist.DeviceAccessListConfigurationSchema,\n", + " 'pyaml.diagnostics.atune_monitor.ABetatronTuneMonitor': pyaml.diagnostics.atune_monitor.ABetatronTuneMonitorConfigurationSchema,\n", + " 'pyaml.diagnostics.tune_monitor.BetatronTuneMonitor': pyaml.diagnostics.tune_monitor.BetatronTuneMonitorConfigurationSchema,\n", + " 'pyaml.lattice.attribute_linker.PyAtAttributeElementsLinker': pyaml.lattice.attribute_linker.PyAtAttributeElementsLinkerConfigurationSchema,\n", + " 'pyaml.lattice.lattice_elements_linker.LatticeElementsLinker': pyaml.lattice.lattice_elements_linker.LatticeElementsLinkerConfigurationSchema,\n", + " 'pyaml.lattice.lattice_elements_linker.LinkerConfigModel': pyaml.lattice.lattice_elements_linker.LinkerConfigModelConfigurationSchema,\n", + " 'pyaml.lattice.simulator.Simulator': pyaml.lattice.simulator.SimulatorConfigurationSchema,\n", + " 'pyaml.magnet.cfm_magnet.CombinedFunctionMagnet': pyaml.magnet.cfm_magnet.CombinedFunctionMagnetConfigurationSchema,\n", + " 'pyaml.magnet.csvcurve.CSVCurve': pyaml.magnet.csvcurve.CSVCurveConfigurationSchema,\n", + " 'pyaml.magnet.csvmatrix.CSVMatrix': pyaml.magnet.csvmatrix.CSVMatrixConfigurationSchema,\n", + " 'pyaml.magnet.curve.Curve': pyaml.magnet.curve.CurveConfigurationSchema,\n", + " 'pyaml.magnet.hcorrector.HCorrector': pyaml.magnet.hcorrector.HCorrectorConfigurationSchema,\n", + " 'pyaml.magnet.identity_cfm_model.IdentityCFMagnetModel': pyaml.magnet.identity_cfm_model.IdentityCFMagnetModelConfigurationSchema,\n", + " 'pyaml.magnet.identity_model.IdentityMagnetModel': pyaml.magnet.identity_model.IdentityMagnetModelConfigurationSchema,\n", + " 'pyaml.magnet.inline_curve.InlineCurve': pyaml.magnet.inline_curve.InlineCurveConfigurationSchema,\n", + " 'pyaml.magnet.inline_matrix.InlineMatrix': pyaml.magnet.inline_matrix.InlineMatrixConfigurationSchema,\n", + " 'pyaml.magnet.linear_cfm_model.LinearCFMagnetModel': pyaml.magnet.linear_cfm_model.LinearCFMagnetModelConfigurationSchema,\n", + " 'pyaml.magnet.linear_model.LinearMagnetModel': pyaml.magnet.linear_model.LinearMagnetModelConfigurationSchema,\n", + " 'pyaml.magnet.linear_serialized_model.LinearSerializedMagnetModel': pyaml.magnet.linear_serialized_model.LinearSerializedMagnetModelConfigurationSchema,\n", + " 'pyaml.magnet.magnet.Magnet': pyaml.magnet.magnet.MagnetConfigurationSchema,\n", + " 'pyaml.magnet.matrix.Matrix': pyaml.magnet.matrix.MatrixConfigurationSchema,\n", + " 'pyaml.magnet.model.MagnetModel': pyaml.magnet.model.MagnetModelConfigurationSchema,\n", + " 'pyaml.magnet.octupole.Octupole': pyaml.magnet.octupole.OctupoleConfigurationSchema,\n", + " 'pyaml.magnet.quadrupole.Quadrupole': pyaml.magnet.quadrupole.QuadrupoleConfigurationSchema,\n", + " 'pyaml.magnet.serialized_magnet.SerializedMagnets': pyaml.magnet.serialized_magnet.SerializedMagnetsConfigurationSchema,\n", + " 'pyaml.magnet.sextupole.Sextupole': pyaml.magnet.sextupole.SextupoleConfigurationSchema,\n", + " 'pyaml.magnet.skewoctu.SkewOctu': pyaml.magnet.skewoctu.SkewOctuConfigurationSchema,\n", + " 'pyaml.magnet.skewquad.SkewQuad': pyaml.magnet.skewquad.SkewQuadConfigurationSchema,\n", + " 'pyaml.magnet.skewsext.SkewSext': pyaml.magnet.skewsext.SkewSextConfigurationSchema,\n", + " 'pyaml.magnet.spline_model.SplineMagnetModel': pyaml.magnet.spline_model.SplineMagnetModelConfigurationSchema,\n", + " 'pyaml.magnet.vcorrector.VCorrector': pyaml.magnet.vcorrector.VCorrectorConfigurationSchema,\n", + " 'pyaml.rf.rf_plant.RFPlant': pyaml.rf.rf_plant.RFPlantConfigurationSchema,\n", + " 'pyaml.rf.rf_transmitter.RFTransmitter': pyaml.rf.rf_transmitter.RFTransmitterConfigurationSchema,\n", + " 'pyaml.tuning_tools.bba.BBA': pyaml.tuning_tools.bba.BBAConfigurationSchema,\n", + " 'pyaml.tuning_tools.bba2.BBA2': pyaml.tuning_tools.bba2.BBA2ConfigurationSchema,\n", + " 'pyaml.tuning_tools.chromaticity.Chromaticity': pyaml.tuning_tools.chromaticity.ChromaticityConfigurationSchema,\n", + " 'pyaml.tuning_tools.chromaticity_monitor.ChromaticityMonitor': pyaml.tuning_tools.chromaticity_monitor.ChromaticityMonitorConfigurationSchema,\n", + " 'pyaml.tuning_tools.chromaticity_response_matrix.ChromaticityResponseMatrix': pyaml.tuning_tools.chromaticity_response_matrix.ChromaticityResponseMatrixConfigurationSchema,\n", + " 'pyaml.tuning_tools.dispersion.Dispersion': pyaml.tuning_tools.dispersion.DispersionConfigurationSchema,\n", + " 'pyaml.tuning_tools.measurement_tool.MeasurementTool': pyaml.tuning_tools.measurement_tool.MeasurementToolConfigurationSchema,\n", + " 'pyaml.tuning_tools.orbit.Orbit': pyaml.tuning_tools.orbit.OrbitConfigurationSchema,\n", + " 'pyaml.tuning_tools.orbit_response_matrix.OrbitResponseMatrix': pyaml.tuning_tools.orbit_response_matrix.OrbitResponseMatrixConfigurationSchema,\n", + " 'pyaml.tuning_tools.orbit_response_matrix_data.OrbitResponseMatrixData': pyaml.tuning_tools.orbit_response_matrix_data.OrbitResponseMatrixDataConfigurationSchema,\n", + " 'pyaml.tuning_tools.response_matrix_data.ResponseMatrixData': pyaml.tuning_tools.response_matrix_data.ResponseMatrixDataConfigurationSchema,\n", + " 'pyaml.tuning_tools.tune.Tune': pyaml.tuning_tools.tune.TuneConfigurationSchema,\n", + " 'pyaml.tuning_tools.tune_response_matrix.TuneResponseMatrix': pyaml.tuning_tools.tune_response_matrix.TuneResponseMatrixConfigurationSchema,\n", + " 'pyaml.tuning_tools.tuning_tool.TuningTool': pyaml.tuning_tools.tuning_tool.TuningToolConfigurationSchema,\n", + " 'pyaml.validation.validation_models.DynamicValidation': pyaml.validation.validation_models.DynamicValidationConfigurationSchema,\n", + " 'pyaml_cs_oa.catalog.Catalog': pyaml_cs_oa.catalog.CatalogConfigurationSchema,\n", + " 'pyaml_cs_oa.controlsystem.OphydAsyncControlSystem': pyaml_cs_oa.controlsystem.OphydAsyncControlSystemConfigurationSchema,\n", + " 'pyaml_cs_oa.dynamic_catalog.DynamicCatalog': pyaml_cs_oa.dynamic_catalog.DynamicCatalogConfigurationSchema,\n", + " 'pyaml_cs_oa.epicsR.EpicsR': pyaml_cs_oa.epicsR.EpicsRConfigurationSchema,\n", + " 'pyaml_cs_oa.epicsRW.EpicsRW': pyaml_cs_oa.epicsRW.EpicsRWConfigurationSchema,\n", + " 'pyaml_cs_oa.epicsW.EpicsW': pyaml_cs_oa.epicsW.EpicsWConfigurationSchema,\n", + " 'pyaml_cs_oa.float_signal.FloatSignalContainer': pyaml_cs_oa.float_signal.FloatSignalContainerConfigurationSchema,\n", + " 'pyaml_cs_oa.signal.OASignal': pyaml_cs_oa.signal.OASignalConfigurationSchema,\n", + " 'pyaml_cs_oa.static_catalog.StaticCatalog': pyaml_cs_oa.static_catalog.StaticCatalogConfigurationSchema,\n", + " 'pyaml_cs_oa.static_catalog_entry.StaticCatalogEntry': pyaml_cs_oa.static_catalog_entry.StaticCatalogEntryConfigurationSchema,\n", + " 'pyaml_cs_oa.tangoAtt.TangoAtt': pyaml_cs_oa.tangoAtt.TangoAttConfigurationSchema,\n", + " 'pyaml_cs_oa.types.EpicsConfigR': pyaml_cs_oa.types.EpicsConfigRConfigurationSchema,\n", + " 'pyaml_cs_oa.types.EpicsConfigRW': pyaml_cs_oa.types.EpicsConfigRWConfigurationSchema,\n", + " 'pyaml_cs_oa.types.EpicsConfigW': pyaml_cs_oa.types.EpicsConfigWConfigurationSchema,\n", + " 'pyaml_cs_oa.types.TangoConfigAtt': pyaml_cs_oa.types.TangoConfigAttConfigurationSchema,\n", + " 'tango.pyaml.attribute.Attribute': tango.pyaml.attribute.AttributeConfigurationSchema,\n", + " 'tango.pyaml.attribute_list.AttributeList': tango.pyaml.attribute_list.AttributeListConfigurationSchema,\n", + " 'tango.pyaml.attribute_list_read_only.AttributeListReadOnly': tango.pyaml.attribute_list_read_only.AttributeListReadOnlyConfigurationSchema,\n", + " 'tango.pyaml.attribute_read_only.AttributeReadOnly': tango.pyaml.attribute_read_only.AttributeReadOnlyConfigurationSchema,\n", + " 'tango.pyaml.catalog.Catalog': tango.pyaml.catalog.CatalogConfigurationSchema,\n", + " 'tango.pyaml.controlsystem.TangoControlSystem': tango.pyaml.controlsystem.TangoControlSystemConfigurationSchema,\n", + " 'tango.pyaml.initializable_element.InitializableElement': tango.pyaml.initializable_element.InitializableElementConfigurationSchema,\n", + " 'tango.pyaml.multi_attribute.MultiAttribute': tango.pyaml.multi_attribute.MultiAttributeConfigurationSchema,\n", + " 'tango.pyaml.static_catalog.StaticCatalog': tango.pyaml.static_catalog.StaticCatalogConfigurationSchema,\n", + " 'tango.pyaml.static_catalog_entry.StaticCatalogEntry': tango.pyaml.static_catalog_entry.StaticCatalogEntryConfigurationSchema,\n", + " 'tango.pyaml.tango_catalog.TangoCatalog': tango.pyaml.tango_catalog.TangoCatalogConfigurationSchema,\n", + ")\n" + ] + } + ], + "source": [ + "from pyaml.validation import SchemaRegistry\n", + "\n", + "registry = SchemaRegistry()\n", + "registry.discover()\n", + "print(registry)" + ] + }, + { + "cell_type": "markdown", + "id": "b3926886", + "metadata": {}, + "source": [ + "## Create Configuration for Accelerator\n", + "\n", + "To create a configuration for an accelerator start by extracting the schema for it from the registry. You can see the required fields and their description using `describe()`.\n", + "\n", + "Unfortunately static type checking does not work if the schema is generated dynamically." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "c3d02b59", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "AcceleratorConfigurationSchema(\n", + " class_path: str — Fully qualified class path.\n", + " facility: str\n", + " machine: str\n", + " energy: float\n", + " alphac: float | None\n", + " harmonic_number: int | None\n", + " controls: list[pyaml.control.controlsystem.ControlSystemConfigurationSchema] | None\n", + " simulators: list[pyaml.lattice.simulator.SimulatorConfigurationSchema] | None\n", + " arrays: list[pyaml.arrays.array.ArrayConfigConfigurationSchema] | None\n", + " devices: list[pyaml.common.element.ElementConfigurationSchema] | None\n", + " data_folder: str | None\n", + " description: str | None\n", + ")\n" + ] + } + ], + "source": [ + "accelerator_schema = registry[\"pyaml.accelerator.Accelerator\"]\n", + "print(accelerator_schema.describe())" + ] + }, + { + "cell_type": "markdown", + "id": "a5a1ae55", + "metadata": {}, + "source": [ + "For the subschemas, you can list available subclasses that are available in the registry." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "0c2c9f7e", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "pyaml_cs_oa.controlsystem.OphydAsyncControlSystem → OphydAsyncControlSystemConfigurationSchema\n", + "tango.pyaml.controlsystem.TangoControlSystem → TangoControlSystemConfigurationSchema\n" + ] + } + ], + "source": [ + "control_system_schema = registry[\"pyaml.control.controlsystem.ControlSystem\"]\n", + "\n", + "control_system_options = {\n", + " class_path: schema\n", + " for class_path, schema in registry.items()\n", + " if schema is not control_system_schema\n", + " and (\n", + " issubclass(schema, control_system_schema)\n", + " or schema.is_virtual_subclass_of(control_system_schema)\n", + " )\n", + "}\n", + "\n", + "for class_path, schema in sorted(control_system_options.items()):\n", + " print(class_path, \"→\", schema.__name__)" + ] + }, + { + "cell_type": "markdown", + "id": "bcb2ce91", + "metadata": {}, + "source": [ + "You can list the required fields for them in the same way as before." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "a16219eb", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "TangoControlSystemConfigurationSchema(\n", + " class_path: str — Fully qualified class path.\n", + " name: str\n", + " tango_host: str | None\n", + " catalog: tango.pyaml.catalog.CatalogConfigurationSchema | None\n", + " debug_level: str | int | None\n", + " lazy_devices: bool\n", + " timeout_ms: int\n", + ")\n" + ] + } + ], + "source": [ + "tango_schema = registry[\"tango.pyaml.controlsystem.TangoControlSystem\"]\n", + "print(tango_schema.describe())" + ] + }, + { + "cell_type": "markdown", + "id": "6a8e0a65", + "metadata": {}, + "source": [ + "In this way you can program the configuration. Since `ConfigurationSchema` inherits from Pydantic `BaseModel` the configuration is validated for each object you create." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "f09e04bf", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "class_path='tango.pyaml.controlsystem.TangoControlSystem' name='live' tango_host=None catalog=None debug_level=None lazy_devices=True timeout_ms=3000\n" + ] + } + ], + "source": [ + "live_mode = tango_schema(\n", + " class_path = \"tango.pyaml.controlsystem.TangoControlSystem\",\n", + " name = \"live\",\n", + " )\n", + "\n", + "print(type(live_mode))\n", + "print(live_mode)" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "13fb5578", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "class_path='pyaml.accelerator.Accelerator' facility='pyaml_facility' machine='storage_ring' energy=1000000.0 alphac=None harmonic_number=None controls=[TangoControlSystemConfigurationSchema(class_path='tango.pyaml.controlsystem.TangoControlSystem', name='live', tango_host=None, catalog=None, debug_level=None, lazy_devices=True, timeout_ms=3000)] simulators=None arrays=None devices=None data_folder=None description=None\n" + ] + } + ], + "source": [ + "accelerator = accelerator_schema(\n", + " class_path='pyaml.accelerator.Accelerator',\n", + " facility = 'pyaml_facility',\n", + " machine = 'storage_ring',\n", + " energy = 1e6,\n", + " controls = [live_mode]\n", + " )\n", + "\n", + "print(type(accelerator))\n", + "print(accelerator)" + ] + }, + { + "cell_type": "markdown", + "id": "cfde8f36", + "metadata": {}, + "source": [ + "You have now created a Pydantic BaseModel which describes the configuration and can dump it to a dictionary or JSON using Pydantic functionality depending on what you prefer." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "ad78da2c", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'class_path': 'pyaml.accelerator.Accelerator', 'facility': 'pyaml_facility', 'machine': 'storage_ring', 'energy': 1000000.0, 'alphac': None, 'harmonic_number': None, 'controls': [{'class_path': 'tango.pyaml.controlsystem.TangoControlSystem', 'name': 'live', 'tango_host': None, 'catalog': None, 'debug_level': None, 'lazy_devices': True, 'timeout_ms': 3000}], 'simulators': None, 'arrays': None, 'devices': None, 'data_folder': None, 'description': None}\n" + ] + } + ], + "source": [ + "print(accelerator.model_dump())" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "7e48e0e6", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\n", + " \"class_path\": \"pyaml.accelerator.Accelerator\",\n", + " \"facility\": \"pyaml_facility\",\n", + " \"machine\": \"storage_ring\",\n", + " \"energy\": 1000000.0,\n", + " \"alphac\": null,\n", + " \"harmonic_number\": null,\n", + " \"controls\": [\n", + " {\n", + " \"class_path\": \"tango.pyaml.controlsystem.TangoControlSystem\",\n", + " \"name\": \"live\",\n", + " \"tango_host\": null,\n", + " \"catalog\": null,\n", + " \"debug_level\": null,\n", + " \"lazy_devices\": true,\n", + " \"timeout_ms\": 3000\n", + " }\n", + " ],\n", + " \"simulators\": null,\n", + " \"arrays\": null,\n", + " \"devices\": null,\n", + " \"data_folder\": null,\n", + " \"description\": null\n", + "}\n" + ] + } + ], + "source": [ + "print(accelerator.model_dump_json(indent=2))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3be782be", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "pyaml-documentation", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.16" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/how-to/configuration/use-meta-configurator.md b/docs/source/how-to/configuration/use-meta-configurator.md new file mode 100644 index 0000000..8367a1b --- /dev/null +++ b/docs/source/how-to/configuration/use-meta-configurator.md @@ -0,0 +1,60 @@ +# Use the MetaConfigurator + +[MetaConfigurator](https://github.com/MetaConfigurator/meta-configurator) is a browser-based editor that can generate an editing form from a [JSON Schema](https://json-schema.org/). It can be used for writing and editing pyAML configuration files without having to remember every available field and its expected type. + +## Prepare a JSON Schema + +Pre-generated schemas are available in [pyaml-schemas](https://github.com/python-accelerator-middle-layer/pyaml-schemas). These include the classes that are part of the pyAML ecosystem. + +If you have facility specific classes that you want to include in your configuration, you can use the `SchemaRegistry` to generate a JSON Schema including them. See [Generate JSON Schemas](../configuration/generate-json-schema) for details. + +The schema must describe the document you want to create. For example, a schema for a quadrupole is suitable for editing a quadrupole object, but not for editing a complete accelerator containing controls, simulators, and devices. + +## Overview of MetaConfigurator + +Open the [MetaConfigurator](https://metaconfigurator.github.io/meta-configurator/data) in a browser. In this guide we use the experimental version to benefit from the latest bug fixes. + +```{figure} /_static/metaconfigurator_frontpage.png +:alt: Frontpage for the MetaConfigurator +:width: 80% + +Frontpage of the MetaConfigurator. +``` + +The editor has two different pages: data and schema. The menu in the top left corner shows which page you are currently on. When loading the site from start it normally opens on the data page. The pages have different purposes: + +- **Data**: edit and validate data against a schema +- **Schema**: edit or generate a schema + +In this guide we will focus on the data page but there is also a lot of other functionality to explore. See the [MetaConfigurator Documentation](https://github.com/MetaConfigurator/meta-configurator/tree/develop/documentation_user) for this. + +On the data page, the editor has two areas: + +- **Text view** (on the left): shows and edits the json or yaml document. +- **GUI view** (on the right): presents the document as a form generated from the loaded schema. Here you can add values and choose options in a menu. + +The GUI View is convenient for knowing which fields exist and their required types. The Text View is useful for checking the final structure and for switching between supported formats. + +## Load the Schema + +1. Use the schema menu to load the JSON Schema. This menu normally shows directly when loading the page. If not, you need to switch to the schema page to load it. You can load the schema from a file or an URL. +2. Switch back to the data page. In the GUI view you should now see a form with fields. Above this view there is an option to choose the format for the text view. + +```{figure} /_static/metaconfigurator_gui_view.png +:alt: GUI View after loading the accelerator schema +:width: 80% + +GUI view after loading the accelerator schema. +``` + +## Create the Configuration + +Start with the top-level object expected by the schema. Set the `class` field to the fully qualified path to the Python class that pyAML should build when loading the configuration. The available options will be shown in a drop down menu. + +The exact fields depend on the schema and the class being configured. Use the GUI view to add properties and enter values, then inspect the generated document in the text view. + +There is also an option to use AI prompts to modify the data but this might require configuring your own backend connection for frequent use. See the [AI Assistance Documentation](https://github.com/MetaConfigurator/meta-configurator/tree/develop/documentation_user/examples/ai_assistance). + +## Export the Result + +When the document is complete, use the menu in the text view to download the file. It can then be used in pyAML. diff --git a/docs/source/how-to/configuration/use-schema-registry.ipynb b/docs/source/how-to/configuration/use-schema-registry.ipynb index 57d7ff5..9b62b72 100644 --- a/docs/source/how-to/configuration/use-schema-registry.ipynb +++ b/docs/source/how-to/configuration/use-schema-registry.ipynb @@ -7,7 +7,9 @@ "source": [ "# Use the Schema Registry\n", "\n", - "This guide shows how to register configuration schemas and use the `SchemaRegistry` to validate configuration and generate JSON Schemas." + "This guide shows how to register configuration schemas in the `SchemaRegistry` and how to explore the contents of the registry.\n", + "\n", + "The registry can be used to [validate configuration](./validate-configuration) or [generate JSON Schema](./generate-json-schema)." ] }, { @@ -211,7 +213,7 @@ "output_type": "stream", "text": [ "SchemaRegistry(\n", - " 'abc.ABC': abc.ABCConfigurationSchema,\n", + " 'pyaml.accelerator.Accelerator': pyaml.accelerator.AcceleratorConfigurationSchema,\n", " 'pyaml.arrays.array.ArrayConfig': pyaml.arrays.array.ArrayConfigConfigurationSchema,\n", " 'pyaml.arrays.bpm.BPM': pyaml.arrays.bpm.BPMConfigurationSchema,\n", " 'pyaml.arrays.cfm_magnet.CombinedFunctionMagnet': pyaml.arrays.cfm_magnet.CombinedFunctionMagnetConfigurationSchema,\n", @@ -221,6 +223,8 @@ " 'pyaml.bpm.bpm.BPM': pyaml.bpm.bpm.BPMConfigurationSchema,\n", " 'pyaml.common.element.Element': pyaml.common.element.ElementConfigurationSchema,\n", " 'pyaml.common.holders.element_holder.ElementHolder': pyaml.common.holders.element_holder.ElementHolderConfigurationSchema,\n", + " 'pyaml.control.controlsystem.ControlSystem': pyaml.control.controlsystem.ControlSystemConfigurationSchema,\n", + " 'pyaml.control.deviceaccess.DeviceAccess': pyaml.control.deviceaccess.DeviceAccessConfigurationSchema,\n", " 'pyaml.diagnostics.atune_monitor.ABetatronTuneMonitor': pyaml.diagnostics.atune_monitor.ABetatronTuneMonitorConfigurationSchema,\n", " 'pyaml.diagnostics.tune_monitor.BetatronTuneMonitor': pyaml.diagnostics.tune_monitor.BetatronTuneMonitorConfigurationSchema,\n", " 'pyaml.lattice.attribute_linker.PyAtAttributeElementsLinker': pyaml.lattice.attribute_linker.PyAtAttributeElementsLinkerConfigurationSchema,\n", @@ -268,7 +272,21 @@ " 'pyaml.tuning_tools.tune_response_matrix.TuneResponseMatrix': pyaml.tuning_tools.tune_response_matrix.TuneResponseMatrixConfigurationSchema,\n", " 'pyaml.tuning_tools.tuning_tool.TuningTool': pyaml.tuning_tools.tuning_tool.TuningToolConfigurationSchema,\n", " 'pyaml.validation.validation_models.DynamicValidation': pyaml.validation.validation_models.DynamicValidationConfigurationSchema,\n", - " 'typing.Any': typing.AnyConfigurationSchema,\n", + " 'pyaml_cs_oa.catalog.Catalog': pyaml_cs_oa.catalog.CatalogConfigurationSchema,\n", + " 'pyaml_cs_oa.controlsystem.OphydAsyncControlSystem': pyaml_cs_oa.controlsystem.OphydAsyncControlSystemConfigurationSchema,\n", + " 'pyaml_cs_oa.dynamic_catalog.DynamicCatalog': pyaml_cs_oa.dynamic_catalog.DynamicCatalogConfigurationSchema,\n", + " 'pyaml_cs_oa.epicsR.EpicsR': pyaml_cs_oa.epicsR.EpicsRConfigurationSchema,\n", + " 'pyaml_cs_oa.epicsRW.EpicsRW': pyaml_cs_oa.epicsRW.EpicsRWConfigurationSchema,\n", + " 'pyaml_cs_oa.epicsW.EpicsW': pyaml_cs_oa.epicsW.EpicsWConfigurationSchema,\n", + " 'pyaml_cs_oa.float_signal.FloatSignalContainer': pyaml_cs_oa.float_signal.FloatSignalContainerConfigurationSchema,\n", + " 'pyaml_cs_oa.signal.OASignal': pyaml_cs_oa.signal.OASignalConfigurationSchema,\n", + " 'pyaml_cs_oa.static_catalog.StaticCatalog': pyaml_cs_oa.static_catalog.StaticCatalogConfigurationSchema,\n", + " 'pyaml_cs_oa.static_catalog_entry.StaticCatalogEntry': pyaml_cs_oa.static_catalog_entry.StaticCatalogEntryConfigurationSchema,\n", + " 'pyaml_cs_oa.tangoAtt.TangoAtt': pyaml_cs_oa.tangoAtt.TangoAttConfigurationSchema,\n", + " 'pyaml_cs_oa.types.EpicsConfigR': pyaml_cs_oa.types.EpicsConfigRConfigurationSchema,\n", + " 'pyaml_cs_oa.types.EpicsConfigRW': pyaml_cs_oa.types.EpicsConfigRWConfigurationSchema,\n", + " 'pyaml_cs_oa.types.EpicsConfigW': pyaml_cs_oa.types.EpicsConfigWConfigurationSchema,\n", + " 'pyaml_cs_oa.types.TangoConfigAtt': pyaml_cs_oa.types.TangoConfigAttConfigurationSchema,\n", ")\n" ] } @@ -377,7 +395,6 @@ "- pyaml.magnet.skewquad.SkewQuad\n", "- pyaml.magnet.skewsext.SkewSext\n", "- pyaml.magnet.vcorrector.VCorrector\n", - "- typing.Any\n", "- pyaml.magnet.cfm_magnet.CombinedFunctionMagnet\n", "- pyaml.magnet.serialized_magnet.SerializedMagnets\n", "- pyaml.diagnostics.tune_monitor.BetatronTuneMonitor\n", @@ -388,10 +405,11 @@ "- pyaml.tuning_tools.measurement_tool.MeasurementTool\n", "- pyaml.arrays.array.ArrayConfig\n", "- pyaml.lattice.lattice_elements_linker.LinkerConfigModel\n", - "- abc.ABC\n", "- pyaml.lattice.lattice_elements_linker.LatticeElementsLinker\n", "- pyaml.lattice.simulator.Simulator\n", "- pyaml.common.holders.element_holder.ElementHolder\n", + "- pyaml.control.controlsystem.ControlSystem\n", + "- pyaml.accelerator.Accelerator\n", "- pyaml.arrays.bpm.BPM\n", "- pyaml.arrays.cfm_magnet.CombinedFunctionMagnet\n", "- pyaml.arrays.element.Element\n", @@ -422,6 +440,22 @@ "- pyaml.tuning_tools.orbit_response_matrix.OrbitResponseMatrix\n", "- pyaml.tuning_tools.tune.Tune\n", "- pyaml.tuning_tools.tune_response_matrix.TuneResponseMatrix\n", + "- pyaml_cs_oa.epicsR.EpicsR\n", + "- pyaml_cs_oa.types.EpicsConfigR\n", + "- pyaml_cs_oa.types.EpicsConfigW\n", + "- pyaml_cs_oa.types.EpicsConfigRW\n", + "- pyaml_cs_oa.types.TangoConfigAtt\n", + "- pyaml_cs_oa.float_signal.FloatSignalContainer\n", + "- pyaml_cs_oa.signal.OASignal\n", + "- pyaml.control.deviceaccess.DeviceAccess\n", + "- pyaml_cs_oa.epicsRW.EpicsRW\n", + "- pyaml_cs_oa.epicsW.EpicsW\n", + "- pyaml_cs_oa.tangoAtt.TangoAtt\n", + "- pyaml_cs_oa.catalog.Catalog\n", + "- pyaml_cs_oa.controlsystem.OphydAsyncControlSystem\n", + "- pyaml_cs_oa.dynamic_catalog.DynamicCatalog\n", + "- pyaml_cs_oa.static_catalog_entry.StaticCatalogEntry\n", + "- pyaml_cs_oa.static_catalog.StaticCatalog\n", "\n", " Registered schema classes:\n", "- BPMConfigurationSchema\n", @@ -437,7 +471,6 @@ "- SkewQuadConfigurationSchema\n", "- SkewSextConfigurationSchema\n", "- VCorrectorConfigurationSchema\n", - "- AnyConfigurationSchema\n", "- CombinedFunctionMagnetConfigurationSchema\n", "- SerializedMagnetsConfigurationSchema\n", "- BetatronTuneMonitorConfigurationSchema\n", @@ -448,10 +481,11 @@ "- MeasurementToolConfigurationSchema\n", "- ArrayConfigConfigurationSchema\n", "- LinkerConfigModelConfigurationSchema\n", - "- ABCConfigurationSchema\n", "- LatticeElementsLinkerConfigurationSchema\n", "- SimulatorConfigurationSchema\n", "- ElementHolderConfigurationSchema\n", + "- ControlSystemConfigurationSchema\n", + "- AcceleratorConfigurationSchema\n", "- BPMConfigurationSchema\n", "- CombinedFunctionMagnetConfigurationSchema\n", "- ElementConfigurationSchema\n", @@ -481,7 +515,23 @@ "- OrbitConfigurationSchema\n", "- OrbitResponseMatrixConfigurationSchema\n", "- TuneConfigurationSchema\n", - "- TuneResponseMatrixConfigurationSchema\n" + "- TuneResponseMatrixConfigurationSchema\n", + "- EpicsRConfigurationSchema\n", + "- EpicsConfigRConfigurationSchema\n", + "- EpicsConfigWConfigurationSchema\n", + "- EpicsConfigRWConfigurationSchema\n", + "- TangoConfigAttConfigurationSchema\n", + "- FloatSignalContainerConfigurationSchema\n", + "- OASignalConfigurationSchema\n", + "- DeviceAccessConfigurationSchema\n", + "- EpicsRWConfigurationSchema\n", + "- EpicsWConfigurationSchema\n", + "- TangoAttConfigurationSchema\n", + "- CatalogConfigurationSchema\n", + "- OphydAsyncControlSystemConfigurationSchema\n", + "- DynamicCatalogConfigurationSchema\n", + "- StaticCatalogEntryConfigurationSchema\n", + "- StaticCatalogConfigurationSchema\n" ] } ], @@ -508,7 +558,7 @@ "output_type": "stream", "text": [ "\n", - "Number of registered schemas: 58\n" + "Number of registered schemas: 74\n" ] } ], @@ -516,630 +566,6 @@ "# Print the number of schemas in the registry\n", "print(f\"\\nNumber of registered schemas: {len(registry)}\")" ] - }, - { - "cell_type": "markdown", - "id": "f693c376", - "metadata": {}, - "source": [ - "## Validate Configuration\n", - "\n", - "Configuration data can be validated using the `SchemaValidator`. It makes use of the schema registry to extract which schema to validate against for a specific class.\n", - "\n", - "For validation to be possible the class must be registered in the schema registry. If the class is not registered, validation will be skipped, a warning given and the data kept unchanged. Beware that this can lead to unexpected errors." - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "id": "73114262", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n" - ] - } - ], - "source": [ - "model_path = \"pyaml.magnet.identity_model.IdentityMagnetModel\"\n", - "print(registry.get(model_path))" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "id": "b2523c34", - "metadata": {}, - "outputs": [], - "source": [ - "from pyaml.validation import SchemaRegistry, SchemaValidator\n", - "\n", - "registry = SchemaRegistry()\n", - "registry.discover()\n", - "\n", - "configuration = {\n", - " \"class_path\": \"pyaml.magnet.quadrupole.Quadrupole\",\n", - " \"name\": \"QF1\",\n", - " \"description\": \"This is the QF1 quadrupole magnet.\"\n", - "}\n", - "validated = SchemaValidator.validate(configuration)" - ] - }, - { - "cell_type": "markdown", - "id": "365329b2", - "metadata": {}, - "source": [ - "Validation also handles nested configuration data." - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "id": "2e5df599", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "class_path='pyaml.magnet.quadrupole.Quadrupole' name='QF1' model=IdentityMagnetModelConfigurationSchema(class_path='pyaml.magnet.identity_model.IdentityMagnetModel', powerconverter=None, physics='', unit='1/m') lattice_names=None description='This is the QF1 quadrupole magnet.'\n" - ] - } - ], - "source": [ - "configuration = {\n", - " \"class_path\": \"pyaml.magnet.quadrupole.Quadrupole\",\n", - " \"name\": \"QF1\",\n", - " \"model\": {\n", - " \"class_path\": \"pyaml.magnet.identity_model.IdentityMagnetModel\",\n", - " \"unit\": \"1/m\",\n", - " \"physics\": \"\"\n", - " },\n", - " \"description\": \"This is the QF1 quadrupole magnet.\"\n", - "}\n", - "\n", - "validated = SchemaValidator.validate(configuration)\n", - "print(validated)" - ] - }, - { - "cell_type": "markdown", - "id": "aa814612", - "metadata": {}, - "source": [ - "The validated result can also be returned as a dictionary." - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "id": "3308aba1", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'class_path': 'pyaml.magnet.quadrupole.Quadrupole',\n", - " 'description': 'This is the QF1 quadrupole magnet.',\n", - " 'lattice_names': None,\n", - " 'model': {'class_path': 'pyaml.magnet.identity_model.IdentityMagnetModel',\n", - " 'physics': '',\n", - " 'powerconverter': None,\n", - " 'unit': '1/m'},\n", - " 'name': 'QF1'}\n" - ] - } - ], - "source": [ - "from pprint import pprint\n", - "\n", - "validated_dict = SchemaValidator.validate_to_dict(configuration)\n", - "pprint(validated_dict )" - ] - }, - { - "cell_type": "markdown", - "id": "2d03ad15", - "metadata": {}, - "source": [ - "## Generate JSON Schema\n", - "\n", - "The registry can also be used together with the `SchemaGenerator` to generate JSON Schema to use with external tools.\n", - "\n", - "If a base schema has registered concrete or virtual subclasses, the generated JSON Schema includes those alternatives. This allows editors and other JSON Schema tools to offer the appropriate fields for each configuration type." - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "id": "dfb96bb7", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.quadrupole.Quadrupole',\n", - " 'description': 'Fully qualified class path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'description': {'default': None,\n", - " 'title': 'Description',\n", - " 'type': ['string', 'null']},\n", - " 'lattice_names': {'default': None,\n", - " 'title': 'Lattice Names',\n", - " 'type': ['string', 'null']},\n", - " 'model': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.identity_cfm_model.IdentityCFMagnetModel',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'multipoles': {'items': {'type': 'string'},\n", - " 'title': 'Multipoles',\n", - " 'type': 'array'},\n", - " 'physics': {'anyOf': [{'items': {'type': ['string',\n", - " 'null']},\n", - " 'type': 'array'},\n", - " {'type': 'null'}],\n", - " 'default': None,\n", - " 'title': 'Physics'},\n", - " 'powerconverters': {'anyOf': [{'items': {'type': ['string',\n", - " 'null']},\n", - " 'type': 'array'},\n", - " {'type': 'null'}],\n", - " 'default': None,\n", - " 'title': 'Powerconverters'},\n", - " 'units': {'anyOf': [{'items': {'type': 'string'},\n", - " 'type': 'array'},\n", - " {'type': 'null'}],\n", - " 'default': None,\n", - " 'title': 'Units'}},\n", - " 'required': ['class',\n", - " 'multipoles'],\n", - " 'title': 'IdentityCFMagnetModelConfigurationSchema',\n", - " 'type': 'object'},\n", - " {'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.identity_model.IdentityMagnetModel',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'physics': {'default': None,\n", - " 'title': 'Physics',\n", - " 'type': ['string',\n", - " 'null']},\n", - " 'powerconverter': {'default': None,\n", - " 'title': 'Powerconverter',\n", - " 'type': ['string',\n", - " 'null']},\n", - " 'unit': {'default': None,\n", - " 'title': 'Unit',\n", - " 'type': ['string',\n", - " 'null']}},\n", - " 'required': ['class'],\n", - " 'title': 'IdentityMagnetModelConfigurationSchema',\n", - " 'type': 'object'},\n", - " {'additionalProperties': False,\n", - " 'properties': {'calibration_factors': {'anyOf': [{'items': {'type': 'number'},\n", - " 'type': 'array'},\n", - " {'type': 'null'}],\n", - " 'default': None,\n", - " 'title': 'Calibration '\n", - " 'Factors'},\n", - " 'calibration_offsets': {'anyOf': [{'items': {'type': 'number'},\n", - " 'type': 'array'},\n", - " {'type': 'null'}],\n", - " 'default': None,\n", - " 'title': 'Calibration '\n", - " 'Offsets'},\n", - " 'class': {'const': 'pyaml.magnet.linear_cfm_model.LinearCFMagnetModel',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'curves': {'items': {'anyOf': [{'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'file': {'title': 'File',\n", - " 'type': 'string'}},\n", - " 'required': ['class',\n", - " 'file'],\n", - " 'title': 'CSVCurveConfigurationSchema',\n", - " 'type': 'object'},\n", - " {'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'mat': {'items': {'items': {'type': 'number'},\n", - " 'type': 'array'},\n", - " 'title': 'Mat',\n", - " 'type': 'array'}},\n", - " 'required': ['class',\n", - " 'mat'],\n", - " 'title': 'InlineCurveConfigurationSchema',\n", - " 'type': 'object'}],\n", - " 'title': 'CurveConfigurationSchema'},\n", - " 'title': 'Curves',\n", - " 'type': 'array'},\n", - " 'hardware_units': {'items': {'type': 'string'},\n", - " 'title': 'Hardware '\n", - " 'Units',\n", - " 'type': 'array'},\n", - " 'matrix': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.csvmatrix.CSVMatrix',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'file': {'title': 'File',\n", - " 'type': 'string'}},\n", - " 'required': ['class',\n", - " 'file'],\n", - " 'title': 'CSVMatrixConfigurationSchema',\n", - " 'type': 'object'},\n", - " {'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.inline_matrix.InlineMatrix',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'mat': {'items': {'items': {'type': 'number'},\n", - " 'type': 'array'},\n", - " 'title': 'Mat',\n", - " 'type': 'array'}},\n", - " 'required': ['class',\n", - " 'mat'],\n", - " 'title': 'InlineMatrixConfigurationSchema',\n", - " 'type': 'object'}],\n", - " 'title': 'MatrixConfigurationSchema'},\n", - " {'type': 'null'}],\n", - " 'default': None},\n", - " 'multipoles': {'items': {'type': 'string'},\n", - " 'title': 'Multipoles',\n", - " 'type': 'array'},\n", - " 'powerconverters': {'items': {'type': ['string',\n", - " 'null']},\n", - " 'title': 'Powerconverters',\n", - " 'type': 'array'},\n", - " 'pseudo_factors': {'anyOf': [{'items': {'type': 'number'},\n", - " 'type': 'array'},\n", - " {'type': 'null'}],\n", - " 'default': None,\n", - " 'title': 'Pseudo '\n", - " 'Factors'},\n", - " 'pseudo_offsets': {'anyOf': [{'items': {'type': 'number'},\n", - " 'type': 'array'},\n", - " {'type': 'null'}],\n", - " 'default': None,\n", - " 'title': 'Pseudo '\n", - " 'Offsets'},\n", - " 'units': {'anyOf': [{'items': {'type': 'string'},\n", - " 'type': 'array'},\n", - " {'type': 'null'}],\n", - " 'default': None,\n", - " 'title': 'Units'}},\n", - " 'required': ['class',\n", - " 'multipoles',\n", - " 'curves',\n", - " 'powerconverters',\n", - " 'hardware_units'],\n", - " 'title': 'LinearCFMagnetModelConfigurationSchema',\n", - " 'type': 'object'},\n", - " {'additionalProperties': False,\n", - " 'properties': {'calibration_factor': {'default': 1.0,\n", - " 'title': 'Calibration '\n", - " 'Factor',\n", - " 'type': 'number'},\n", - " 'calibration_offset': {'default': 0.0,\n", - " 'title': 'Calibration '\n", - " 'Offset',\n", - " 'type': 'number'},\n", - " 'class': {'const': 'pyaml.magnet.linear_model.LinearMagnetModel',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'crosstalk': {'default': 1.0,\n", - " 'title': 'Crosstalk',\n", - " 'type': 'number'},\n", - " 'curve': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'file': {'title': 'File',\n", - " 'type': 'string'}},\n", - " 'required': ['class',\n", - " 'file'],\n", - " 'title': 'CSVCurveConfigurationSchema',\n", - " 'type': 'object'},\n", - " {'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'mat': {'items': {'items': {'type': 'number'},\n", - " 'type': 'array'},\n", - " 'title': 'Mat',\n", - " 'type': 'array'}},\n", - " 'required': ['class',\n", - " 'mat'],\n", - " 'title': 'InlineCurveConfigurationSchema',\n", - " 'type': 'object'}],\n", - " 'title': 'CurveConfigurationSchema'},\n", - " {'type': 'null'}],\n", - " 'default': None},\n", - " 'hardware_unit': {'title': 'Hardware '\n", - " 'Unit',\n", - " 'type': 'string'},\n", - " 'powerconverter': {'default': None,\n", - " 'title': 'Powerconverter',\n", - " 'type': ['string',\n", - " 'null']},\n", - " 'unit': {'title': 'Unit',\n", - " 'type': 'string'}},\n", - " 'required': ['class',\n", - " 'unit',\n", - " 'hardware_unit'],\n", - " 'title': 'LinearMagnetModelConfigurationSchema',\n", - " 'type': 'object'},\n", - " {'additionalProperties': False,\n", - " 'properties': {'calibration_factors': {'anyOf': [{'type': 'number'},\n", - " {'items': {'type': 'number'},\n", - " 'type': 'array'},\n", - " {'type': 'null'}],\n", - " 'default': None,\n", - " 'title': 'Calibration '\n", - " 'Factors'},\n", - " 'calibration_offsets': {'anyOf': [{'type': 'number'},\n", - " {'items': {'type': 'number'},\n", - " 'type': 'array'},\n", - " {'type': 'null'}],\n", - " 'default': None,\n", - " 'title': 'Calibration '\n", - " 'Offsets'},\n", - " 'class': {'const': 'pyaml.magnet.linear_serialized_model.LinearSerializedMagnetModel',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'crosstalk': {'anyOf': [{'type': 'number'},\n", - " {'items': {'type': 'number'},\n", - " 'type': 'array'}],\n", - " 'default': 1.0,\n", - " 'title': 'Crosstalk'},\n", - " 'curves': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'file': {'title': 'File',\n", - " 'type': 'string'}},\n", - " 'required': ['class',\n", - " 'file'],\n", - " 'title': 'CSVCurveConfigurationSchema',\n", - " 'type': 'object'},\n", - " {'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'mat': {'items': {'items': {'type': 'number'},\n", - " 'type': 'array'},\n", - " 'title': 'Mat',\n", - " 'type': 'array'}},\n", - " 'required': ['class',\n", - " 'mat'],\n", - " 'title': 'InlineCurveConfigurationSchema',\n", - " 'type': 'object'}],\n", - " 'title': 'CurveConfigurationSchema'},\n", - " {'items': {'anyOf': [{'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'file': {'title': 'File',\n", - " 'type': 'string'}},\n", - " 'required': ['class',\n", - " 'file'],\n", - " 'title': 'CSVCurveConfigurationSchema',\n", - " 'type': 'object'},\n", - " {'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'mat': {'items': {'items': {'type': 'number'},\n", - " 'type': 'array'},\n", - " 'title': 'Mat',\n", - " 'type': 'array'}},\n", - " 'required': ['class',\n", - " 'mat'],\n", - " 'title': 'InlineCurveConfigurationSchema',\n", - " 'type': 'object'}],\n", - " 'title': 'CurveConfigurationSchema'},\n", - " 'type': 'array'}],\n", - " 'title': 'Curves'},\n", - " 'hardware_unit': {'default': None,\n", - " 'title': 'Hardware '\n", - " 'Unit',\n", - " 'type': ['string',\n", - " 'null']},\n", - " 'powerconverter': {'default': None,\n", - " 'title': 'Powerconverter',\n", - " 'type': ['string',\n", - " 'null']},\n", - " 'unit': {'default': None,\n", - " 'title': 'Unit',\n", - " 'type': ['string',\n", - " 'null']}},\n", - " 'required': ['class', 'curves'],\n", - " 'title': 'LinearSerializedMagnetModelConfigurationSchema',\n", - " 'type': 'object'},\n", - " {'additionalProperties': False,\n", - " 'properties': {'alpha': {'default': 0.0,\n", - " 'title': 'Alpha',\n", - " 'type': 'number'},\n", - " 'calibration_factor': {'default': 1.0,\n", - " 'title': 'Calibration '\n", - " 'Factor',\n", - " 'type': 'number'},\n", - " 'calibration_offset': {'default': 0.0,\n", - " 'title': 'Calibration '\n", - " 'Offset',\n", - " 'type': 'number'},\n", - " 'class': {'const': 'pyaml.magnet.spline_model.SplineMagnetModel',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'crosstalk': {'default': 1.0,\n", - " 'title': 'Crosstalk',\n", - " 'type': 'number'},\n", - " 'curve': {'anyOf': [{'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'file': {'title': 'File',\n", - " 'type': 'string'}},\n", - " 'required': ['class',\n", - " 'file'],\n", - " 'title': 'CSVCurveConfigurationSchema',\n", - " 'type': 'object'},\n", - " {'additionalProperties': False,\n", - " 'properties': {'class': {'const': 'pyaml.magnet.inline_curve.InlineCurve',\n", - " 'description': 'Fully '\n", - " 'qualified '\n", - " 'class '\n", - " 'path.',\n", - " 'title': 'Class',\n", - " 'type': 'string'},\n", - " 'mat': {'items': {'items': {'type': 'number'},\n", - " 'type': 'array'},\n", - " 'title': 'Mat',\n", - " 'type': 'array'}},\n", - " 'required': ['class',\n", - " 'mat'],\n", - " 'title': 'InlineCurveConfigurationSchema',\n", - " 'type': 'object'}],\n", - " 'title': 'CurveConfigurationSchema'},\n", - " 'hardware_unit': {'default': None,\n", - " 'title': 'Hardware '\n", - " 'Unit',\n", - " 'type': ['string',\n", - " 'null']},\n", - " 'powerconverter': {'default': None,\n", - " 'title': 'Powerconverter',\n", - " 'type': ['string',\n", - " 'null']},\n", - " 'unit': {'default': None,\n", - " 'title': 'Unit',\n", - " 'type': ['string',\n", - " 'null']}},\n", - " 'required': ['class', 'curve'],\n", - " 'title': 'SplineMagnetModelConfigurationSchema',\n", - " 'type': 'object'}],\n", - " 'title': 'MagnetModelConfigurationSchema'},\n", - " {'type': 'null'}],\n", - " 'default': None},\n", - " 'name': {'title': 'Name', 'type': 'string'}},\n", - " 'required': ['class', 'name'],\n", - " 'title': 'QuadrupoleConfigurationSchema',\n", - " 'type': 'object'}\n" - ] - } - ], - "source": [ - "from pprint import pprint\n", - "\n", - "from pyaml.validation import SchemaGenerator\n", - "\n", - "json_schema = SchemaGenerator.generate(\"pyaml.magnet.quadrupole.Quadrupole\")\n", - "pprint(json_schema)" - ] - }, - { - "cell_type": "markdown", - "id": "d329d7e5", - "metadata": {}, - "source": [ - "The result can also be saved directly to a file." - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "id": "12610af9", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "PosixPath('quadrupole-schema.json')" - ] - }, - "execution_count": 18, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "SchemaGenerator.save(\"pyaml.magnet.quadrupole.Quadrupole\",\"quadrupole-schema.json\")" - ] } ], "metadata": { diff --git a/docs/source/how-to/configuration/use-vscode-json-schema.md b/docs/source/how-to/configuration/use-vscode-json-schema.md new file mode 100644 index 0000000..e18b3c0 --- /dev/null +++ b/docs/source/how-to/configuration/use-vscode-json-schema.md @@ -0,0 +1,61 @@ +# Use JSON Schema in VS Code + +[VS Code](https://code.visualstudio.com/) can use a JSON Schema to help writing and editing JSON and YAML files. The schema provides completion, property descriptions, and diagnostics for invalid values. + +This guide shows how to associate a JSON Schema with JSON and YAML files and use it to write pyAML configuration files. + +## Required Extensions + +- **JSON**: JSON support is built into VS Code. JSON files are recognized automatically when their names end in `.json`. + +- **YAML**: For YAML you need to install an extension. Open the `Extensions` view, search for `YAML`, and install the extension published by Red Hat. + +## Enable Remote Schema Downloads + +To be able to use schemas that are hosted online (for example on GitHub), VS Code must be allowed to download the schema. Open File -> Preferences -> Settings and search for **JSON: Schema Download: Enable** and enable it. + +You also need to add the source to trusted domains. Search for **JSON: Schema Download: Trusted Domains** and add the domain you want to download from there. For GitHub that should be `https://raw.githubusercontent.com/`. + +## Add the JSON Schema + +You can associate a JSON Schema with a file in two ways, directly in the file or in the VS Code settings for a more permanent setup. The examples here show the option to do it directly in the file. See the [VS Code Documentation](https://code.visualstudio.com/docs/languages/json#_mapping-in-the-user-settings) for details how to do it in settings. + +The examples use the accelerator schema published in [pyaml-schemas](https://github.com/python-accelerator-middle-layer/pyaml-schemas) but it is also possible to use a local file. Place this as the first sentence in the file: + +**JSON**: + +```JSON +{ + "$schema": "https://raw.githubusercontent.com/python-accelerator-middle-layer/pyaml-schemas/main/schemas/accelerator.schema.json", +} +``` + +**YAML**: + +```YAML +# yaml-language-server: $schema=https://raw.githubusercontent.com/python-accelerator-middle-layer/pyaml-schemas/main/schemas/accelerator.schema.json +``` + +## Edit the file + +VS Code should now provide code completion when you type a property name. Hover over a property to see its description and red or yellow squiggles show when a value does not match the schema. + +The way the completion looks and works is slightly different between JSON and YAML. + +**JSON**: + +```{figure} /_static/vscode-json-schema-json.png +:alt: Hints for the JSON Schema in VS Code when writing JSON +:width: 80% + +Hints for the JSON Schema in VS Code when writing JSON. +``` + +**YAML**: + +```{figure} /_static/vscode-json-schema-yaml.png +:alt: Hints for the JSON Schema in VS Code +:width: 80% + +Hints for the JSON Schema in VS Code when writing YAML. +``` diff --git a/docs/source/how-to/configuration/validate-configuration.ipynb b/docs/source/how-to/configuration/validate-configuration.ipynb new file mode 100644 index 0000000..593f2f1 --- /dev/null +++ b/docs/source/how-to/configuration/validate-configuration.ipynb @@ -0,0 +1,161 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "f693c376", + "metadata": {}, + "source": [ + "# Validate Configuration\n", + "\n", + "The configuration can be validated using the `SchemaValidator`. It makes use of the schema registry to extract which schema to validate against for a specific class.\n", + "\n", + "For validation to be possible the class must be registered in the schema registry. If the class is not registered, validation will be skipped, a warning given and the data kept unchanged. Beware that this can lead to unexpected errors." + ] + }, + { + "cell_type": "markdown", + "id": "f2865e86", + "metadata": {}, + "source": [ + "## Create the Registry\n", + "\n", + "First create the registry and register the classes you want included in the schema.\n", + "\n", + "Here the discovery function is used to register all classes in `pyaml` and other packages which define entry points." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "fbbf800f", + "metadata": {}, + "outputs": [], + "source": [ + "from pyaml.validation import SchemaRegistry\n", + "\n", + "registry = SchemaRegistry()\n", + "registry.discover()" + ] + }, + { + "cell_type": "markdown", + "id": "671f48ba", + "metadata": {}, + "source": [ + "## Validate Data" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "b2523c34", + "metadata": {}, + "outputs": [], + "source": [ + "from pyaml.validation import SchemaValidator\n", + "\n", + "configuration = {\n", + " \"class_path\": \"pyaml.magnet.quadrupole.Quadrupole\",\n", + " \"name\": \"QF1\",\n", + " \"description\": \"This is the QF1 quadrupole magnet.\"\n", + "}\n", + "validated = SchemaValidator.validate(configuration)" + ] + }, + { + "cell_type": "markdown", + "id": "365329b2", + "metadata": {}, + "source": [ + "Validation also handles nested configuration data." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "2e5df599", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "class_path='pyaml.magnet.quadrupole.Quadrupole' name='QF1' model=IdentityMagnetModelConfigurationSchema(class_path='pyaml.magnet.identity_model.IdentityMagnetModel', powerconverter=None, physics='', unit='1/m') lattice_names=None description='This is the QF1 quadrupole magnet.'\n" + ] + } + ], + "source": [ + "configuration = {\n", + " \"class_path\": \"pyaml.magnet.quadrupole.Quadrupole\",\n", + " \"name\": \"QF1\",\n", + " \"model\": {\n", + " \"class_path\": \"pyaml.magnet.identity_model.IdentityMagnetModel\",\n", + " \"unit\": \"1/m\",\n", + " \"physics\": \"\"\n", + " },\n", + " \"description\": \"This is the QF1 quadrupole magnet.\"\n", + "}\n", + "\n", + "validated = SchemaValidator.validate(configuration)\n", + "print(validated)" + ] + }, + { + "cell_type": "markdown", + "id": "aa814612", + "metadata": {}, + "source": [ + "The validated result can also be returned as a dictionary." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "3308aba1", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'class_path': 'pyaml.magnet.quadrupole.Quadrupole',\n", + " 'description': 'This is the QF1 quadrupole magnet.',\n", + " 'lattice_names': None,\n", + " 'model': {'class_path': 'pyaml.magnet.identity_model.IdentityMagnetModel',\n", + " 'physics': '',\n", + " 'powerconverter': None,\n", + " 'unit': '1/m'},\n", + " 'name': 'QF1'}\n" + ] + } + ], + "source": [ + "from pprint import pprint\n", + "\n", + "validated_dict = SchemaValidator.validate_to_dict(configuration)\n", + "pprint(validated_dict )" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "pyaml-documentation", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.16" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/how-to/index.md b/docs/source/how-to/index.md index 8bee4f4..7a075ab 100644 --- a/docs/source/how-to/index.md +++ b/docs/source/how-to/index.md @@ -23,6 +23,11 @@ installation/developer-installation configuration/create-configuration configuration/use-schema-registry +configuration/validate-configuration +configuration/generate-json-schema +configuration/use-configuration-schema +configuration/use-meta-configurator +configuration/use-vscode-json-schema ```