Generate mappers ahead of time instead of mapping with reflection

.gitignore

@@ -8,3 +8,4 @@ infection.log
 infection.html
 .phpbench/
 docs_php/
+var

Makefile

@@ -34,6 +34,10 @@ static: phpstan cs                                               				## run stat
 
 test: phpunit                                                                   ## run tests
 
+.PHONY: snapshot
+snapshot: vendor                                                                ## regenerate the snapshots of the generated mappers
+	UPDATE_SNAPSHOTS=1 vendor/bin/phpunit --no-coverage --filter testGeneratedCodeMatchesSnapshot
+
 .PHONY: benchmark
 benchmark: vendor                                                               ## run benchmarks
 	vendor/bin/phpbench run tests/Benchmark --report=default

docs/extensions.md

@@ -23,13 +23,14 @@ $hydrator = (new StackHydratorBuilder())
 ```
 ## Built-in extensions
 
-The library ships with three extensions out of the box:
+The library ships with four extensions out of the box:
 
 | Extension | Purpose |
 | --- | --- |
 | `LifecycleExtension` | [Lifecycle hooks](lifecycle-hooks.md), run code before and after the extract and hydrate process. |
 | `CryptographyExtension` | [Cryptography](cryptography.md), encrypt and decrypt sensitive data with crypto-shredding. |
 | `UpcastExtension` | [Upcasting](upcasting.md), reshape outdated stored data while it is hydrated. |
+| `GeneratedMapperExtension` | [Generated code](generated-code.md), use generated mapping code instead of reflection. |
 
 ## Middleware
 
@@ -232,4 +233,5 @@ final class AuditExtension implements Extension
 * [How to run code before extract and after hydrate](lifecycle-hooks.md)
 * [How to encrypt sensitive data](cryptography.md)
 * [How to reshape outdated stored data](upcasting.md)
+* [How to speed up hydration with generated code](generated-code.md)
 * [How to cache the metadata](caching.md)

docs/generated-code.md

@@ -0,0 +1,108 @@
+# Generated Code
+
+By default the hydrator maps arrays to objects with reflection: the
+`ReflectionMapper` iterates over the metadata of a class, sets each
+property through a `ReflectionProperty` and calls the [normalizers](normalizer.md)
+dynamically. This is fast enough for most applications, but when you hydrate
+millions of objects the reflection overhead adds up. The
+`GeneratedMapperExtension` uses plain PHP code generated for your classes
+instead.
+
+The generated mappers plug in as a [mapper factory](extensions.md#mapper-factories).
+The generation and the runtime are strictly separated: a warmup step generates
+the code ahead of time, and the hydrator only loads it.
+
+## Enable the extension
+
+Register the `GeneratedMapperExtension` with the directory which contains
+the generated code.
+
+```php
+use Patchlevel\Hydrator\Extension\Generated\GeneratedMapperExtension;
+use Patchlevel\Hydrator\StackHydratorBuilder;
+
+$hydrator = (new StackHydratorBuilder())
+    ->useExtension(new GeneratedMapperExtension(__DIR__ . '/var/cache/hydrator'))
+    ->build();
+```
+For every class the hydrator looks for a generated mapper in this
+directory. Classes without one are mapped with reflection as usual.
+
+:::success
+The generated code behaves exactly like the `ReflectionMapper`: objects
+are created without calling the constructor, missing fields keep their promoted
+default value, private and readonly properties are supported and the same
+exceptions are thrown. You can switch between both without changing your
+classes.
+:::
+
+## Warm up
+
+The `GeneratedMapperWarmer` generates the mappers and writes them to
+the directory. Run it in a cache warmer or a deployment step, with the metadata
+factory of the builder, so the code matches the metadata of the hydrator,
+including the guessers and enrichers of all registered extensions.
+
+```php
+use Patchlevel\Hydrator\Extension\Generated\GeneratedMapperExtension;
+use Patchlevel\Hydrator\Extension\Generated\GeneratedMapperWarmer;
+use Patchlevel\Hydrator\StackHydratorBuilder;
+
+$cachePath = __DIR__ . '/var/cache/hydrator';
+
+$builder = (new StackHydratorBuilder())
+    ->useExtension(new GeneratedMapperExtension($cachePath));
+
+$warmer = new GeneratedMapperWarmer($builder->metadataFactory(), $cachePath);
+$warmer->warmup([
+    ProfileCreated::class,
+    NameChanged::class,
+]);
+```
+Classes referenced by these classes, such as nested value objects, get a
+mapper too. The warmup throws a `ClassNotGeneratable` exception for a
+class which can not be generated, for example an abstract class or a class with
+a class normalizer.
+
+:::note
+Only the warmer knows how to generate code. At runtime the extension only loads
+existing files, the generator is never loaded.
+:::
+
+## Changed classes
+
+The name of a generated file contains a fingerprint of everything the code
+depends on: the properties with their field names, normalizers, visibility and
+default values. If a class changes after the warmup, its generated mapper
+is simply not found and the class is mapped with reflection until you warm
+up again.
+
+:::warning
+A missing warmup does not fail, it only makes the hydration slower. Warm up the
+mappers on every deployment, like you do for other generated files.
+:::
+
+## Nested objects
+
+The biggest speedup comes from nested objects: instead of going through the
+normalizer and the hydrator again for every nested object, the generated code
+calls the [mapper](extensions.md#mapper-factories) the hydrator built for the
+nested class directly. That mapper already contains the
+[middlewares](extensions.md) and the [lazy](lazy.md) proxy of the nested class,
+so nothing changes in behaviour. With the [cryptography](cryptography.md)
+extension for example, value objects without sensitive data are mapped by their
+generated mapper right away, while classes with sensitive data still go
+through the middleware, without the detour through the hydrator.
+
+Only classes with a class normalizer are left to the normalizer.
+
+:::note
+If a hydrator wraps the `StackHydrator`, nested objects take the way through
+the normalizer for its calls, so it sees every nested object.
+:::
+
+## Learn more
+
+* [How to build the hydrator](hydrator.md)
+* [How to cache the metadata](caching.md)
+* [How to write your own extension](extensions.md)

docs/hydrator.md

@@ -180,3 +180,4 @@ value does not fit the property type.
 * [How normalizers convert complex types](normalizer.md)
 * [How to hydrate objects lazily](lazy.md)
 * [How to hook into the hydration process](extensions.md)
+* [How to speed up hydration with generated code](generated-code.md)

docs/project.json

@@ -18,7 +18,8 @@
       "subEntries": [
         { "title": "Lifecycle Hooks", "file": "lifecycle-hooks.md" },
         { "title": "Cryptography", "file": "cryptography.md" },
-        { "title": "Upcasting", "file": "upcasting.md" }
+        { "title": "Upcasting", "file": "upcasting.md" },
+        { "title": "Generated Code", "file": "generated-code.md" }
       ]
     },
     {

phpcs.xml.dist

@@ -10,6 +10,8 @@
 
     <file>src</file>
     <file>tests</file>
+    <!-- generated code, kept as snapshot -->
+    <exclude-pattern>tests/Unit/Extension/Generated/Snapshot/*</exclude-pattern>
 
     <rule ref="PatchlevelCodingStandard">
         <exclude name="Generic.CodeAnalysis.UselessOverridingMethod.Found"/>

phpstan.neon.dist

@@ -8,6 +8,8 @@ parameters:
 	paths:
 		- src
 		- tests
+	excludePaths:
+		- tests/Unit/Extension/Generated/Snapshot
 	reportUnmatchedIgnoredErrors: false
 
 services:
@@ -19,3 +21,7 @@ services:
 		class: Patchlevel\Hydrator\Tests\Architecture\ExceptionImplementsHydratorExceptionTest
 		tags:
 			- phpat.test
+	-
+		class: Patchlevel\Hydrator\Tests\Architecture\NoCodeGenerationAtRuntimeTest
+		tags:
+			- phpat.test

src/Extension/Generated/ChainNestedMapper.php

@@ -0,0 +1,41 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Patchlevel\Hydrator\Extension\Generated;
+
+use Patchlevel\Hydrator\Mapper\Mapper;
+
+/**
+ * A nested class whose chain is not a generated mapper, for example because middlewares run for it. The generated code
+ * then calls the whole chain, so the nested objects take the same way as through the hydrator.
+ *
+ * @internal
+ */
+final class ChainNestedMapper implements NestedMapper
+{
+    /** @param Mapper<object> $mapper */
+    public function __construct(
+        private readonly Mapper $mapper,
+    ) {
+    }
+
+    /**
+     * @param array<string, mixed> $data
+     * @param array<string, mixed> $context
+     */
+    public function hydrateNested(array $data, array $context): object
+    {
+        return $this->mapper->hydrate($data, $context);
+    }
+
+    /**
+     * @param array<string, mixed> $context
+     *
+     * @return array<string, mixed>
+     */
+    public function extractNested(object $object, array $context): array
+    {
+        return $this->mapper->extract($object, $context);
+    }
+}

src/Extension/Generated/ClassFingerprint.php

@@ -0,0 +1,104 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Patchlevel\Hydrator\Extension\Generated;
+
+use Patchlevel\Hydrator\Metadata\ClassMetadata;
+use Patchlevel\Hydrator\Normalizer\ArrayNormalizer;
+use Patchlevel\Hydrator\Normalizer\InvalidType;
+use Patchlevel\Hydrator\Normalizer\Normalizer;
+use Patchlevel\Hydrator\Normalizer\ObjectNormalizer;
+use ReflectionClass;
+
+use function class_exists;
+use function hash;
+use function serialize;
+use function substr;
+
+/**
+ * Fingerprint of everything the generated code of a class depends on: the properties with their field names,
+ * normalizers, visibility, declaring class and inlined defaults, and the nested classes. It is part of the name of the
+ * generated mapper, so a changed class has no generated mapper until it is warmed up again.
+ *
+ * @internal
+ */
+final class ClassFingerprint
+{
+    /**
+     * @param ClassMetadata<T> $metadata
+     *
+     * @template T of object
+     */
+    public static function of(ClassMetadata $metadata): string
+    {
+        $defaults = $metadata->promotedConstructorDefaults();
+        $shape = [$metadata->reflection->isFinal()];
+
+        foreach ($metadata->properties as $property) {
+            $reflection = $property->reflection;
+            $default = null;
+
+            if ($reflection->isPromoted() && isset($defaults[$property->propertyName])) {
+                // defaults which can not be inlined are resolved at runtime, their value does not matter here
+                $default = Literal::export($defaults[$property->propertyName]->getDefaultValue()) ?? 'runtime';
+            }
+
+            $declaringClass = $reflection->getDeclaringClass()->getName();
+            $nested = null;
+
+            if ($property->normalizer !== null) {
+                $nested = self::nested($property->normalizer);
+            }
+
+            $shape[] = [
+                $property->propertyName,
+                $property->fieldName,
+                $property->normalizer !== null,
+                $reflection->getModifiers(),
+                // inherited properties may need the scope of their declaring class
+                $declaringClass === $metadata->className ? null : $declaringClass,
+                $default,
+                $nested,
+            ];
+        }
+
+        return substr(hash('sha256', serialize($shape)), 0, 16);
+    }
+
+    /**
+     * Nested objects are mapped in place if they are normalized with an object normalizer, directly or in an array.
+     *
+     * @return array{class: class-string, array: bool, final: bool}|null
+     */
+    public static function nested(Normalizer $normalizer): array|null
+    {
+        $array = false;
+
+        if ($normalizer instanceof ArrayNormalizer) {
+            $array = true;
+            $normalizer = $normalizer->innerNormalizer();
+        }
+
+        if (!$normalizer instanceof ObjectNormalizer) {
+            return null;
+        }
+
+        try {
+            $class = $normalizer->className();
+        } catch (InvalidType) {
+            return null;
+        }
+
+        if (!class_exists($class)) {
+            return null;
+        }
+
+        return [
+            'class' => $class,
+            'array' => $array,
+            // the generated code checks the exact class of nested objects, which is cheaper for final classes
+            'final' => (new ReflectionClass($class))->isFinal(),
+        ];
+    }
+}

src/Extension/Generated/ClassNotGeneratable.php

@@ -0,0 +1,18 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Patchlevel\Hydrator\Extension\Generated;
+
+use InvalidArgumentException;
+use Patchlevel\Hydrator\HydratorException;
+
+use function sprintf;
+
+final class ClassNotGeneratable extends InvalidArgumentException implements HydratorException
+{
+    public function __construct(string $class, string $reason)
+    {
+        parent::__construct(sprintf('A mapper for class "%s" can not be generated: %s', $class, $reason));
+    }
+}