ArchLoader.Build() throws System.OutOfMemoryException

[Gargameleto]

When calling following code multiple times with different lists of assemblies:

            var archLoader = new ArchLoader();
            archLoader = archLoader.LoadAssembliesIncludingDependencies(assemblies);
            ArchUnitNET.Domain.Architecture architectureUnderTest = archLoader.Build();

somewhen an OutOfMemoryException is thrown.

The reason is that the internal static cache (ArchitectureCache) grows until there is no memory.

The problem can be avoided by following HACK, (which is no real option):

        private static void ClearInternalCache(ArchLoader archLoader)
        {
            System.Reflection.FieldInfo archBuilderField = archLoader.GetType().GetField("_archBuilder", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance);
            object archBuilderValue = archBuilderField.GetValue(archLoader);
            System.Reflection.FieldInfo architectureCacheField = archBuilderValue.GetType().GetField("_architectureCache", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance);
            object architectureCacheValue = architectureCacheField.GetValue(archBuilderValue);
            System.Reflection.MethodInfo clearMethod = architectureCacheValue.GetType().GetMethod("Clear");
            clearMethod.Invoke(architectureCacheValue, null);
        }

Therefore the cache mechanism shall be extended to be configurable or to be turned off so that OutOfMemoryException can be avoided.

[fl3pp]

This issue would likely be resolved if the ArchitectureCache made use of a MemoryCache instead of ConcurrentDictionary. MemoryCache is also thread-safe but automatically clears cached data once the available memory runs out.

@alexanderlinne would you be interested in such a PR?

[alexanderlinne]

Hi @fl3pp, sounds like a good short-term solution to me. I'd be happy to accept a PR on this.

I think it should also be possible to control the caching behaviour, but I'm planning to replace the Mono.Cecil-based ArchLoader with a System.Reflection-based one in the future, which will likely also involve changes to the caching behaviour anyways.

[fl3pp]

@alexanderlinne thanks for the quick response.

I've tried to implement the fix using the memory cache but quickly noticed that .NET doesn't deliver the MemoryCache built-in anymore, but through a NuGet package. Adding a dependency to ArchUnitNET is, of course, not something that is done without some further thought, even if it's a MS package.

Some less invasive fixes could involve making the .Clear() public or implementing an automatic cache clearing based on some metric, such as a maximum number of elements. Obviously, both come with downsides as well.

What do you think?

[fgather]

Hi @fl3pp,

IMO you can go forward with including the Microsoft.Extensions.Caching.Memory package :) Making .Clear() public seems to be a workaround which would reflect itself on the API which I'd rather not do. And before we invent our own cache, let's use the one from MS.

[Gargameleto]

Me and @fl3pp tried to use System.Runtime.Caching.MemoryCache. Unfortunately the amount of memory to use for the cache cannot be restricted. Therefore also when using the MemoryCache, we run into an OutOfMemoryException. The new Microsoft.Extensions.Caching.MemoryCache would provide limits but is not available for .net 4.5.

What about providing a property bool UseCache on the ArchLoader which is by default set to true?

[Gargameleto]

@alexanderlinne :
The bug seems to have been fixed :-). Would it be possible to create a release containing the fix?

[alexanderlinne]

@Gargameleto 0.13.4 is now published