Migrate the Scope generator off classic Xtend onto Xbase + JvmModelInferrer by rubenporras · Pull Request #1405 · dsldevkit/dsl-devkit

rubenporras · 2026-06-10T05:10:16Z

The com.avaloq.tools.ddk.xtext.scope plugin compiled the embedded
Expression DSL to Java through expression.generator.CodeGenerationX /
CompilationContext, whose type system is the legacy classic Xtend
runtime (org.eclipse.xtend.expression.,
org.eclipse.xtend.typesystem.emf.). That was the only reason the plugin
required those bundles.

This PR replaces the IGenerator2-based ScopeGenerator with an Xbase
JvmModelInferrer, introduces a self-contained (Xtend-free)
expression-translation layer, deletes the classic-Xtend execution
context, and removes org.eclipse.xtend and
org.eclipse.xtend.typesystem.emf from the scope plugin.

Key decisions

Scope generation moves to a JvmModelInferrer + Xbase
JvmModelGenerator Same pattern as Format/Check; lets the standard
XbaseCompiler emit the provider classes. The
*ScopeProvider/*ScopeNameProvider are now inferred JVM types rather than
hand-templated .java.

Hybrid expression backend: an Xbase translator + a string compiler
fallback String + concatenation (very common in .scope) has no
operator_plus in xbase.lib — Xbase special-cases it in its compiler — so
it can't be pre-linked as an XExpression tree. We translate to Xbase
what maps cleanly and fall back to a faithful port of the legacy string
output for +, arithmetic and relational operators.

Model-type names resolve via imported EPackages, not the classpath In
real sources, (Cast)x / typeSelect(T) / T.isInstance(x) name EMF model
types (FormDef, ILogicalTable, intfdef::AlternatingGroup), not Java
FQNs. javaType resolves them through the model's imported EPackages →
GenModelUtilX.instanceClassName, exactly as the legacy
EmfRegistryMetaModel did. findDeclaredType would fail on these.

.ext (JAVA extension) support dropped Scope sources no longer reference
Xtend extension files; removed the .ext-reading validation and the
isJavaExtensionCall/isExtension generator branches.

factory becomes a direct Xbase static call Must be written
Type.method(args); the generator resolves the type to a
JvmDeclaredType.qualifiedName and emits the static call. Bare factory
foo() is no longer supported.

Downstream projects need to adapt their .scope/.ext sources.

joaodinissf

Thanks for tackling the classic-Xtend retirement here — the Xbase/JvmModelInferrer direction looks right and the structure is clean. I went deep on the hand-written translation layer (the src-gen is noise) and found one blocker plus a semantics change worth confirming. Flagging the key risk up front: dsl-devkit has no .scope/.export inputs, so the inferrers/compilers never run in CI — only compilation is checked. Everything below is therefore invisible to the green build and would only surface as wrong generated Java in ASMD.

Blocker: arithmetic - / * / / emits non-compilable Java (-(a, 1) instead of a - 1) — details inline, affects both scope and export. This is a regression vs the legacy engine, which handled it correctly and had a test for it (CodeGenerationXTest.testArithmetics, removed in this PR).
Confirm intended: == / != now compile to Objects.equals(...) (value equality) rather than Java == (identity).

Could you attach the before/after diff of regenerating the ASMD .scope/.export sources? That's the only artifact that actually exercises these paths, and it would let me confirm the fixes land. One concrete question: do any real ASMD sources use -/*// arithmetic, or ==/!= on non-null object/String operands?

_{🤖 Surfaced via an adversarial multi-agent review and manually verified against the diff. — Claude Code}

joaodinissf · 2026-06-17T11:01:56Z

+          null
+        }
+      }
+      default:


Blocker (regression): this resolveType switch has no IntegerLiteral / RealLiteral case, so a numeric literal resolves to null. That makes isNumber (ScopeExpressionCompiler.xtend:343) return false for literals, so isArithmeticOperatorCall (:372) fails and - / * / / fall through to the final else (ScopeExpressionCompiler.xtend:276), emitting the operator as a method call: a - 1 → -(a, 1), count * 2 → *(count, 2) — non-compilable Java. It triggers unconditionally for -/*// (even 4 - 2), since arithmetic is never routed through translate() (see the doc-comment at :416).

This is a regression, not a faithful carryover: legacy isNumber (ExpressionExtensionsX.xtend:44, findType('Real').isAssignableFrom(analyze(it))) typed literals as numeric, so the arithmetic branch fired and produced a - 1. The removed CodeGenerationXTest.testArithmetics asserted exactly this.

Fix: add IntegerLiteral / RealLiteral cases to resolveType returning the numeric builtin (this also fixes the +-parenthesization comment below).

_{🤖 Adversarial review + manual verification. — Claude Code}

joaodinissf · 2026-06-17T11:01:56Z

+          null
+        }
+      }
+      default:


Same blocker as in the scope translator: no IntegerLiteral / RealLiteral case here either, so ExportExpressionCompiler emits -(a, 1) for -/*//. Same fix applies. (Confirmed the scope and export engines are otherwise consistent.)

_{🤖 Adversarial review + manual verification. — Claude Code}

joaodinissf · 2026-06-17T11:01:56Z

+      case '==': toBinaryOperation(xLeft, xRight, ObjectExtensions, 'operator_equals', sourceElement)
+      case '!=': toBinaryOperation(xLeft, xRight, ObjectExtensions, 'operator_notEquals', sourceElement)


Confirm this is intended: linking ==/!= to ObjectExtensions.operator_equals makes Xbase inline them to java.util.Objects.equals(a, b) (null-safe value equality), whereas the legacy generator emitted raw Java == (identity for objects, primitive compare for numbers). For non-null object/String operands this is a behavioral flip; numeric operands also pick up boxing (NaN/-0.0 edge cases). Compilable, so CI won't catch it — just want to make sure the semantics change is deliberate.

_{🤖 Adversarial review + manual verification. — Claude Code}

joaodinissf · 2026-06-17T11:01:56Z

+      (if (target !== null) target.javaExpression(ctx) + '.' else '') + name + '(' + ', '.join(params.map[javaExpression(ctx)]) + ')'
+    } else if (isArithmeticOperatorCall(ctx)) {
+      autoBracket((' ' + name + ' ').join(params.map(e|e.javaExpression(ctx))), ctx)
+    } else if (isSimpleConcatCall()) {


Minor, and fixed by the same resolveType change as the arithmetic blocker: because numeric literals miss the arithmetic branch, + is handled here as concatenation without autoBracket. Top-level output matches legacy, but a + nested under a prefix operator loses its grouping, e.g. -(1 + 2) → -1 + 2. Adding the literal cases to resolveType restores the bracketed arithmetic branch.

_{🤖 Adversarial review + manual verification. — Claude Code}

joaodinissf · 2026-06-17T11:01:56Z

+  def dispatch factoryArgument(Expression it) {
+    error('unsupported scope factory argument ' + it.toString())


Intended narrowing? The legacy generator emitted full javaExpression(param) for factory arguments; this version supports only literals / type-refs / context vars and error(...)s on anything else. It fails loud at generation time rather than miscompiling, so low-risk — just flagging in case ASMD factory calls pass arbitrary expressions.

_{🤖 Adversarial review + manual verification. — Claude Code}

joaodinissf · 2026-06-17T11:01:56Z

+    }
+    String firstSegment = segments.getFirst();
+    if (segments.size() == 1) {
+      return findClassifier(firstSegment);


The new resolver is a bit more permissive than the legacy EmfRegistryMetaModel path: an unqualified single segment now resolves to a classifier (:69), and an unknown first segment falls back to a global last-segment search (:86), where legacy returned the raw name. Only diverges on names that were already unqualified/unknown, so low-risk — worth a glance in the ASMD regen diff. (Same applies to the export twin.)

_{🤖 Adversarial review + manual verification. — Claude Code}

joaodinissf · 2026-06-17T11:01:56Z

  // org.eclipse.osgi needed for NLS
  // org.apache.logging.log4j needed for logging in generated StandaloneSetup
-  private static final List<String> REQUIRED_BUNDLES = newArrayList("org.eclipse.xtext.xbase.lib", "org.eclipse.xtend.lib", "org.eclipse.emf.ecore", "com.avaloq.tools.ddk.check.core", "com.avaloq.tools.ddk.check.runtime.core", "com.avaloq.tools.ddk.check.lib", "com.avaloq.tools.ddk.xtext", "org.eclipse.xtext", "org.eclipse.osgi", "org.eclipse.xtend", "org.eclipse.core.runtime", "org.eclipse.xtext.xbase", "org.apache.logging.log4j.api");
+  private static final List<String> REQUIRED_BUNDLES = newArrayList("org.eclipse.xtext.xbase.lib", "org.eclipse.xtend.lib", "org.eclipse.emf.ecore", "com.avaloq.tools.ddk.check.core", "com.avaloq.tools.ddk.check.runtime.core", "com.avaloq.tools.ddk.check.lib", "com.avaloq.tools.ddk.xtext", "org.eclipse.xtext", "org.eclipse.osgi", "org.eclipse.xtend", "org.eclipse.core.runtime", "org.eclipse.xtext.xbase");


Looks unrelated to the scope/export migration: this drops "org.apache.logging.log4j.api" from REQUIRED_BUNDLES, while the comment right above still says log4j is needed for the generated StandaloneSetup. Could you justify or revert — and if it's intended, update that comment so it's not self-contradicting?

_{🤖 Adversarial review + manual verification. — Claude Code}

I can revert part of this PR once this once is merged.

The com.avaloq.tools.ddk.xtext.scope plugin compiled the embedded Expression DSL to Java through expression.generator.CodeGenerationX / CompilationContext, whose type system is the legacy classic Xtend runtime (org.eclipse.xtend.expression.*, org.eclipse.xtend.typesystem.emf.*). That was the only reason the plugin required those bundles. This PR replaces the IGenerator2-based ScopeGenerator with an Xbase JvmModelInferrer, introduces a self-contained (Xtend-free) expression-translation layer, deletes the classic-Xtend execution context, and removes org.eclipse.xtend and org.eclipse.xtend.typesystem.emf from the scope plugin. Key decisions Scope generation moves to a JvmModelInferrer + Xbase JvmModelGenerator Same pattern as Format/Check; lets the standard XbaseCompiler emit the provider classes. The *ScopeProvider/*ScopeNameProvider are now inferred JVM types rather than hand-templated .java. Hybrid expression backend: an Xbase translator + a string compiler fallback String + concatenation (very common in .scope) has no operator_plus in xbase.lib — Xbase special-cases it in its compiler — so it can't be pre-linked as an XExpression tree. We translate to Xbase what maps cleanly and fall back to a faithful port of the legacy string output for +, arithmetic and relational operators. Model-type names resolve via imported EPackages, not the classpath In real sources, (Cast)x / typeSelect(T) / T.isInstance(x) name EMF model types (FormDef, ILogicalTable, intfdef::AlternatingGroup), not Java FQNs. javaType resolves them through the model's imported EPackages → GenModelUtilX.instanceClassName, exactly as the legacy EmfRegistryMetaModel did. findDeclaredType would fail on these. .ext (JAVA extension) support dropped Scope sources no longer reference Xtend extension files; removed the .ext-reading validation and the isJavaExtensionCall/isExtension generator branches. factory becomes a direct Xbase static call Must be written Type.method(args); the generator resolves the type to a JvmDeclaredType.qualifiedName and emits the static call. Bare factory foo() is no longer supported. Downstream projects need to adapt their .scope/.ext sources.

generated code is identical

…ed work This PR is the migration tracker: as per-module migrations land on master through dedicated PRs, their counterparts here are dropped, shrinking this PR until it is empty and can be closed. Dropped in this rebase (master wins): - com.avaloq.tools.ddk.check.core/**: migration fully landed on master (fd18e86..30ff3e0 polish series; no .xtend remains there) - com.avaloq.tools.ddk.xtext.{export,expression,scope}/**: superseded by the dsldevkit#1405 Xbase rework (2973c5b); the conversions here predated it - ddk-parent/pom.xml: global xtend-maven-plugin removal was premature (master still carries .xtend sources) Kept (not yet landed via dedicated PRs): xtext.test.core Tag conversion (pending dsldevkit#1423), .ide/.ui/generator Xtend build-infra cleanups, migration docs and configuration. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…rt expression translation Follow-up to dsldevkit#1405 (does NOT touch its commits): restores two behaviours the Xbase migration silently changed in the scope/export expression code generators. 1. ==/!= reference equality. The translator linked == / != to ObjectExtensions.operator_equals/notEquals, which Xbase inlines to Objects.equals(a, b) (null-safe VALUE equality). The legacy generator - and this PR's own string fallback compiler - emit Java reference equality (a == b). Since the inferrer prefers the translated XExpression, the value-equality path won, silently changing cross-reference resolution semantics for EObject operands. Fix: leave == / != untranslated so they fall through to the faithful string compiler (raw a == b). The relational operators < > <= >= were already left untranslated, so this also makes the operator family consistent. 2. Nested arithmetic. isNumber() delegated to resolveType(), which returns null for an OperationCall operand, so a nested arithmetic operand (e.g. the (a + b) in (a + b) * c) was not "numeric"; the outer operator then missed the arithmetic branch and was emitted as operator-as-method, e.g. *(a + b, c) - non-compilable Java. Fix: isNumber() now recognises an arithmetic OperationCall as numeric, restoring (a + b) * c. Applied symmetrically to scope and export; removes the now-unused ObjectExtensions import. A scope.test module with a CodeGenerationXTest-style regression test (re-introducing the coverage dsldevkit#1405 dropped) follows in the next commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

… pass ScopeExpressionTranslator and ExportExpressionTranslator built a new Scope/ExportModelTypeResolver for every compilation context, i.e. for nearly every expression rendered into a generated method body. Each construction queries the IMPORT__PACKAGE scope and EcoreUtil.resolves every EPackage visible to the model (the EMF registry plus every EPackage in the index), so the cost was O(visible EPackages) per expression. The pre-dsldevkit#1405 generator built one CompilationContext per generated file. GeneratorSupport gains memoize(key, supplier): values memoized by the innermost executeWithProjectResourceLoader call on the current thread, discarded when that call returns; outside of such a call nothing is memoized. The translators memoize the resolver under a key holding the model. Equivalence: a resolver is immutable once constructed and is fully determined by its model and by the EPackages visible to that model, i.e. by the index and the resource set. Neither changes while one body is rendered, and EcoreUtil.resolve returns the same (already loaded) instances when it runs again, so a resolver built by the first compilation context is equal to the one each later context would have built. Lifetime: the memoized values live on a ThreadLocal of the executeWithProjectResourceLoader call and are dropped in its finally block. Today every method body is rendered inside its own such call, so a resolver is shared by the compilation contexts of one body and never outlives it - never across resources or builds, where the index and the classpath may change. Compilation contexts created outside such a call (tests, inference) still build a fresh resolver each time. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

… pass ScopeExpressionTranslator and ExportExpressionTranslator built a new Scope/ExportModelTypeResolver for every compilation context, i.e. for nearly every expression rendered into a generated method body. Each construction queries the IMPORT__PACKAGE scope and EcoreUtil.resolves every EPackage visible to the model (the EMF registry plus every EPackage in the index), so the cost was O(visible EPackages) per expression. The pre-#1405 generator built one CompilationContext per generated file. GeneratorSupport gains memoize(key, supplier): values memoized by the innermost executeWithProjectResourceLoader call on the current thread, discarded when that call returns; outside of such a call nothing is memoized. The translators memoize the resolver under a key holding the model. Equivalence: a resolver is immutable once constructed and is fully determined by its model and by the EPackages visible to that model, i.e. by the index and the resource set. Neither changes while one body is rendered, and EcoreUtil.resolve returns the same (already loaded) instances when it runs again, so a resolver built by the first compilation context is equal to the one each later context would have built. Lifetime: the memoized values live on a ThreadLocal of the executeWithProjectResourceLoader call and are dropped in its finally block. Today every method body is rendered inside its own such call, so a resolver is shared by the compilation contexts of one body and never outlives it - never across resources or builds, where the index and the classpath may change. Compilation contexts created outside such a call (tests, inference) still build a fresh resolver each time. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

rubenporras marked this pull request as draft June 10, 2026 05:10

rubenporras force-pushed the scope_and_export_migration_to_xbase branch 9 times, most recently from 61fa966 to 6b0bcb2 Compare June 17, 2026 07:20

rubenporras marked this pull request as ready for review June 17, 2026 07:23

rubenporras requested a review from joaodinissf June 17, 2026 07:23

rubenporras changed the title ~~Migrate the Scope generator off classic Xtend onto Xbase +~~ Migrate the Scope generator off classic Xtend onto Xbase + JvmModelInferrer Jun 17, 2026

rubenporras enabled auto-merge (squash) June 17, 2026 08:15

rubenporras force-pushed the scope_and_export_migration_to_xbase branch from 6b0bcb2 to 633c4aa Compare June 17, 2026 08:22

joaodinissf previously requested changes Jun 17, 2026

View reviewed changes

rubenporras force-pushed the scope_and_export_migration_to_xbase branch 2 times, most recently from 343963b to 252d2f2 Compare June 18, 2026 12:03

joaodinissf mentioned this pull request Jun 18, 2026

fix: restore scope and export expression code generation #1436

Merged

rubenporras force-pushed the scope_and_export_migration_to_xbase branch 2 times, most recently from 1b7ddcb to f5898b3 Compare June 25, 2026 07:43

joaodinissf mentioned this pull request Jul 19, 2026

refactor: migrate from org.eclipse.xtend to local type system #1277

Closed

2 of 3 tasks

rubenporras force-pushed the scope_and_export_migration_to_xbase branch 7 times, most recently from 66decfc to 660bd19 Compare August 19, 2026 05:46

rubenporras requested a review from gregdyke August 19, 2026 08:02

rubenporras requested a review from andrewL-avlq August 19, 2026 08:02

rubenporras force-pushed the scope_and_export_migration_to_xbase branch from 660bd19 to ff7e96c Compare August 19, 2026 08:45

gregdyke previously approved these changes Aug 19, 2026

View reviewed changes

rubenporras dismissed gregdyke’s stale review via 93aa2c5 August 19, 2026 09:55

rubenporras force-pushed the scope_and_export_migration_to_xbase branch from ff7e96c to 93aa2c5 Compare August 19, 2026 09:55

rubenporras requested a review from joaodinissf August 19, 2026 10:01

rubenporras requested a review from gregdyke August 19, 2026 10:09

gregdyke approved these changes Aug 19, 2026

View reviewed changes

rubenporras merged commit 2973c5b into dsldevkit:master Aug 19, 2026
4 checks passed

rubenporras deleted the scope_and_export_migration_to_xbase branch August 19, 2026 10:39

Navigation Menu

Repository navigation

Migrate the Scope generator off classic Xtend onto Xbase + JvmModelInferrer - #1405

Migrate the Scope generator off classic Xtend onto Xbase + JvmModelInferrer#1405
rubenporras merged 1 commit into
dsldevkit:masterfrom
rubenporras:scope_and_export_migration_to_xbase

rubenporras commented Jun 10, 2026 •

edited

Loading

Uh oh!

joaodinissf left a comment

Uh oh!

joaodinissf Jun 17, 2026

Uh oh!

joaodinissf Jun 17, 2026

Uh oh!

joaodinissf Jun 17, 2026

Uh oh!

joaodinissf Jun 17, 2026

Uh oh!

joaodinissf Jun 17, 2026

Uh oh!

joaodinissf Jun 17, 2026

Uh oh!

joaodinissf Jun 17, 2026

Uh oh!

rubenporras Jun 17, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		case '==': toBinaryOperation(xLeft, xRight, ObjectExtensions, 'operator_equals', sourceElement)
		case '!=': toBinaryOperation(xLeft, xRight, ObjectExtensions, 'operator_notEquals', sourceElement)

		def dispatch factoryArgument(Expression it) {
		error('unsupported scope factory argument ' + it.toString())

Conversation

rubenporras commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joaodinissf left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rubenporras commented Jun 10, 2026 •

edited

Loading