Add initial AdvPL/TLPP language support for TOTVS Protheus

[TonelliMG]

Summary

I would like to add initial support for AdvPL and TLPP source code used by the TOTVS Protheus ERP ecosystem.

Graphify currently does not classify the main source and header extensions used by these languages. As a result, repositories containing Protheus source code are skipped during code extraction and produce an empty graph.

The proposed implementation would introduce a dedicated deterministic extractor, following the same general approach currently used by the Dart extractor.

Background

AdvPL, or Advanced Protheus Language, is an xBase-based programming language created by TOTVS for developing ERP applications in the TOTVS Protheus ecosystem.

AdvPL supports procedural and object-oriented programming. Its source code is compiled and executed by the TOTVS Application Server.

TLPP, or TOTVS Language Plus Plus, is an evolution of AdvPL that introduces additional language capabilities while remaining part of the same development ecosystem.

Official references:

• [TOTVS TDN — AdvPL](https://tdn.totvs.com/display/tec/AdvPL)
• [TOTVS TDN — TLPP](https://tdn.totvs.com/display/tec/TLPP)

Current behavior

Running Graphify against a Protheus repository:

graphify extract . --code-only

currently produces output similar to:

found 0 code, 0 docs, 0 papers, 0 images
file(s) not classified, skipped: APWEBEX.CH, Auth2Fireb.prw, CSAA100VLD.prw, ...
graph is empty — extraction produced no nodes

The repository may contain thousands of valid AdvPL/TLPP source files, but they are currently treated as unsupported files.

Proposed implementation

Add a dedicated extractor:

graphify/extractors/advpl.py

The extractor would use deterministic textual and structural analysis, similar in principle to the existing Dart extractor.

The initial implementation would not require a Tree-sitter grammar.

It would:

1. read the source file;
2. support UTF-8, UTF-8 with BOM, and Windows-1252 source encoding;
3. mask comments and string contents where necessary;
4. identify supported declarations;
5. determine basic declaration scopes;
6. identify direct calls and include relationships;
7. return Graphify-compatible nodes and edges.

AdvPL and TLPP would be handled by the same extractor because TLPP is an evolution of AdvPL and both languages coexist within the TOTVS Protheus development ecosystem.

The implementation would be submitted as a single pull request.

File extensions

The initial extractor would recognize:

• .prw — standard AdvPL source files;
• .tlpp — TLPP source files;
• .ch — AdvPL header/include files;
• .th — TLPP header/include files.

According to the official TLPP documentation, .th files are TLPP header files and are used similarly to AdvPL .ch files. The .ch extension remains supported in TLPP projects.

Header files would be modeled as includes rather than independent executable source units.

Initial supported structures

AdvPL functions

User Function ProcessOrder()
    ValidateOrder()
    SaveOrder()
Return

Static Function ValidateOrder()
Return .T.

Function SaveOrder()
Return

Expected concepts:

• source file;
• ProcessOrder();
• ValidateOrder();
• SaveOrder().

Expected relationships:

source file --contains--> ProcessOrder()
source file --contains--> ValidateOrder()
source file --contains--> SaveOrder()

ProcessOrder() --calls--> ValidateOrder()
ProcessOrder() --calls--> SaveOrder()

Includes

#Include "Protheus.ch"
#Include "FWMVCDef.ch"

Expected relationships:

source file --imports--> Protheus.ch
source file --imports--> FWMVCDef.ch

For TLPP:

#include "tlpp-core.th"

Expected relationship:

source file --imports--> tlpp-core.th

Basic classes and methods

The initial implementation would recognize common AdvPL/TLPP class and method declarations, including structures such as:

Class OrderService
    Data cBranch

    Method New()
    Method Save()
EndClass

Method New() Class OrderService
Return Self

Method Save() Class OrderService
Return .T.

Expected relationships:

source file --contains--> OrderService
OrderService --method--> New()
OrderService --method--> Save()

Basic inheritance relationships would be represented using the existing inherits relation when they can be identified reliably.

Graph relationships

The extractor would reuse existing Graphify relationship names:

• contains;
• calls;
• imports;
• method;
• inherits;
• references, where reliably identifiable.

No new AdvPL-specific relationship vocabulary is proposed for the initial implementation.

Header behavior

.ch and .th files are preprocessor header/include files within the Protheus development model.

The initial implementation would primarily use them as targets of imports relationships.

For example:

orders.prw --imports--> Protheus.ch
service.tlpp --imports--> tlpp-core.th

Extraction of constants, macros, and preprocessor declarations from headers may be added where it can be performed deterministically, but full macro expansion would not be part of the initial implementation.

Encoding requirements

Existing Protheus repositories frequently contain legacy Windows-1252 source files.

The extractor should support:

• UTF-8;
• UTF-8 with BOM;
• Windows-1252/CP1252.

Encoding support should be covered by automated tests using synthetic source fixtures.

False-positive prevention

The extractor must avoid creating declarations or call relationships from:

• commented code;
• strings;
• comments containing function declarations;
• strings containing call-like expressions;
• language keywords or control-flow commands that resemble function calls.

Example:

// User Function FakeFunction()

User Function RealFunction()
    cExample := "FakeCall()"
Return

This should create only the RealFunction() declaration and should not create FakeFunction() or FakeCall() nodes.

Initial limitations

The first implementation would intentionally not attempt to support:

• complete AdvPL or TLPP grammar parsing;
• macro expansion;
• conditional preprocessor evaluation;
• dynamic function-name resolution;
• ExecBlock target resolution;
• runtime reflection;
• complete variable or type inference;
• embedded SQL analysis;
• complete TOTVS framework or MVC semantic analysis;
• resolution of dynamically dispatched Protheus framework functions.

These limitations would be documented and could be addressed incrementally in future pull requests.

Tests

The pull request would include synthetic, non-proprietary fixtures covering:

• User Function;
• Static Function;
• regular Function;
• direct function calls;
• .ch includes;
• .th includes;
• basic classes;
• basic methods;
• basic inheritance;
• comments;
• strings;
• UTF-8;
• UTF-8 with BOM;
• Windows-1252;
• prevention of dangling edges;
• repository-level extraction using .prw and .tlpp files.

No proprietary TOTVS or company source code would be included.

Acceptance criteria

The implementation should be considered successful when:

• .prw files are classified as supported code;
• .tlpp files are classified as supported code;
• .ch and .th files are recognized as headers/includes;
• basic functions are represented as graph nodes;
• basic classes and methods are represented as graph nodes;
• direct calls generate calls relationships;
• includes generate imports relationships;
• comments and strings do not create false declarations or calls;
• UTF-8 and Windows-1252 files are processed successfully;
• extraction produces no dangling edges;
• graphify extract . --code-only generates a non-empty graph for a synthetic Protheus project;
• the existing Graphify test suite continues to pass.

Implementation scope

I intend to implement this as a single pull request because AdvPL and TLPP belong to the same TOTVS Protheus language ecosystem and will share the same extraction architecture.

The first version will focus on reliable structural extraction rather than complete language parsing.

[Huaian666]

nice work btw. grabbed a few ideas from the code for my own project (hope that's ok lol)