Skip to content

Migrate xlnet to the semantic Task SDK #1537

Description

@yifeif-nv

Parent epic: #1430

Status: Not started — comment here before beginning work.

Family: families/xlnet/
Current declared task mode(s): embedding, encoding, reranking
Target semantic Task(s): Identify the existing semantic Task contract(s) that preserve every current capability; record the exact IDs in the PR.

Work

  • Inventory this family's primary and secondary behavior, including batch, streaming, raw outputs and stateful operations where supported. Record the current defaults, profiles, checkpoint cases, and reference criteria.
  • Implement and bind the matching semantic Task interface(s) in this family's runtime. Preserve its graph, preprocessing, inference, postprocessing, configuration and complete owned results.
  • Align family-owned support, build metadata and manifests with the primary Task. Advertise only interfaces implemented by the loaded bundle; document whether existing bundles need rebuilding.
  • Add family-owned CPU contracts and public C/C++ SDK consumers. Cover invalid inputs, configuration handling, result lifetime and every relevant lifecycle.
  • Run a selected real checkpoint E2E with the existing oracle and thresholds, then check the remaining declared capabilities and public CLI, example and benchmark paths. Preserve meaningful passing criteria.

PR evidence

Link the migration PR here. Report the tested revision, exact commands and results, selected checkpoint cases, original reference checks, current-head CI, and any untested configurations. Keep a normal migration within families/xlnet/**; raise a separate linked prerequisite issue if a shared contract is genuinely missing. Do not weaken an acceptance criterion to pass CI.

References: #1226 · family validation guide.

Activity

  1. jkzhang7 commented on Oct 4, 2026

    @jkzhang7
    Contributor

    I'd like to work on this, under parent epic #1430.

    Scope understanding:

    • Family: families/xlnet/, declaring embedding/encoding/reranking through the same EncoderPipeline shape as distilbert (Migrate distilbert to the semantic Task SDK #1446, PR refactor(distilbert): migrate to Task SDK #1581, which carries the Task mapping and GPU evidence for this shape).
    • Mapping: embedding -> text_to_embedding, encoding -> text_to_pooled_features, reranking -> text_pair_to_relevance plus text_query_documents_to_relevance (to keep rerank_batch). The existing manifests and benchmark YAML only change their task field; the benchmark tooling already maps these Task IDs.
    • The migration stays inside families/xlnet/**. I plan to submit this family together with electra, mpnet and xlnet in one PR, each with its own contract test and real-checkpoint E2E, and will link that PR on each child issue.

    Starting now.

  2. jkzhang7 commented on Oct 4, 2026

    @jkzhang7
    Contributor

    Opened #1584 (draft; covers #1450, #1484 and #1537)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions