Train a Wordle agent: SFT, then GRPO

docs/source/_toctree.yml

@@ -48,7 +48,7 @@
     - local: tutorials/end-to-end-walkthrough
       title: Train a Reasoning Model
     - local: tutorials/wordle-grpo
-      title: Play Wordle with GRPO
+      title: Train a Wordle Agent
     - local: tutorials/rl-training-2048
       title: Play 2048 with GRPO
     - local: tutorials/sft-warmup

docs/source/environments/textarena.md

@@ -11,6 +11,7 @@ An OpenEnv wrapper for [TextArena](https://github.com/textarena/textarena) game
 > Generic wrapper for any [TextArena](https://www.textarena.ai/docs/overview) game inside OpenEnv. This module exposes the TextArena `Env` interface through the standard HTTP server/client APIs used by other OpenEnv environments, enabling quick experimentation with the full suite of word, reasoning, and multi-agent games.
 
 Try Wordle on the [`openenv/wordle`](https://huggingface.co/spaces/openenv/wordle) Space, or connect a client to `https://openenv-wordle.hf.space`.
+To train a model on it, follow the [Wordle tutorial](https://huggingface.co/docs/openenv/tutorials/wordle-grpo): an SFT warmup, then GRPO with TRL.
 
 ## Quick Start
 

docs/source/tutorials/index.md

@@ -3,7 +3,7 @@
 Choose a learning path from the sidebar:
 
 - **Basics:** start with [Hello World](openenv-tutorial), build [Your First Environment](../guides/first-environment) and [deploy it](../getting_started/environment-builder).
-- **Training:** [the ways to train and the supported frameworks](../guides/training), then train a [reasoning model](end-to-end-walkthrough), play [Wordle](wordle-grpo) or [2048](rl-training-2048) with GRPO, or [collect rollouts for SFT](sft-warmup). Each framework's own examples (ART, Miles, Oumi, SkyRL, torchforge, TRL, Unsloth, …) are listed in [Integrations](../guides/training#integrations).
+- **Training:** [the ways to train and the supported frameworks](../guides/training), then train a [reasoning model](end-to-end-walkthrough), a [Wordle agent](wordle-grpo) with SFT and GRPO, play [2048](rl-training-2048) with GRPO, or [collect rollouts for SFT](sft-warmup). Each framework's own examples (ART, Miles, Oumi, SkyRL, torchforge, TRL, Unsloth, …) are listed in [Integrations](../guides/training#integrations).
 - **Harnesses:** [pick a path](harnesses): train with the trainer's own loop on [BrowserGym](browsergym-harness) (white-box), train real agents through [Harbor](harbor-harness) (black-box), or [evaluate Claude Code inside an environment](claude-code-harness).
 - **Evals:** follow [Evaluating with Environments](evaluation-inspect).
 
@@ -33,7 +33,7 @@ Already familiar with the basics? These tutorials cover specific workflows in de
 | [Train a Reasoning Model](end-to-end-walkthrough) | The full pipeline: connect to `reasoning_gym`, wire it into TRL via `environment_factory`, fine-tune with GRPO, and push the checkpoint to the Hub. | Yes | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/OpenEnv/blob/main/examples/end_to_end_walkthrough.ipynb) |
 | [MCP Environments](mcp-environment) | Consume and build MCP-backed environments: list and call tools through `step()`, register Python functions as tools with FastMCP. | No | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/OpenEnv/blob/main/examples/mcp_environment.ipynb) |
 | [Rubrics](rubrics) | Compose reward functions from reusable pieces using `Gate`, `WeightedSum`, `LLMJudge`, and `TrajectoryRubric`. | No | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/OpenEnv/blob/main/examples/rubrics.ipynb) |
-| [Play Wordle with GRPO](wordle-grpo) | Train an agent to play Wordle using GRPO via TRL's `environment_factory`. | Yes | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/trl/blob/main/examples/grpo_wordle/grpo_wordle.ipynb) |
+| [Train a Wordle Agent](wordle-grpo) | Take Qwen3-1.7B from 0 to 19 wins out of 30: generate SFT data with a solver, run a short SFT warmup, then GRPO with TRL's `environment_factory`. Every step with its hardware, time and cost. | Yes | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openenv/OpenEnv/blob/main/examples/wordle/wordle.ipynb) |
 | [Play 2048 with GRPO](rl-training-2048) | Train a language model to play 2048 on the OpenSpiel environment with TRL's `GRPOTrainer` and `environment_factory`. Unsloth's own [2048 notebook](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/OpenEnv_gpt_oss_(20B)_Reinforcement_Learning_2048_Game.ipynb) trains `gpt-oss-20b` to write a 2048 strategy. | Yes | — |
 | [Evaluating with Environments](evaluation-inspect) | Wrap an OpenEnv environment in an Inspect AI `Task`, run it via `InspectAIHarness`, and get a structured `EvalResult`. | No | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/OpenEnv/blob/main/examples/evaluation_inspect.ipynb) |
 | [White-Box: Train a Web Agent on BrowserGym](browsergym-harness) | Train a vision-language model on BrowserGym web tasks with TRL's `GRPOTrainer` and `environment_factory`, so TRL runs the multi-turn tool loop. | Yes | — |