Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 131 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# Agent Development Guide

**Skyrim Together** (codename TiltedEvolution) is a multiplayer mod for Skyrim Special Edition: co-op
questing, combat and exploration for 2-8 players.

- Stack: C++20 / MSVC, xmake 3.x (deps pinned in the root `xmake.lua`, `Libraries/Tilted*` are
submodules). The UI is Angular (`Code/skyrim_ui`).
- Platforms: the client and launcher are Windows x64 only. The server also runs on Linux (amd64 and arm64),
so keep server and shared code portable.
- Game versions: SE **1.7.104** is the main target (`kSupportedGameVersions` in `Code/client/main.cpp`).
1.6.1170 is unofficially supported. Engine code and address IDs are effectively identical.
- Branches: open PRs against `dev`. `master` is the release branch.
- Logs: `build\windows\x64\<mode>\logs`.

## Architecture

**Client-authoritative, the server mostly relays.** Clients run the simulation. The server keeps a light world
model (players, parties, cells, which actors and objects exist, who owns them) and decides who receives what.
It never simulates Skyrim.

- The party leader effectively hosts the world. Quest progress, weather, time etc. come from the leader and
are applied on the members' clients.
- Every synced actor or object has one owner client that runs the real AI and physics and streams its state.
Other clients show a *remote* copy that replays it. Ownership moves when the owner leaves the cell or
disconnects. Players are always owned by their own client.
- Clients are trusted. Validate on the server only where it matters (parties, ownership, admin commands).

**Engine integration (not an SKSE plugin).** `SkyrimTogether.exe` (`Code/immersive_launcher`) loads
`SkyrimSE.exe` into its own process, with the client (`Code/client`) linked in. `Code/client/main.cpp` loads
the Address Library, checks the game version, installs hooks, then loads SKSE if present so that other SKSE
plugins keep working.

- Resolve game functions and globals through Address Library IDs, never raw addresses.
- Hooks are MinHook detours added with `TP_HOOK` in a static `TiltedPhoques::Initializer` at the bottom of the
engine file, calling the original through a saved pointer. Copy `Code/client/Games/Skyrim/Actor.cpp`.
- We don't use CommonLibSSE. Engine types are our own reversed headers in `Code/client/Games/Skyrim/**`
(`Actor`, `TESObjectREFR`, `ExtraData`, `Havok`...), with layouts pinned by `static_assert(offsetof(...))`/
`sizeof`. Add members and asserts as needed. A local CommonLibSSE checkout is fine as a reference; in its
`Offsets*.h` the 1.6.x IDs are in the first `#ifdef SKYRIM_SUPPORT_AE` block.
- `GameFiles/Skyrim` holds what ships alongside the binaries: the ESPs, Papyrus scripts, meshes and behavior
files.

**Data flow.**
1. Upstream: an engine hook or per-frame service poll notices a change. Hooks often run on engine worker
threads, so queue the work onto the main thread through the runner service. A client service then sends a
request through the transport service.
2. Server: a service handles the request with the sending player, updates its world model and relays a
`Notify*` to everyone, players in range or the party, usually excluding the sender.
3. Downstream: a client service subscribed to the `Notify*` applies it to the game. While applying remote
changes, guard against local hooks echoing them back upstream (existing services show how).

**ECS (`entt`).** Client and server each have a `World` (an `entt::registry` subclass) and an `entt::dispatcher`.

- **Services** (`Code/{client,server}/Services`): long-lived singletons in `World::ctx()` holding feature
logic. They connect in their constructors (`aDispatcher.sink<T>().connect<&Svc::Fn>(this)`) and keep the
connections in `entt::scoped_connection` members.
- **Components** (`Code/{client,server}/Components`): plain data. Data that belongs to an actor or object
goes in a component on its entity. Use a service-side map keyed by form ID only if there's no other
option. On the client, `FormIdComponent` links an entity to its form and `LocalComponent`/`RemoteComponent`
mark the owner side.
- **Systems** (`Code/client/Systems`): near-stateless helpers over components (interpolation, animation...).
- **Events** (`Code/{client,server}/Events`): small structs passed through the dispatcher.

## Network protocol

- Transport is `Libraries/TiltedConnect` (Valve GameNetworkingSockets). All messages live in `Code/encoding`
(`SkyrimEncoding`), shared by client and server. New messages follow the existing ones' naming, structure
and registration.
- Reuse payload types from `Code/encoding/Structs`: `GameId` (`ModId` + `BaseId`, **never raw runtime form
IDs**), `Vector3_NetQuantize`, `Rotator2_NetQuantize`.
- **No backward compatibility:** the client sends `BUILD_COMMIT` in `AuthenticationRequest` and the server
rejects any mismatch (`kWrongVersion`), so reorder or change opcodes and fields freely.
- Watch bandwidth. Sends are reliable by default; send high-rate, loss-tolerant data (movement) unreliably.

## Building

From the repo root (VS 2022 developer shell or PowerShell with MSVC):

```sh
xmake f -m releasedbg -y # configure (debug | releasedbg | release); first run fetches deps
xmake -y # build everything
xmake build SkyrimTogetherClient # one target; also SkyrimImmersiveLauncher, SkyrimTogetherServer
xmake build TPTests && xmake run TPTests # main unit tests (Catch2)
xmake project -k vsxmake # regenerate the VS solution; rerun after adding/removing files
```

- The solution is `vsxmake2022/<repo dir>.sln`. Most contributors build and debug from it; agents don't need it.
- UI: `cd Code/skyrim_ui && pnpm install && pnpm deploy:production`. `xmake install -o distrib` gathers
binaries.
- Main targets:
- `SkyrimImmersiveLauncher` → `SkyrimTogether.exe`, with the `SkyrimTogetherClient` static lib linked in
- `SkyrimTogetherServer` → `STServer.dll`, run by `SkyrimServerRunner` → `SkyrimTogetherServer.exe`
- `ImmersiveElf` → `EarlyLoad.dll` (launcher early loader), `TPProcess` (UI overlay helper process)
- Shared static libs: `SkyrimEncoding`, `CommonLib`, `BaseLib`
- Tests: `TPTests`, plus `Console_Tests`, `ESLoader_Tests` and `Resources_Tests` for the component libs
in `Code/components`

## Testing

- **Most gameplay features can only be verified by hand:** run a local server plus two or more game clients
on one machine and exercise the feature from both sides. Say plainly when a change still needs an
in-game test. Compiling is not proof that it works.
- Add unit tests only for mission-critical, Skyrim-independent code: protocol encoding, timing and
interpolation math, pure data structures. Such code can be compiled into `TPTests`. Ask before writing
tests for gameplay code.

## Reverse engineering

- **IMPORTANT: if a task depends on engine behavior the source doesn't show (hook points, struct layouts,
vtable slots, threading) and no IDA or Ghidra MCP tools are available, stop before implementing.**
Say what needs verifying in the binary, warn that a source-only version may be wrong or suboptimal (guessed
offsets, wrong hook point, missed side effects), and ask whether to go ahead.
- With an MCP, check the Skyrim SE 1.6.1170 database first. Use 1.7.104 only when something differs.
- Map Address Library IDs to IDB addresses and back with `python Tools/Scripts/addrlib.py <decimal id | IDB address>...`
(1.6.1170 by default, `-v 1.7.104` for another version; usage at the top of the script).
- `Tools/Scripts` has other helpers for addresses, RTTI dumps and IDB matching.

## Conventions

- **Match the surrounding style.** Hungarian-style prefixes where the file uses them: `a` arguments, `c` const,
`p` pointers, combined as needed (`acEvent`, `apActor`). No `r` prefix for references.
- Members use `m_` (plain-data structs use bare `PascalCase` fields). Statics use `s_`, globals `g_`; hook originals are
`Real<Name>`, constants `k<Name>`. Types and functions are `PascalCase`, and most functions are `noexcept`.
Reversed engine types keep the game's names (`hCaster`, `slotType`).
- No exceptions and no throwing STL calls. No `auto` for primitive types.
- Use `TiltedPhoques` containers (`String`, `Vector`, `UniquePtr`...) where the surrounding code does. Log
with `spdlog`.
- Gate debug-only UI and behavior with `IS_MASTER` / `DEBUG`. Mod compatibility shims live in
`Code/client/ModCompat`.
- Commits: short imperative title describing the user-visible effect ("Fix X when Y", "Stop X doing Y"), and
a short body explaining why.
9 changes: 1 addition & 8 deletions CODE_GUIDELINES.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Language

We are using C++20, any C++20 feature supported by vs2019 is allowed.
We are using C++20, any C++20 feature supported by vs2022 is allowed.

Please try to use templates responsibly, we don't want compilation times to explode and to deal with bloated binaries.

Expand Down Expand Up @@ -45,10 +45,3 @@ Names must be self explanatory, ``size_t a;`` is not acceptable, ``size_t incomi
Don't use java style blocks, a ``{`` needs to be on a new line.

Don't use exceptions, don't use STL code that can throw, use the nothrow version if available or the unsafe version.

## Commit Naming
Preface commit messages with one of the following tags:
* `feat:` for new features
* `tweak:` for tweaks to an existing system
* `fix:` for bug fixes and crash fixes
* `refactor:` for refactoring
35 changes: 21 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
# Tilted Online
# Skyrim Together
#### codename "TiltedEvolution"
![Build status](https://github.com/tiltedphoques/TiltedEvolution/workflows/Build%20windows/badge.svg?branch=master) [![Build linux](https://github.com/tiltedphoques/TiltedEvolution/actions/workflows/linux.yml/badge.svg)](https://github.com/tiltedphoques/TiltedEvolution/actions/workflows/linux.yml) [![Discord](https://img.shields.io/discord/247835175860305931.svg?label=&logo=discord&logoColor=ffffff&color=7389D8&labelColor=6A7EC2)](https://discord.gg/skyrimtogether)

<img src="https://avatars.githubusercontent.com/u/52131158?s=200&v=4" align="right"
alt="Size Limit logo by Anton Lovchikov" width="110" height="100">

Tilted Online is a framework created to enable multiplayer in Bethesda games, currently supporting **Skyrim Special Edition**.
TiltedEvolution is a framework created to enable multiplayer in Bethesda games, currently supporting **Skyrim Special Edition**.

## Getting started
To play Tilted Online, go to the [nexus page](https://www.nexusmods.com/skyrimspecialedition/mods/69993).
To play Skyrim Together, go to the [nexus page](https://www.nexusmods.com/skyrimspecialedition/mods/69993).

For general information, go to the [Tilted Online Wiki](https://wiki.tiltedphoques.com/tilted-online/).
For general information, go to the [TiltedEvolution Wiki](https://wiki.tiltedphoques.com/tilted-online/).

Check out the [build guide](https://wiki.tiltedphoques.com/tilted-online/technical-documentation/build-guide) for setup and development info on the project. When writing code, check the CODE_GUIDELINES.md and make sure to run clang-format!

Expand All @@ -20,24 +21,30 @@ If you would like to report a bug please report them in the "Issues" tab on this
Have some experience in C++, and want to help advance the project faster? Contribute!
- Check the issues, it's a good place to start when you don't know what to do.
- Fork the repository and create pull requests to this repository.
- Create pull requests to the "dev" branch, not to master or prerel.
- Create pull requests to the "dev" branch, not to master.
- Try to keep your code clean, following the code guidelines.

## Main project source tree

* [**client/**](./Code/client): Sources for the SkyrimSE and FO4 clients.
* [**immersive_launcher/**](./Code/immersive_launcher): Game starter/updater.
* [**common/**](./Code/common): Common code shared between plugin and server.
* [**encoding/**](./Code/encoding): Net-message definitions.
* [**server/**](./Code/server): GameServer implementation.
* [**skyrim_ui/**](./Code/skyrim_ui): Source code for the ui, written in typescript.
* [**tests/**](./Code/tests): Tests for the encoding and serialization code.
* [**tp_process/**](./Code/tp_process): Worker for CEF (Chromium Embedded Framework) overlay.
* [**client/**](./Code/client): Skyrim SE client: engine hooks, reverse-engineered game types and the sync logic.
* [**immersive_launcher/**](./Code/immersive_launcher): `SkyrimTogether.exe`, which starts the game with the client loaded.
* [**immersive_elf/**](./Code/immersive_elf): `EarlyLoad.dll` helper DLL for the launcher. Detects if Mod Organizer 2 was injected first.
* [**tp_process/**](./Code/tp_process): Worker for the CEF (Chromium Embedded Framework) overlay.
* [**skyrim_ui/**](./Code/skyrim_ui): In-game UI, written in Angular/TypeScript.
* [**server/**](./Code/server): Game server (`STServer.dll`).
* [**server_runner/**](./Code/server_runner): Dedicated server executable that hosts the server DLL.
* [**encoding/**](./Code/encoding): Network message and struct definitions shared by the client and server.
* [**common/**](./Code/common), [**base/**](./Code/base): Shared utilities.
* [**components/**](./Code/components): Shared modules (console/settings, ESP loader, resources, ImGui).
* [**tests/**](./Code/tests): Unit tests.
* [**GameFiles/**](./GameFiles): Plugins, scripts and assets shipped with the mod.
* [**Libraries/**](./Libraries): Networking, hooking, reverse-engineering and UI libraries (git submodules).
* [**Tools/**](./Tools): Reverse-engineering and address scripts.

## License
[![GNU GPLv3 Image](https://www.gnu.org/graphics/gplv3-127x51.png)](http://www.gnu.org/licenses/gpl-3.0.en.html)

Tilted Online is free software: you can redistribute it and/or modify
TiltedEvolution is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
43 changes: 0 additions & 43 deletions Tools/Scripts/address_patcher.py

This file was deleted.

107 changes: 107 additions & 0 deletions Tools/Scripts/addrlib.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
"""Map Skyrim Address Library IDs <-> addresses (format follows Code/client/VersionDb.h).

python Tools/Scripts/addrlib.py 39004 0x1406cf760 ID -> address, address -> ID
python Tools/Scripts/addrlib.py -v 1.7.104 39004 other game version (default 1.6.1170, the main IDB)

Addresses are 0x140000000 + offset, as in the IDBs; the 0x is optional for 9+ digit addresses.
An address inside a function or object resolves to the closest ID below it, e.g. "39004+0x1a".
-v also takes a path to a .bin. Bins are read from <game>/Data/SKSE/Plugins; the game dir comes
from SKYRIM_SE_DIR or the Steam registry key.
"""

import bisect
import os
import struct
import sys

BASE = 0x140000000

# Format 2 delta codes: kind -> (stored value, sign). Sign 0 means the value replaces the
# previous one, otherwise it's added/subtracted. Kind 1 is "previous + 1" with nothing stored.
DELTAS = {0: ("<Q", 0), 2: ("<B", 1), 3: ("<B", -1), 4: ("<H", 1), 5: ("<H", -1), 6: ("<H", 0), 7: ("<I", 0)}


def find_bin(version):
if version.endswith(".bin"):
return version
game = os.environ.get("SKYRIM_SE_DIR")
if not game:
try:
import winreg
key = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, r"SOFTWARE\WOW6432Node\Bethesda Softworks\Skyrim Special Edition")
game = winreg.QueryValueEx(key, "installed path")[0]
except (ImportError, OSError):
sys.exit("Skyrim SE not found: set SKYRIM_SE_DIR or pass -v <path to .bin>")
parts = (version.split(".") + ["0"] * 4)[:4]
path = os.path.join(game, "Data", "SKSE", "Plugins", f"versionlib-{'-'.join(parts)}.bin")
if not os.path.isfile(path):
sys.exit(f"{path} not found (is Address Library for {version} installed?)")
return path


def delta(d, p, kind, prev):
"""Decodes one format 2 value at d[p]. Returns (value, next p)."""
if kind == 1:
return prev + 1, p
fmt, sign = DELTAS[kind]
v = struct.unpack_from(fmt, d, p)[0]
return prev + sign * v if sign else v, p + struct.calcsize(fmt)


def load(path):
"""Returns {id: offset}."""
with open(path, "rb") as f:
d = f.read()
fmt = struct.unpack_from("<i", d)[0] # then 4 x i32 game version

if fmt == 5: # char name[64], u64 ptr size, u32 count, then u32 offset per ID (0 = none)
count = struct.unpack_from("<I", d, 92)[0]
return {i: o for i, o in enumerate(struct.unpack_from(f"<{count}I", d, 96)) if o}

if fmt != 2:
sys.exit(f"unsupported format {fmt}")

# i32 name length + name, i32 ptr size, i32 count, then records: a type byte whose low
# nibble is the ID's delta kind and high nibble the offset's. High bit 8 means the offset
# delta is in units of ptr size.
p = 24 + struct.unpack_from("<i", d, 20)[0]
ptr, count = struct.unpack_from("<ii", d, p)
p += 8
out = {}
vid = off = 0
for _ in range(count):
t = d[p]
vid, p = delta(d, p + 1, t & 0xF, vid)
scale = ptr if t & 0x80 else 1
off, p = delta(d, p, (t >> 4) & 7, off // scale)
off *= scale
out[vid] = off
return out


def main():
args = sys.argv[1:]
version = "1.6.1170"
if args[:1] == ["-v"]:
version, args = args[1], args[2:]
if not args or args[0] in ("-h", "--help"):
sys.exit(__doc__)
ids = load(find_bin(version))
rev = sorted((o, i) for i, o in ids.items())

for arg in args:
if not (arg.lower().startswith("0x") or len(arg) >= 9):
o = ids.get(int(arg))
print(f"{arg} -> {o + BASE:#x}" if o is not None else f"{arg} -> not found")
continue
target = int(arg, 16) - BASE
n = bisect.bisect_right(rev, (target, float("inf"))) - 1
if n < 0 or target - rev[n][0] > 0x1000000: # outside the image (largest real gap is 0x1000000)
print(f"{arg} -> not found")
else:
o, i = rev[n]
print(f"{arg} -> {i}" + (f"+{target - o:#x}" if target != o else ""))


if __name__ == "__main__":
main()
Loading