Skip to content

AI context check for ambiguous badwords - #2

Merged
raf-li merged 3 commits into
mainfrom
feature/ai-context-check
Oct 5, 2026
Merged

raf-li merged 3 commits into
mainfrom
feature/ai-context-check

Conversation

@raf-li

@raf-li raf-li commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

Why

"anjing tetanggaku suka berisik kalau malam" is about a dog; "anjing lah" is a curse. The word list counts both, so students get warned for talking about their pets (or a tahi lalat, or mata telanjang).

What

  • When every badword in a message is in AI_AMBIGUOUS_WORDS (animals, tahi, telanjang, seks, gay, ...), the bot asks an AI model on Cloudflare Workers AI whether it was an insult, in the background:
    • insult → counted as usual
    • ok → not counted
    • no answer (down, slow, busy) → not counted (agreed policy: better to miss a curse than warn someone about their dog)
  • Clear badwords (kontol, bangsat, ...), or a message mixing one with an ambiguous word, are still counted immediately, without the AI.
  • The bot never waits for the AI: the HTTP call runs on a background thread and results are applied on the main thread (the only one allowed to use the TeamTalk SDK).
  • Conversation context: a lone "anjing" after "aku punya binatang baru waaa" is the pet, so the bot waits 8 s and also sends up to AI_CONTEXT_MESSAGES (default 4) earlier messages from the last 2 minutes and up to 2 follow-ups, labelled only [same person] / [someone else] (per channel: everyone in it; per PM: the sender only, since the SDK does not say who a PM was for). Kept in memory only while the check is on; commands and answers to the bot's prompts (passwords) are never kept. AI_CONTEXT_MESSAGES=0 sends the flagged message alone.
  • Never nicknames, usernames or IPs. PMs included (agreed); mention it in the server rules.
  • Off by default. /abt ai on switches it on and is refused until AI_CLOUDFLARE_ACCOUNT_ID and AI_CLOUDFLARE_API_TOKEN are in config.json. Merging this changes nothing on the server until then.
  • /bwt <text> also shows the AI's verdict; tools/ai_eval.py scores the configured model on 38 labelled sentences (some with context) (dog vs curse, sate babi vs babi lu, mata telanjang vs harassment, ...).
  • Default model @cf/meta/llama-3.3-70b-instruct-fp8-fast (configurable via AI_MODEL). Setup: docs/ai.md.

Tested

  • Against a fake Workers AI server: request path (account, model), Authorization: Bearer, message layout, temperature: 0, answer parsing (JSON, extra text, bare word, errors), timeouts.
  • In the bot with the real TeamTalk DLL: AI off → counted immediately as before; AI ok/insult/failure/busy → not counted / counted / not counted / not counted; clear and mixed badwords counted immediately; /bwt verdict PM; /abt ai on refused without credentials.
  • Context: earlier message, someone else's question, follow-up, 4-message and 2-minute limits, PM sender only, passwords/commands never kept, nothing kept with the AI off or AI_CONTEXT_MESSAGES=0.
  • All earlier suites still pass.
  • Not yet run against the real Workers AI: run python3 tools/ai_eval.py on the server after adding the token.

🤖 Generated with Claude Code

raf-li and others added 2 commits October 5, 2026 00:41
"anjing tetanggaku berisik" is about a dog, "anjing lah" is a curse; a word
list cannot tell them apart.

- ai_review.py: Cloudflare Workers AI client on a background thread; results
  are handled on the main thread (the only one allowed to use the SDK)
- When every badword in a message is in AI_AMBIGUOUS_WORDS, the bot asks the
  AI first: insult -> counted, ok -> not counted, no answer -> not counted
  (agreed policy: better to miss a curse than warn a student about their dog)
- Clear badwords, or a mix with one, are still counted immediately
- Off by default; /abt ai switches it and refuses without credentials;
  /bwt shows the AI's verdict; tools/ai_eval.py scores a model on 32
  labelled sentences; setup in docs/ai.md
- Only the flagged message text is sent, never names or IPs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A lone "anjing" right after "aku punya binatang baru waaa" names the new
pet, but on its own the AI reads it as a curse.

- Recent messages are kept in memory (only while the AI check is on): per
  channel for everyone in it, per sender for private messages (the SDK does
  not say who a PM was for). Commands and answers to the bot's prompts,
  such as passwords, are never kept; old entries are dropped
- On an ambiguous hit the bot waits 8 s, then sends up to
  AI_CONTEXT_MESSAGES (default 4) earlier messages from the last 2 minutes
  and up to 2 follow-ups, labelled only [same person] / [someone else]
- AI_CONTEXT_MESSAGES=0 sends the flagged message alone, as before
- Prompt explains the context and treats a lone swear word as an
  exclamation; examples and tools/ai_eval.py (38 cases) cover it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@raf-li
raf-li merged commit b8ddd8a into main Oct 5, 2026
2 checks passed
@raf-li
raf-li deleted the feature/ai-context-check branch October 5, 2026 00:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant