Discord REST client, cache and #225 settings (server-26#225 S2) #231

2026-10-05T01:58:10-04:00

logan commented

2026-10-05 01:58:10 -04:00

Slice S2 of #225 (plan: #225 comment 3108). Built on origin/main c4e1f9a (includes #227 multi-stream and #226). Do not merge until reviewed; S3 depends on it.

What changed

New app/internal/discord_rest.py:
- One shared httpx.AsyncClient, User-Agent DiscordBot (https://drb.cusano.net, 1). The token goes per request in Authorization; it is never logged or put in an error string, and rate-limit state is keyed by a hash of it.
- Per-token buckets keyed by X-RateLimit-Bucket + guild id. A wait over discord_rest_max_retry_after_seconds (3 s) is not sent: the result is unknown. A 429 is retried once only if retry_after is 3 s or less. A global 429 pauses that token only.
- Process-wide breaker: at 4000 responses of 401, 403 or non-shared 429 in 10 min, nothing more is sent and results are unknown until the window drains. This protects the egress IP the main bot shares.
- guild_snapshot(token_doc, guild_id): cached, single-flight read of /guilds/{g}, /guilds/{g}/members/{bot} and /guilds/{g}/channels, for S3's permission check. 401 sets valid=False. 10004 and 10007 drop the server from guild_ids. 50001 is confirmed by one read of the server list and never changes guild_ids on its own. Cache TTLs: ok 120 s, not-member or invalid 600 s, unknown 30 s. invalidate(token_id=, guild_id=) drops entries, and a load that overlaps an invalidate is not stored.
inspect_token, refresh_membership, leave_guild and TokenRejected moved off discord_leases.py onto the new client. Callers in discord_streams.py, routers/tokens.py and routers/discord.py were repointed. refresh_membership and leave_guild also drop the cache.
Pagination fix: /users/@me/guilds now follows after= pages. Before, it read one page of 200 and silently dropped the rest. A failed page now returns unknown, never a truncated list.
config.py has every setting the #225 slices use: discord_perm_check_enabled, discord_access_{,negative_,unknown_}ttl_seconds, discord_perm_check_concurrency / _budget_seconds, discord_rest_invalid_budget / _window_seconds, discord_rest_max_retry_after_seconds, discord_rest_timeout_seconds, discord_directory_refresh_seconds, discord_oauth_redirect_uri (empty means off), discord_oauth_state_ttl_seconds, discord_oauth_link_check_seconds, discord_pool_bot_guild_cap=90 (owner Q3).

Behaviour change: limited to pagination, the User-Agent, the rate-limit handling and the breaker. Nothing reads guild_snapshot until S3.

Tests: new tests/test_discord_rest.py has 29 tests covering hygiene, pagination, 429 and buckets, the breaker, snapshot status mapping, cache TTLs, single flight and invalidate races. The 4 discord test files now monkeypatch discord_rest.refresh_membership. The leave-guild test uses httpx.MockTransport.
~/venvs/drb-5c/bin/python -m pytest (drb-c2-core): 715 passed, 4 warnings (baseline on main: 686 passed).
No frontend files changed, so tsc was not run.

🤖 Generated with Claude Code

Slice **S2** of #225 (plan: #225 comment 3108). Built on `origin/main` c4e1f9a (includes #227 multi-stream and #226). Do not merge until reviewed; S3 depends on it. **What changed** - New `app/internal/discord_rest.py`: - One shared `httpx.AsyncClient`, User-Agent `DiscordBot (https://drb.cusano.net, 1)`. The token goes per request in `Authorization`; it is never logged or put in an error string, and rate-limit state is keyed by a hash of it. - Per-token buckets keyed by `X-RateLimit-Bucket` + guild id. A wait over `discord_rest_max_retry_after_seconds` (3 s) is not sent: the result is unknown. A 429 is retried once only if `retry_after` is 3 s or less. A global 429 pauses that token only. - Process-wide breaker: at 4000 responses of 401, 403 or non-shared 429 in 10 min, nothing more is sent and results are unknown until the window drains. This protects the egress IP the main bot shares. - `guild_snapshot(token_doc, guild_id)`: cached, single-flight read of `/guilds/{g}`, `/guilds/{g}/members/{bot}` and `/guilds/{g}/channels`, for S3's permission check. 401 sets `valid=False`. 10004 and 10007 drop the server from `guild_ids`. 50001 is confirmed by one read of the server list and never changes `guild_ids` on its own. Cache TTLs: ok 120 s, not-member or invalid 600 s, unknown 30 s. `invalidate(token_id=, guild_id=)` drops entries, and a load that overlaps an invalidate is not stored. - `inspect_token`, `refresh_membership`, `leave_guild` and `TokenRejected` moved off `discord_leases.py` onto the new client. Callers in `discord_streams.py`, `routers/tokens.py` and `routers/discord.py` were repointed. `refresh_membership` and `leave_guild` also drop the cache. - **Pagination fix:** `/users/@me/guilds` now follows `after=` pages. Before, it read one page of 200 and silently dropped the rest. A failed page now returns unknown, never a truncated list. - `config.py` has every setting the #225 slices use: `discord_perm_check_enabled`, `discord_access_{,negative_,unknown_}ttl_seconds`, `discord_perm_check_concurrency` / `_budget_seconds`, `discord_rest_invalid_budget` / `_window_seconds`, `discord_rest_max_retry_after_seconds`, `discord_rest_timeout_seconds`, `discord_directory_refresh_seconds`, `discord_oauth_redirect_uri` (empty means off), `discord_oauth_state_ttl_seconds`, `discord_oauth_link_check_seconds`, `discord_pool_bot_guild_cap=90` (owner Q3). **Behaviour change:** limited to pagination, the User-Agent, the rate-limit handling and the breaker. Nothing reads `guild_snapshot` until S3. **Tests:** new `tests/test_discord_rest.py` has 29 tests covering hygiene, pagination, 429 and buckets, the breaker, snapshot status mapping, cache TTLs, single flight and invalidate races. The 4 discord test files now monkeypatch `discord_rest.refresh_membership`. The leave-guild test uses `httpx.MockTransport`. `~/venvs/drb-5c/bin/python -m pytest` (drb-c2-core): **715 passed, 4 warnings** (baseline on main: 686 passed). No frontend files changed, so tsc was not run. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

logan added 1 commit 2026-10-05 01:58:10 -04:00

Discord REST client, cache and #225 settings (server-26#225 S2) 2b3b875cdc

New app/internal/discord_rest.py: one shared httpx client with the
DiscordBot User-Agent, per-token rate-limit buckets (X-RateLimit-Bucket +
guild), one retry on a 429 of <=3s, global-429 pause per token, and a
process-wide breaker at 4000 401/403/unshared-429 responses per 10 min so
C2 can never get the shared egress IP banned. Tokens are never logged.

inspect_token / refresh_membership / leave_guild / TokenRejected move off
discord_leases onto it. /users/@me/guilds now follows after= pages (it read
one page of 200 and silently dropped the rest); a failed page is unknown,
never a truncated list.

guild_snapshot(token_doc, guild_id): cached, single-flight read of
/guilds/{g}, the bot's own member and /channels for S3's permission check.
401 marks the token invalid; 10004/10007 drop the server from guild_ids;
50001 is confirmed with one membership read and never changes guild_ids
on its own. TTLs: ok 120s, not-member/invalid 600s, unknown 30s.

config.py gets every setting the #225 slices use (perm-check kill switch,
TTLs, breaker budget, OAuth redirect URI, discord_pool_bot_guild_cap=90).
No behaviour change beyond pagination, User-Agent and the breaker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

logan referenced this pull request

2026-10-05 01:58:17 -04:00

Discord: customers shouldn't need the main DRB bot in their server — radio bots (via C2) should cover linking, channel directory and permission checks #225

logan added 1 commit 2026-10-05 02:02:44 -04:00

discord_rest: guild_snapshot fails open on non-Discord errors (server-26#225 S2 review) 1a6e3ca6a6

A Firestore error inside classification (dropping membership, marking a
token invalid) propagated out of guild_snapshot and, behind asyncio.shield,
could die unretrieved. S3's start_stream must fail open, so the snapshot
now returns UNKNOWN (cached briefly) and logs only the exception type.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

logan commented

2026-10-05 02:03:09 -04:00

S2 review: ship (not merged).

Re-ran c2-core pytest on 1a6e3ca: 716 passed, 4 warnings; tests/test_discord_rest.py: 30 passed, 1 warning. No frontend change, tsc not needed.
Matches plan S2 and owner Q3 (discord_pool_bot_guild_cap=90). No other callers of the moved functions remain; token stays out of logs, errors and dict keys; guild_id is digit-checked at the router before leave_guild.
Fixed on branch (1a6e3ca): guild_snapshot now fails open (UNKNOWN, cached 30 s) on non-Discord errors such as a Firestore write inside classification, instead of raising into S3's start_stream. New test fails on 2b3b875, passes now.
Notes for S3, not bugs today: the bucket remaining is not decremented locally, so concurrent same-bucket requests can draw an extra 429 (retried/counted by the breaker); a 401 during the 50001 membership confirm marks the token invalid but returns UNKNOWN for that snapshot (pick_token already skips valid: False).

S2 review: ship (not merged). - Re-ran c2-core pytest on 1a6e3ca: `716 passed, 4 warnings`; `tests/test_discord_rest.py`: `30 passed, 1 warning`. No frontend change, tsc not needed. - Matches plan S2 and owner Q3 (`discord_pool_bot_guild_cap=90`). No other callers of the moved functions remain; token stays out of logs, errors and dict keys; guild_id is digit-checked at the router before `leave_guild`. - Fixed on branch (1a6e3ca): `guild_snapshot` now fails open (UNKNOWN, cached 30 s) on non-Discord errors such as a Firestore write inside classification, instead of raising into S3's start_stream. New test fails on 2b3b875, passes now. - Notes for S3, not bugs today: the bucket `remaining` is not decremented locally, so concurrent same-bucket requests can draw an extra 429 (retried/counted by the breaker); a 401 during the 50001 membership confirm marks the token invalid but returns UNKNOWN for that snapshot (pick_token already skips `valid: False`).

logan merged commit a75149b2eb into main

2026-10-05 02:03:36 -04:00

logan referenced this pull request

2026-10-05 02:23:49 -04:00

Discord: check bot permissions before handing a token to a node (#225 S3) #249

logan referenced this pull request

2026-10-05 03:31:21 -04:00

Discord: customers shouldn't need the main DRB bot in their server — radio bots (via C2) should cover linking, channel directory and permission checks #225

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: logan/server-26#231