Paused — no sweeper/batch/client refresh, not in the stale queue, no >1h cache mail. Force refresh still works.
Trader
Paused
Reason
Actions
Loading…
Auto refresh (cron sweeper)
Loading…
Each sweep (DO alarm every N ms) lists traders whose last refresh start is older than TTL and kicks off refreshes.
Fan-out is how each trader refresh is started:
SELF binding stays in the sweeper colo;
Public HTTP goes through the front door like dashboard “Run scrape”;
Auto tries both, then keeps the healthier path (success rate, then speed) and re-checks.
Max in-flight is the concurrent scrape cap.
Starts/sweep limits how many new refreshes one sweep may start.
Pick is which stale traders those starts go to:
Oldest first (the original queue) starts the most overdue;
Newest first starts the least overdue among those already stale.
0 — fill available slots up to max in-flight (start all stale traders that fit; fastest catch-up)
1 — one new refresh per sweep (gentle on scrape worker; stale queue builds up)
2, 3, 5… — that many new refreshes per sweep at most (e.g. 3 → up to 3 new starts each sweep)
Sweep interval is the gap after each sweep finishes (not while it runs). With starts/sweep 0, a sweep should finish in ms and the DO alarm fires about every interval.
TTL: target gap between scrape starts (Δ prev). Scrapes often take ~0.6–1.0s; cache age stays a bit under TTL.
1s is usually too tight (scrape can exceed TTL).
2s is a good balance; 3s is stable. See notes/ttl-cadence.md.
Presets:
pageSize jitter: one size per scrape, whole fleet, rotates every Open jitter seconds.
30s was for 10–20 (11 keys ≈ 5.5 min walk). Open range is now 21–50 (30 keys) — same 30s is a 15 min walk.
Parallel runs two families (open+closed each) and watermark-merges — enable Run 1 (10s 21–50) and/or Run 2 (30s 10–20).
Bybit caches each pageSize ~5 min (shared across clients). Random jitter is not Math.random() — same clock bucket → same size for every trader.
Full walk = every value in the range once. Match that to ~one cache life (~5 min).
Range
Keys
10s
15s
30s
21–50 (open default)
30
5 min
7.5 min
15 min
10–50
41
6 min 50s
10 min
20.5 min
10–20 (old open / retry)
11
1 min 50s
2 min 45s
5.5 min
5–20
16
2 min 40s
4 min
8 min
On 21–50 use Open jitter 10s (or 15s). On 10–50 10s is a ~7 min walk. One scrape = one size. Probe tries 3 sizes in the same range. Mixed-book retry uses the other family (21–50 ↔ 10–20).
Proven 100/100 — saved when the last 100 refresh scrapes all succeeded with the same knobs. Same combo is updated, not duplicated.
Success rate alert
Email when fleet scrape success over the last N minutes is below the threshold.
Checked every cron minute. Sends once when it drops, then at most every 15 minutes until it recovers.