Talking to
–
–
Last clientnone yet
Status
–
Loaded modelnone
Context–
In memory–
CPU
–%
Load 1 / 5 / 15 min–
Cores–
Memory
–GB
Total–
Available–
Live
–tok/s
Requests in flight0
Tokens streaming0
Elapsed–
Totals
–avg tok/s
Requests0
Generated tokens0
Prompt tokens0
Last 3 minutes — CPU % (blue) and tokens in flight (green)
Recent requests
| Time | Client | Endpoint | Model | Prompt tok | Gen tok | Prompt tok/s | Gen tok/s | Total s |
|---|
No requests recorded yet.
Models
Active–
Status–
| Model | Params | Quant | Size | State |
|---|
Limits & context
Context window of the active model:
–
Bigger context = more GPU memory. After applying, the line here tells you whether the model still fits in the GPU.
Max output tokens per reply (forced on every request; 0 = let the client decide):
Conversation compaction
Context used by the last chat request (estimate)–
summarises the older part of the conversation on the next chat request, keeping the last
messages verbatim
of the context window
The model writes the summary itself, so the request that triggers it takes longer (a minute or so for a long chat). Clients keep sending the full history; the proxy swaps the summarised part in on every request afterwards.
Download model
Downloads run on the server in the background; the model is imported with a 64K context and appears in the switcher when done.