8.0k
New

Context

Previous Next

How much of the context window the conversation has used.

Epicenter

Local-first, open source apps

Special Sponsor 
<script lang="ts">
	import * as HoverCard from "$lib/components/ui/hover-card/index.js";
	import { Button } from "$lib/components/ui/button/index.js";
	import { Progress } from "$lib/components/ui/progress/index.js";
 
	const usage = {
		input: 8420,
		output: 1260,
		reasoning: 980,
		limit: 128000,
	};
 
	const used = usage.input + usage.output + usage.reasoning;
	const percent = Math.round((used / usage.limit) * 100);
 
	function formatTokens(value: number) {
		if (value < 1000) {
			return String(value);
		}
 
		return `${(value / 1000).toFixed(1).replace(/.0$/, "")}k`;
	}
 
	const rows = [
		{ label: "Input", value: usage.input },
		{ label: "Output", value: usage.output },
		{ label: "Reasoning", value: usage.reasoning },
	];
</script>
 
<div class="flex justify-center py-12">
	<HoverCard.Root openDelay={10} closeDelay={100}>
		<HoverCard.Trigger>
			{#snippet child({ props })}
				<Button {...props} variant="outline" size="sm">
					{percent}% context
				</Button>
			{/snippet}
		</HoverCard.Trigger>
		<HoverCard.Content class="flex w-64 flex-col gap-3">
			<div class="flex items-center justify-between gap-3 text-sm">
				<span class="font-medium">Context</span>
				<span class="text-muted-foreground">
					{formatTokens(used)} / {formatTokens(usage.limit)}
				</span>
			</div>
			<Progress value={percent} aria-label="Context used" />
			<div class="flex flex-col gap-1 text-sm">
				{#each rows as row (row.label)}
					<div class="flex justify-between gap-3">
						<span class="text-muted-foreground">{row.label}</span>
						<span>{formatTokens(row.value)}</span>
					</div>
				{/each}
			</div>
		</HoverCard.Content>
	</HoverCard.Root>
</div>

Context shows how much of the model's context window the conversation has used. It is a Hover Card with a Progress bar.

Installation

Context is built with the Hover Card and Progress components.

pnpm dlx shadcn-svelte@latest add hover-card progress button

Usage 

<HoverCard.Root openDelay={10} closeDelay={100}>
  <HoverCard.Trigger>
    {#snippet child({ props })}
      <Button {...props} variant="outline" size="sm">
        {percent}% context
      </Button>
    {/snippet}
  </HoverCard.Trigger>
  <HoverCard.Content class="flex w-64 flex-col gap-3">
    <div class="flex items-center justify-between gap-3 text-sm">
      <span class="font-medium">Context</span>
      <span class="text-muted-foreground">
        {used} / {limit}
      </span>
    </div>
    <Progress value={percent} aria-label="Context used" />
  </HoverCard.Content>
</HoverCard.Root>

AI SDK

Read usage from the metadata on the agent's message. messageMetadata runs when the stream finishes.

src/routes/api/chat/+server.ts
return result.toUIMessageStreamResponse({
  messageMetadata: ({ part }) => {
    if (part.type === "finish") {
      return { usage: part.totalUsage };
    }
  },
});
const usage = $derived(message.metadata?.usage);
const used = $derived((usage?.inputTokens ?? 0) + (usage?.outputTokens ?? 0));
const percent = $derived(Math.round((used / limit) * 100));

limit is the model's context window. The AI SDK does not send it, so set it from the model you use.