Workload Segments

Segments provide a narrower lens than categories. They help distinguish request classes that share a broad task family but behave differently in production.

Segment purpose

Segments can represent region, latency sensitivity, output complexity, or other operational traits that materially change provider behaviour. They keep routing decisions grounded in comparable requests.

Benchmark alignment

When segments are introduced, benchmark suites should reflect them so merit rankings remain predictive. A routing segment without evaluation coverage weakens the feedback loop.

Policy usage

Routing policies can apply different provider preferences or fallback rules by segment. This lets teams protect the most sensitive traffic without cloning entire category policies.

Review cadence

Segments should be revisited as traffic shapes evolve. Over-segmentation creates operational overhead, while under-segmentation hides important differences in provider performance.

Next step

Use segments alongside Routing Logic when designing policy behaviour.