Skip to content
New on arXiv: a judge's uncertainty decomposes, so expert labels go exactly where they remove error
← Back to Blog

Demo: Failure Mode Clustering

Luke Markham · CTO & Co-founder ·

Once traces start failing your criteria, the question stops being which trace and becomes which problem. Composo clusters failing traces by similarity and names each cluster, so you triage a handful of recurring failure modes instead of scrolling a list.

In the video: the failure map, how each mode is trending against last week and which ones are new, opening a mode to read the traces behind it, and promoting a mode into an evaluator so the failure you just found gets measured from now on.

The method behind it is written up in A Living Map of LLM Failure Modes. To turn it on for your own traces, start with the clustering docs.