How we measure

The standard preset (standard-v1)

Every published row is produced by one fixed configuration, matching llama-bench defaults so results are comparable with the wider ecosystem: a 512-token prompt, 128 generated tokens, and 3 repetitions, using one shared prompt identical for every user.

The preset is versioned. If it ever changes, the new rows carry a new preset id rather than silently mixing with the old ones. A run's preset id follows the parameters it actually executed, not the mode it was started from, so a run that happens to match the standard values is treated as standard whichever way it was launched.

Runs made with a different configuration are published too, but in their own custom-preset section beneath the leaderboard, never in the comparable table and never merged with it. They are not ranked against anything: a shorter generation is a faster number for reasons that have nothing to do with the device. Each such row states the parameters it ran, or says plainly that they were not recorded — app builds up to 1.3.8+109 did not send them. No device recommendation anywhere on this site is derived from a custom-preset run.

What we exclude

• Debug builds — materially slower than release.
• Runs started on an already-hot device, where the thermal state at start was worse than “light”.
• Anything not produced by standard-v1.

What each metric means

TTFT

Time to first token, in seconds. Measured from submission to the first token surfacing, averaged across repetitions.

Prompt (PP)

Prompt-processing throughput in tokens/second over a fixed 512-token prompt.

Decode (TG)

Token-generation throughput in tokens/second over 128 generated tokens. This is the number you feel while reading a reply.

Peak RAM

Resident set size read once, after the run completes. RSS undercounts some native and GPU allocations, and the Ollama and cloud paths do not report it at all, so treat it as a rough floor rather than a ceiling.

Sustained

Decode throughput after sustained load, reported alongside time-to-throttle. Absent until a run includes the sustained block.

Provenance tiers

• verified — run by us on hardware we hold.
• community — submitted from the app under the standard preset. Shown as a median with a sample count, so one outlier cannot move a cell.
• estimated — derived arithmetic, never measured. Always labelled.

What a fit verdict assumes

The “tight” marker on the device advisor is arithmetic, and it uses the same two rules the FluentAI app itself applies before it will load a model. Both products now answer this question the same way; before, the site called a model comfortable that the app flagged as borderline on the same phone. The gigabyte figure beside the marker is the measured peak RAM where the configuration quoted for that row published one, and this same arithmetic everywhere else — the row says which. The verdict is arithmetic either way, so a model the app refuses is refused here too even when a run measured smaller.

• A runtime footprint of 1.4× the file size. Loading a model costs more than its weights: the KV cache and the context buffer come too. That multiplier is the app's own, tuned against real out-of-memory crashes rather than derived — a 1.8 GB model really does fail on a 4 GB phone, though its file is under half the memory.
• Borderline above 60% of total memory. Mobile systems kill an app well before RAM is actually full, so a model whose footprint passes that share is reported as a tight fit rather than a comfortable one.
• Larger than total memory means it will not run. A file bigger than the whole device cannot be loaded at all, so it is separated out rather than ranked.
• Chat models only. Image models are left out of the advisor entirely: their memory profile is not file size plus KV at all, so this arithmetic would say nothing true about them.

Where the speeds come from

The advisor ranks by how fast a model should run, and the bands are drawn on the app's own bar for usable: below 5 tokens per second it stops being something you can hold a conversation with. Where somebody has published runs of that exact model file on that exact device, the figure is one published configuration's median — the configuration with the most samples behind it — stated exactly as the leaderboard states it. Runs differing in backend, context size, KV cache type or speculative decoding are different numbers and are never averaged together, so the advisor quotes one of them rather than a blend of all. Everywhere else the figure is an estimate, produced by the same model the app uses to rank your local models — and every row says which of the two it is. Measured and estimated speeds are never averaged together.

A published run speaks for a catalogue entry only when it measured that same artifact, because a model id alone does not identify a file — the same id can be a GGUF build in our catalogue and a LiteRT bundle in the submissions, at different sizes. A run therefore has to match on all of:

• The same model id, the same quantisation, and a runtime that produces the catalogued format.
• The same device. Another phone's throughput is not yours, however similar the two look.
• A measured row. Estimated rows are arithmetic already, and publishing one as a measurement is the blend this site exists not to do.

Where nothing qualifies, the row says “estimated”. Silence is the honest answer; a nearest match is not.