Benchmark scores say nothing about whether the release runs

On Nov 8 I went through a fresh speech-model release the way most people in that scene go through them, model card first and benchmark table second, and I never argued with a single score. Their own README asked for one 80 GB GPU, my largest card carries 24 GB, and the disk holding my weights had 12 GB free against roughly an 80 GB download. The hardware gate killed it twice before accuracy came up at all. The claim underneath that story: adoption decisions in the open-weight scene get made from a model card and a benchmark row, while the two hardware gates, VRAM floor and free disk, rarely get checked before the download starts. Both failed here, and the eval number behind them can't be checked either. The gate nobody checks The org is adventists-ai, 16 models. The family page Sophon-Realtime-v1 is a README only, 5 files, zero bytes of weights, 32 likes, downloads untracked. The weight repos sit next to it: Sophon-Realtime-32B-Tag at 44.61 GB over 68 files, created 2026-10-29 with 202 downloads and 3 likes, Sophon-Realtime-32B-Flash at 40.89 GB over 55 files, created 2026-11-07 with 10 downloads, plus a research preview E2E repo. Their README states the floor plainly: Tag needs "one 80 GB GPU, ~65 GB per instance", with stage0 at gpu_memory_utilization: 0.62 and stage1 at 0.08. E2E wants two 80 GB GPUs. My fleet maxes out at 2x RTX 3090, 24,576 MiB each, so 48 GB total across both cards, no single card anywhere near 65 GB, and their deploy shape has no supported split for that layout. Then disk: df -h on the volume holding everything came back 930G size, 917G used, 12G free. Their requirement is roughly 80 GB of weights. Driver and CUDA were fine (610.57.04 against a floor of 580), which is the only gate that passed. What the card actually is The GitHub repo adventists-ai/sophon-realtime was created 2026-11-04 and pushed 2026-11-08, Apache-2.0, Python, 176 tree entries, 8,227 lines across the tag/flash plugin, the sophon_expressive engine and a live gateway. Underneath the packaging sits a stack of models I already run. thinker/config.json declares model_type qwen3_vl, arch DuplexJevVisionForConditionalGeneration, dtype bfloat16, but quantization_config is compressed-tensors float8 with channel-symmetric static weights and dynamic fp8 activations, and the thinker shards total about 33.7 GB on disk, already FP8. Their NOTICE file says the thinker derives from Qwen3-VL-32B-Instruct through DuplexJev-32B-Para plus a merged tag-writing LoRA, the audio encoder derives from Qwen3-ASR, the talker is tag-tts off Fun-CosyVoice3-0.5B with the flow distilled to 4 steps, and the Flash talker comes from Qwen3-TTS-12Hz-1.7B-Base. Three stages share one GPU over CUDA MPS: thinker, talker, then flow plus a HiFT vocoder in TensorRT. Packaged, it is one vLLM-Omni pipeline with three stages resident at a time, and that residency is where their 65 GB per instance comes from. The citation check The arXiv id printed on their own model page is 2609.21967, which resolves to NVIDIA's paper, "NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilities", submitted 18 Sep 2026 and last revised 1 Oct 2026. That means the citation printed on their model page points at research from another lab. Adventists AI ships no first-party writeup and no eval harness, git ls-tree shows no eval or bench directory, and the repo has zero HF discussions. The comparison rows they borrowed are faithful: their VoiceBench entries for MiniCPM-o 4.5 at 76.1, Freeze-Omni 55.2, PersonaPlex 30.6, Moshi 29.5 and NemotronLabs 55.1 match that paper's Table 3 exactly, so the baseline rows are copied straight off NVIDIA's table. Sophon's own scores go the other way. VoiceBench 82.8, Big Bench Audio 81.8%, FDB v3 Pass@1 48%, NVV-SuperBench 0.86/0.80 and paralinguistic 48.6 appear in no public table I could open, and the digits 82.8 do not occur anywhere inside the paper they cite. Two more gates before any install License. The headline says Apache-2.0 weights, and Tag, Flash, tag-tts plus the family README check out per file on cardData, LICENSE and the GitHub license field. The DuplexJev checkpoints under the same org are CC BY-NC 4.0. Personal use is fine, commercial use isn't, and the marketing line hides that difference. Network defaults. live/sophon_live/backend.py sets SOPHON_LIVE_BACKEND_URL with a default pointing at Alibaba's DashScope compatible-mode endpoint, so delegated lookups leave the box unless overridden to a local lane. finetune/tools/regress.py opens a URL against an OpenAI-compatible judge for its regression checks, and the sibling duplexjev repo advertises api.adventists.cn/duplexjev as its demo endpoint. Reading the tag and flash plugins myself turned up network calls only to localhost once weights land, so the engine is local and the gateway wrapped around it isn't. Every lane I own is supposed to answer without leaving the house, so a cloud default is something I patch first and test after. The one piece that fits my cards The only piece fitting one 3090 today is tag-tts standalone at 7.73 GB, Apache-2.0, which takes tagged text ([laughter], [sigh], ) and performs it. My brain models already emit those tags, so it's a candidate swap for the TTS stage in my duplex voice lane without needing the 32B thinker. Unverified, because that repo ships no standalone server, talker_tag.py references upstream cosy_server.py from the FunAudioLLM/CosyVoice runtime. Adoption means running their weights under CosyVoice's code path. DuplexJev-4B at 8.5 GB could serve turn-taking and barge-in decisions over audio locally behind duplexjev-vllm==0.3.0, with the CC BY-NC pin standing in front of any commercial use. Flagship Sophon stays parked. What would change the verdict: a single GPU of 80 GB or more on the box, or a 4B-thinker variant of Sophon, which does not exist in that org listing, plus about 80 GB of free disk. And a published repro harness, which is what turns 82.8 from self-reported into something I can test against my own lanes. The pattern that carried the verdict is order of operations. Hardware floor from their README, free space on the target volume, citation resolution, license pin per file, network defaults in the gateway code. Five checks, all read-only, all inside twenty minutes, and they answered "is this worth installing" without one gigabyte moving onto my disks. A release that passes those gates still needs an end-to-end run against whatever it replaces, but a release that fails them never earns a benchmark line in your notes. What's your habit here? Do you try the weights before you check the VRAM floor, or has someone got a card big enough that the ladder never matters?