OrcaCyber Nano 1.0: First of 21 on a Board With No Judge in It

OrcaCyber Nano 1.0 — the 27B Mix-13G build, 13.0 GB on disk — sits at the top of a 21-model leaderboard for 16 GB GPUs with a score of 89.8 out of 100, at 60 tokens per second on a single Tesla V100. The board is run by a third-party GitHub repository, not by us. It grades every answer with code rather than with an LLM judge. And the second thing worth saying is the one a vendor normally leaves out: OrcaCyber Nano 1.0 is not the fastest model on that board. MiniCPM-V 4.7 runs at 93.4 tokens per second there. The Nano wins on the average of seven categories, not on speed, and the two nearest competitors — Qwen3.8-27B in two different community quantizations — are 1.2 and 1.4 points behind it.

What follows is a reading of that evaluation as the repository publishes it on 2026-10-19, plus the model's own published metadata from 2026-10-19. Where a number comes from OrcaRouter's own announcement rather than from the third-party board, it says so in the sentence. Nothing below has been re-run by us.

The board, and who runs it

The V100-LLM repository packages a GUI, a set of inference engines and a scoring suite for running local models on one NVIDIA Tesla V100 16 GB card under Windows. Picking a model, a preset and an engine, downloading what is missing, chatting with images and watching live tokens per second all happen in that GUI. The scoring suite is the part that produced the ranking, and the repository publishes the whole test harness next to the scores, which is unusual enough to be worth the paragraph.

The board's first suite was too easy. The repository says so plainly: most models scored between 90 and 100 on version one, which meant it could not tell them apart, and those older results are still in the repo for anyone who wants to see the failure. Version two — the hard suite — was designed so that no 16 GB-class model gets everything right. It holds 72 tasks across seven categories:

• Coding, 16 tasks — an expression parser without eval, regex matching, Dijkstra with tie-breaking, an LRU cache with TTL, topological sort, edit distance, Roman numerals, sudoku, knapsack, spiral matrix, a JSON minifier. Graded by hidden edge-case unit tests run against the generated code in a sandbox with a timeout.

• Reasoning, 20 tasks — multi-step math, logic, dates, combinatorics and word problems. The exact answer has to appear on the last line.

• Instructions, 12 tasks — three to five rules at once: word counts, forbidden letters, exact formats, ordering, JSON shape. Every rule of a task has to hold, not most of them.

• Web search, 8 tasks — multi-hop questions where you search, read the result, search again with what you found, and sometimes compute with a calculator tool. Scored on the right answer and on the minimum number of distinct searches. The "web" is a fixed offline index of 13 pages.

• Agent, 6 tasks — a small simulated project with file tools and run_tests: fix a bug, add a function, rename across files, fix a config, build a CSV report. Scored on the final file state and the project's own tests.

• Vision, 8 tasks — generated images: sum a table column, look up a row, read a bar chart, count dots, read small text, spatial relations, an analog clock, a receipt.

• Long context, 2 tasks — a text of about 14,000 tokens with three facts hidden far apart, one of them a vault code split into three parts. Exact answer.

What "no judge" is worth

Most leaderboards this size are scored by a language model reading the answer and deciding whether it is good. That is cheap, and it makes the board impossible to reproduce: the judge is a model, the model has a version, and the same answer can score differently on a Tuesday than it did on a Monday. The hard suite removes the judge and states five rules that make the score repeatable.

• Everything is checked by code. The same answer always receives the same score.

• Deterministic and offline. Temperature zero, thinking disabled, no internet access — the web-search category's web is the 13-page fixed index.

• The same preset a user gets. Each model runs with the preset the GUI recommends, so the speed and the score are the ones a person sees after installing, not a tuned-to-win configuration.

• Every task is proven solvable. The repository holds a correct reference solution for every coding and agent task, and two self-test scripts check that those solutions pass the graders. Computed answers — dates, sums, codes — are asserted in the task file.

• The answer is the last line. A model may think out loud; the grader reads the final line. Tool calls are accepted in the OpenAI format and in the Qwen XML tool_call text formats.

The scoring arithmetic is equally unfashionable. Each category is the percentage of tasks passed. Overall is the mean of the seven categories, every category weighted the same. Speed is the mean generation rate over replies of 64 tokens or more during the run. Every run is saved as a per-model JSON file recording each task's answer and whether it passed, so a disagreement about a score can be taken to the record.

Where OrcaCyber Nano 1.0 actually lands

The seven category scores, out of 100: coding 88, reasoning 95, instructions 58, web search 100, agent 100, vision 88, long context 100. Mean: 89.8.

The 58 is the interesting number, and it is the one a marketing page would round away. Instructions is the category where a model has to hold three to five simultaneous constraints — a word count and a forbidden letter and an exact output shape — and Nano loses five of twelve tasks there. Everything else is at 88 or above. Two categories are perfect.

The nearest competitor is Qwen3.8-27B in HauhauCS's Aggressive IQ3_XS quantization at 88.6 overall and 42.2 tokens per second; third is the Unsloth UD-IQ3_S build of the same model at 88.4 and 52.1 tokens per second. Both land 94 to 100 on coding and reasoning, both drop to 50 on instructions — so the instruction-following weakness is a property of the 27B class at this compression level, not a defect unique to the Nano. What separates first from third is 1.4 points spread thinly: a few tasks in a category, not a different tier of model.

Sixty tokens per second, and why the fastest model is elsewhere

The headline speed is 60 tokens per second at a 32k context, and the repository is careful about what that number is. It is the speed of the model's tuned fast-32k preset with multi-token prediction at depth 3, measured as 66.6 tokens per second on code and 54.0 on chat for a mean of 60.3. The plain mean over every evaluation reply is 56.9. Both figures appear in the footnote of the table rather than only the flattering one.

Speed falls as context grows, and the repository publishes the ladder: 57 tokens per second at 96k, 32 at 160k, and 24 at 262k once 20,000 tokens have been filled in. Prompt processing on short prompts is 501 tokens per second. Up to 19,600 tokens, every preset passes.

The fastest model on the board is a different one. MiniCPM-V 4.7 in the 35B-A3B Q3 mix build runs at 93.4 tokens per second at a 128k context, and it gets there by moving the vision encoder onto the CPU so that all of its experts stay on the card. It ranks fourth overall at 87.5. The repository's own framing is a trade, not a sweep: the Nano is described there as the fastest dense 27B in the table, which is a narrower claim than the fastest model, and a true one.

The measurement environment is stated once and applies to the whole table: one Tesla V100-SXM2-16GB, a Xeon E5-2660 v4, 64 GB of DDR4. The engines under test are PXA, llama.cpp with a V100-specific patch, a Prism ML fork and Strata, each under the MIT licence; CUDA DLLs follow NVIDIA's EULA. Model weights are not bundled — each one is downloaded from its author's own page on Hugging Face.

What the model is, from its own metadata

The Hugging Face repository orcarouter/OrcaCyber-Nano-1.0 reports a 262,144-token context window, an image-text-to-text pipeline (it takes images as well as text), the GGUF library, and the English and Chinese languages. It is gated: you sign in to download. The licence field reads other with the licence name qwen. As of 2026-10-19 the repository showed 18,606 downloads and 151 likes, and its base model field names OrcaCyber-Nano-1.0-AI-Defense with the relation quantized — the open-weight files are quantizations of a defense-tuned parent, which is the honest way to read that field.

There are four files, and their sizes explain most of the deployment decisions in the next section:

• Mix-13G — 12,993,304,352 bytes, the 13.0 GB build that scored 89.8.

• Mix-15.7G — 15,695,148,832 bytes, the larger build, still inside a 16 GB card's budget.

• DFlash2 Q8_0 — 2,056,414,112 bytes, the draft model for speculative decoding.

• mmproj F16 — 927,606,848 bytes, the vision projector, which is what makes the eight vision tasks possible.

Add the engine and a working install is about 14 GB on disk: 13.0 GB of weights, roughly 0.9 GB of projector, and the runtime.

Thirteen gigabytes fits on the card. Two hundred and sixty-two thousand tokens does not.

These are two different claims and press coverage tends to run them together. The 13.0 GB weight file fits inside 16 GB of VRAM, which is why the model runs on a V100 with room for the KV cache of a 32k context. The 262,144-token maximum context does not fit on the card; it needs the KV cache in system memory, and the repository's requirements list 64 GB of system RAM for the 262k preset against 32 GB for ordinary use. Push to that ceiling and the speed is the 24 tokens per second from the ladder above.

The rest of the hardware requirement is modest and specific. Any 16 GB NVIDIA card from Volta onward will run the stack; the driver must be from the R576 series or newer; a V100 left in TCC mode is fine. There is no WSL, no Docker, no CUDA toolkit, no Visual Studio and no Python to install, because the folder ships its own engines and its own portable Python. About 20 GB of free disk is needed per model.

The vendor numbers, labelled

OrcaRouter announced OrcaCyber Nano 1.0 on 2026-10-18 as an emergency release, open-weight, and placed it in a three-model series alongside OrcaCyber Zero 1.5 and OrcaCyber Ultra 1.0. The same week, the company opened free priority access to its OrcaCyber models for teams actively responding to an intrusion. The next day's post led with the leaderboard result and described Nano as, on the vendor's own early evaluations, the most capable local model in its under-16GB class.

That last phrase is a vendor claim and has not been independently reproduced. So is the benchmark the announcement repeated: 78.4% on SWE-bench Verified, set beside DeepSeek's reported 79.0% for DeepSeek V4 Flash. Both of those numbers are reported by their respective vendors; neither the 78.4 nor the 79.0 has been run by this publication, and the comparison is interesting mainly because a 27B model is being placed within 0.6 points of a much larger hosted one on a vendor's own harness. Read it as an announcement, not as a result.

The model card itself is worth a note for what it does not contain. Its tags mention cybersecurity, reverse engineering, function calling and vision, but the card carries no cyber-specific benchmark numbers — no incident-response score, no vulnerability-detection rate, no CTF result. Those live on the Zero pages instead. And the CyberGym figure that circulates around our cyber line belongs to a combined setup — Zero 1.0 plus DeepSeek V4 Flash plus adaptive routing — not to any single model's solo run. If you are choosing this model for defense work, the honest state of evidence is: a code-graded general suite put it first of 21 on a 16 GB card, and the cyber-specific evidence is elsewhere and belongs to other models.

Where it sits next to the hosted cyber models

OrcaCyber Nano 1.0 is a weight release for local hardware. You download 13 GB, you run it yourself, and the score above is what it does on your card. It is not an endpoint on our catalogue today, and nothing in this article should be read as saying otherwise. What the catalogue does carry is the rest of the series — OrcaCyber Zero 1.5 and OrcaCyber Ultra 1.0 — under the same key and the same OpenAI-compatible endpoint as 200-plus other models, including Claude Opus 5.5, GPT-6.1 Sol, Kimi K3, Qwen3.8-Max and DeepSeek V4 Flash, with no token markup over provider list price and automatic failover across providers.

That split is the useful one to hold in your head. Nano is the model you own and run offline, with no data leaving the machine — which is the entire point of a cyber model for incident response, because the machines you most need to inspect are frequently the ones you cannot send to an API. Zero 1.5 and Ultra 1.0 are the models you call. A local tier for the air-gapped work, a hosted tier for the throughput, one vocabulary of model names between them.

What to do with the 89.8 depends on what you needed it for. If you were waiting for evidence that a 27B model at 13 GB is competitive on a 16 GB card — not merely runnable, competitive against 21 alternatives on tasks that are all graded by executable code — this is that evidence, with its instruction-following gap visible at 58 and its speed ceiling visible at 60 tokens per second. If you needed the fastest thing on a V100, that is a different model on the same board at 93.4, and it is one row down the table.