esp32-eye: The Cat Water-Bowl Monitor That Calls MiniCPM-V 4.7 Twice

Two ESP32-S3 boards, one MacBook, and two different sizes of MiniCPM-V 4.7 now keep a log of which cat drank from the water bowl and when. A camera board watches the bowl, a MacBook runs the small model locally to answer one narrow question, and the large model is called only after an event has already happened. The build is called esp32-eye, it lives at github.com/emwstudio/esp32-eye, and it went public on 25 October 2026. OpenBMB showed it off in the first week of November, which is how most people met it.

The model underneath is the part worth being precise about. OpenBMB uploaded MiniCPM-V 4.7 to Hugging Face on 6 and 7 October 2026, published the model card and the Apache-2.0 licence on 12 October, and put out an English press release through PR Newswire on 15 October. So this is not a launch story, and the cat monitor is not a vendor benchmark. It is the most detailed public deployment of the model that anyone outside the lab has documented so far, and it happens to be a household appliance.

What the build actually does

The hardware is deliberately cheap and slightly awkward. A Waveshare ESP32-S3-CAM-OV5640 sits on the bowl rim and streams 1024×768 JPEG frames plus microphone audio over Wi-Fi. A second board, a Waveshare ESP32-S3-Touch-AMOLED-2.16, is the desk display: a 480×480 panel showing a live video strip, four cat cards, a daily count against a target of eight drinks, and a small highlight animation when a cat is confirmed. The MacBook is the brain. A FastAPI server takes the stream, runs the detection logic, writes to SQLite, and cuts an MP4 clip for each event with five seconds of lead-in, three seconds of tail, and the original audio.

The camera placement is the first thing to understand, because it explains the whole detection design. The lens looks up at the cat's face from the bowl rim, which means the water surface is not in frame. The system cannot see a cat drinking. It can only see a cat occupying the bowl area, and it infers the rest from persistence and identity. Anyone who has built a camera project knows this is where the honesty usually leaves the room; here it is written into the README.

The pipeline before any model runs is a grayscale background difference against an empty-bowl reference frame. If the foreground share is below 15 percent, the frame is discarded without touching a neural network. Above 15 percent, the local MiniCPM-V 4.7 1B is asked one question: is there a cat in this image? A positive answer plus a foreground share above 35 percent becomes one affirmative vote. Three consecutive affirmative votes open an event; four consecutive negative votes close it.

The two-model split is the interesting engineering

Running a vision model every 1.5 seconds forever is a bad idea, and the build treats it as one. The 1B runs locally through llama.cpp as a Q4_K_M quantisation on an OpenAI-compatible llama-server on port 8080. It is cheap enough that the project can afford to call it repeatedly, but it is still gated by the background difference so that an empty room costs nothing. The 1B's job is deliberately narrow: presence, not identity, not behaviour.

Identity is the expensive question, and it is asked once per event. When an event closes, the server samples frames from the clip and sends them to MiniCPM-V 4.7 35B-A3B through ModelBest's cloud API, with a reference photo for each of the four cats. The large model returns a cat identifier, a confidence value, and a yes/no on whether the scene is a drinking scene. Frames that fail the drinking check are thrown away and not counted. Ambiguous identities are stored as unknown rather than forced onto the nearest cat, and if the cloud call fails the event goes into a retry path instead of being backfilled with the 1B's guess.

That division of labour is the same argument the on-device literature has been making for two years, now with a receipt. A small model is a sensor. A large model is a judge. The cost model follows directly: the 1B is paid for in milliseconds and battery, the 35B is paid for in tokens, and the token bill scales with the number of drinking events rather than with the number of frames. In a four-cat household that is a handful of calls a day.

What the 89 events do and do not prove

OpenBMB's post reports that in a four-cat household over three days, 89 drinking events were manually checked and every cat identity was correct. That is a real result and it is also a narrow one. The identity task is a closed-set four-way choice among cats the system already has reference photos for. It is not open-set re-identification, it is not a claim about a hundred cats, and it is not a claim that the system works in a house it has never seen.

Eighty-nine events across three days is roughly thirty a day for four cats, or about seven and a half per cat per day, which is plausible for a water intake study and far too small to estimate an error rate with any confidence. A system that is right 89 times out of 89 could still be wrong one time in fifty. The number to hold onto is not 100 percent; it is that nobody had to correct it in three days of ordinary use.

There is also a provenance caveat. The author is an OpenBMB community ambassador and the showcase ran on OpenBMB's own account. That does not make the result false, but it does mean the validation was not designed by a sceptic, and no third party has repeated it.

Why the 1B does not identify the cat

The README contains the most useful sentence in the whole project, and it is a negative result: with three light-coloured cats that look alike, the local MiniCPM-V 4.7 1B was not reliable at picking one of four. The team tried it, found the four-way discrimination unreliable, and moved identity to the 35B rather than shipping a guess.

This is worth dwelling on because it cuts against the way small vision models are usually sold. A 1.55-billion-parameter model that scores well on OCR and chart benchmarks is not automatically good at telling two similar animals apart in a fixed camera view. Fine-grained identity is exactly the kind of task where a compressed visual token budget hurts: the differences between two tabbies are small, local, and easy to average away. The build's answer — use the small model for the easy question and pay for the hard one — is more useful than any leaderboard row.

What it costs to run, and what happens when the cloud leg fails

Only one leg of this pipeline has a bill. The 1B is a local GGUF file on a laptop, so the marginal cost of a frame is electricity. The 35B is a hosted call against ModelBest's API, which is the vendor's own endpoint rather than a third-party host, and the project needs a key for it. The README is explicit that the 35B is not deployed locally: the MacBook does not hold 35 billion parameters, it holds a client.

That split creates a failure mode the project handles rather than hides. If the cloud call fails, the event is retried; the 1B result is never promoted into an identity. For a hobby project that is unusually disciplined. For anything production-shaped, the same design puts a single external endpoint on the critical path of a system whose whole purpose is to record events unattended.

This is the gap a router fills, and it is worth being precise about what a router can and cannot do here. OrcaRouter gives one API across more than 200 models with provider list price passed through at 0% markup, plus automatic failover between upstreams, which is the right shape for a pipeline like this one. It does not currently carry MiniCPM-V 4.7, so the 35B leg in esp32-eye would still run on the vendor's own API. The honest version of the advice is: keep the retry path, and if the identity step becomes load-bearing, put a second model you can actually reach behind the same call.

What is still unconfirmed

Three things would change how much weight this build deserves. The first is a second household. Everything reported so far comes from one home with four known cats and a fixed camera. The second is a longer run: three days cannot distinguish a system that works from a system that works until a cat changes its coat, the light shifts, or a visitor arrives. The third is an independent check of the 89 events, which currently rests on the builder's own manual review.

What is already established is more modest and more useful. MiniCPM-V 4.7 exists in two sizes that can be composed: a 1.55B dense model that runs in a Q4_K_M file on a laptop, and a 35.2B sparse mixture-of-experts that activates roughly 3 billion parameters per token and is good enough at a four-way identity task to be trusted with it. A cat bowl is a strange place to prove that, but it is a real one.

If you want to reproduce it, the repository has the camera firmware, the AMOLED firmware, the FastAPI server and the model test scripts, and the only account you need is a ModelBest key for the identity leg. If you want to read the model rather than run the cat, the vendor's numbers are the place to start and the caveats are in the repo.