Follow us
Add us as a preferred source on Google
- Tiny Corp’s $30,000 tinybox Red Kimi pairs four Radeon AI Pro R9700 GPUs (128 GB GDDR6) with sixteen 1 TB NVMe SSDs
- A Tinygrad driver lets GPUs directly read the 238.4 GB/s SSD array to run Moonshot’s 2.8 trillion parameter Kimi K3
- Tiny claims 40 tokens per second, but the math involves caching almost every token’s weight in VRAM
There’s nothing small about Tiny Corp’s latest Tinybox, the Tinybox Red Kimi: It costs a mammoth $30,000 and pairs four AMD Radeon AI Pro R9700 graphics cards with sixteen 1 TB NVMe SSDs, all with one goal in mind: running Moonshot AI’s 2.8-trillion AI in a K3-para center.
Each R9700 carries 32 GB of GDDR6 at 640 GB / s, the four cards 128 GB of GPU memory and 2,560 GB / s of aggregate bandwidth to be with a system built on Tiny Corp G5 platform, with a 32-core AMD Epyc Genoa processor which exact model the DDR3 mode and the name of the TinyBox not 5, 32. System memory in the rope.
Tiny Corp claims that the machine can run the Kimi K3 at 40 tokens per second, a figure that no one outside the company has yet tested.
Latest Videos FromTechRadar
Sixteen drives, four GPUs and an AMD EPYC CPU rule the roost
In-game storage is the unique selling point for Tiny Corp’s behemoth: its 16 drives form a RAID array rated at 238.4 GB/s, and Tiny Corp says its Tinygrad software now includes a driver that lets AMD GPUs control NVMe drives directly, rather than routing all reads through the CPU.
Sharing the topology of the system on X, the company pegged the result as “frontier models at home at usable speeds for a good price”.
The reasoning comes down to capacity. Moonshot’s model map shows the Kimi K3 ships with native MXFP4 weights, and Hyperstack measured the published checkpoint at around 1.56 TB. The four GPUs can hold about 8% of that, so the rest has to live somewhere, and Tiny Corp chose considerably cheaper flash over DRAM.
The concept is not entirely new: researchers from Nvidia and the University of Illinois presented GPU-initiated SSD access in the BaM system architecture at ASPLOS 2023, treating storage as another tier of accelerator memory. Tiny Corp’s offering, however, is a working version on AMD workstation cards, in a box you can actually order.
The configurator’s own code estimates the Red Kimi’s installed weight at about 56 pounds, in a chassis 12 inches wide, 19.25 inches tall and 18 inches deep. It also raises the noise rating from below 50 dB to below 60 dB for R9700 builds, warning that the card blower fans are “audible”.
Kimi K3 is a mix-of-experts model that activates 16 of its 896 routed experts, plus two shared experts, for 104 billion active parameters per token. At MXFP4’s effective 4.25 bits per weight, that means reading at least 55 GB of weights for each generated token.
If each of these bytes were read from the SSDs, the array would only manage four tokens per second. Reaching 40 tokens per second means moving about 2.2 TB/s of weights, about 86% of the four GPUs combined memory bandwidth, while the SSDs can contribute a maximum of about 6 GB per token, or about 11% of what each one needs.
In a nutshell, the claim only works if the GPUs’ 128 GB already contains almost everything that touches each token, even if it fits less than a tenth of the model. That’s achievable if the expert selection is predictable enough to cache well, or if Tiny Corp runs a lower precision quantization than Moonshot’s native format, speculative decoding, or a mix of the three.
Tiny Corp has not published the precision, context length, batch size or method behind its number, and it would be prudent to treat it as a manufacturer claim at best. The aggregate bandwidth figure also assumes that the workload is evenly split across four cards communicating via PCIe.
An expensive proposition that goes neck and neck with Apple’s Mac Studio
At $30,000, the Red Kimi is a lot of money for a desktop computer, but the alternatives to running a 2.8 trillion parameter model locally are not cheaper. Apple’s Mac Studio with M5 Ultra and 512 GB of unified memory starts at $22,299 and reaches $29,799 with a 16 TB SSD, Macworld reports, just $201 shy of Tiny’s box. It offers four times the fast memory at 1.2 TB/s, according to Apple, but 512 GB still can’t keep up with the K3’s native checkpoint.
Nvidia’s GB300-based DGX Station is sold for almost $91,100 with 748 GB of memory, which is also short. AMD’s Threadripper Halo Station, with up to 2.6 TB of combined memory, would fit the model comfortably, but it’s not out until 2027, and industry estimates put a complete system at around $150,000.
Against such prolific (and expensive) competition, the Red Kimi is the cheapest way to keep the Kimi K3’s native precision on a single desk. The catch is that it gets there by asking Flash to do the work of memory, and every token per second it delivers depends on how well that bet pays off. Until someone outside of Tiny Corp runs the numbers, “usable speed” is a promise rather than a result.
Follow TechRadar on Google News in the add us as a preferred source to get our expert news, reviews and opinion in your feeds.