Home → Models → 500 Milliseconds to Local

500 Milliseconds to Local

JERSEY CITY HOBOKEN MANHATTAN LONG ISLAND CITY BUSHWICK BAY RIDGE STATEN ISLAND (pending fiber) THE BRONX PARK SLOPE (HQ) 500 ms
Our local inference radius. Red dots are member GPUs; most are in closets.

If you live in New York City, your data already lives somewhere else: northern Virginia, central Oregon, a server campus in a state you couldn't find on a map. We think that's too far for something you're going to ask about your rash.

Why 500 milliseconds?

We define local as anything inferred within 500 milliseconds of our Park Slope basement. That's about how long a New Yorker will wait before assuming something is broken.

A token picked fresh at 8:00 a.m. can be sampled, detokenized and on your screen by 8:00:00.5 a.m. No freezing. No cross-country shipping.

Where our AI comes from

In season (roughly October through April, when it's cold enough to cool the racks with an open window), most of what we serve is grown within 40 milliseconds of home. Our growers include:

  • More than 40 small-scale GPUs owned by members
  • A Mac Mini in Park Slope and a Raspberry Pi cluster in Bay Ridge
  • A former bodega walk-in in Bushwick
  • Gaming PCs borrowed from members' teenagers on school nights
  • On a bad Wi-Fi day, Hoboken

In a summer heat wave, we buy compute from New Jersey before California. Storage crops like embeddings and cached prompts stay local all year.

“I used to get my AI from Virginia. Now I can hear it thinking through the floor.”

Dana R., 7th Avenue, Park Slope

Why buy local?

Two reasons. First, freshness: our models have never been frozen, shipped or quantized below four bits without their consent. Second, it keeps compute money in the neighborhood.

We also go for “little”: small-batch models under eight billion parameters, raised on family-owned graphics cards by hobbyists who are, mostly, good stewards of the electrical grid. Every model is free-range and pasture-raised on openly licensed text.

Read the label

Every model in our aisle says where it was grown. If several racks helped, the tag reads locally inferred within 500 ms. If it came from one machine, we name it: Raised on an RTX 4090 in Crown Heights.

It makes “local” hit home, even in one of the biggest clouds in the world.