We've got a new update. We now offer our own Web Console. Open the console and type ssh root@your-server-ip to reach any server you own, right from your browser. No SSH app needed.
When you run an open-weight model yourself, your prompts and documents go to a server you rent, not to a third-party API. The question that decides everything is whether the model fits in GPU memory. This page gives you the arithmetic and the matching server, with real prices.
A rule of thumb: weights = parameters × bytes per weight, plus about 20% for working memory. Real needs vary with context length, batch size and software.
| Model and precision | Estimated memory | 1x RTX 4090 (24 GB) | 1x RTX 5090 (32 GB) | 1x H100 (80 GB) | 2x H100 (160 GB) |
|---|---|---|---|---|---|
| 8B parameters, 16-bit | ~20 GB | fits | fits | fits | fits |
| 8B parameters, 8-bit | ~10 GB | fits | fits | fits | fits |
| 8B parameters, 4-bit | ~5 GB | fits | fits | fits | fits |
| 32B parameters, 16-bit | ~77 GB | no | no | fits | fits |
| 32B parameters, 8-bit | ~39 GB | no | no | fits | fits |
| 32B parameters, 4-bit | ~20 GB | fits | fits | fits | fits |
| 70B parameters, 16-bit | ~168 GB | no | no | no | no |
| 70B parameters, 8-bit | ~84 GB | no | no | no | fits |
| 70B parameters, 4-bit | ~42 GB | no | no | fits | fits |
| 120B parameters, 16-bit | ~288 GB | no | no | no | no |
| 120B parameters, 8-bit | ~144 GB | no | no | no | fits |
| 120B parameters, 4-bit | ~72 GB | no | no | fits | fits |
These are estimates for planning, not benchmarks. A model that just fits leaves little room for long prompts, so leave margin. Quantised models lose some quality; test yours.
Prices change with the location you choose. A longer term takes up to 50% off.
or $54.63/mo on a 12-month term (−50%)
Fits small and mid-size models, and 32B at 4-bit.
Configure and orderor $87.88/mo on a 12-month term (−50%)
More headroom for context and larger quantised models.
Configure and orderor $375.25/mo on a 12-month term (−50%)
A 70B model at 4-bit, or 8-bit for mid-size.
Configure and orderor $707.75/mo on a 12-month term (−50%)
Very large models and 16-bit mid-size models.
Configure and orderWhatever you choose, do not expose the model API to the internet without authentication. Our walkthrough is in run LLMs on a GPU server.
Your prompts stay on a server you rent and nobody else shares the GPU. They are not sent to a model vendor. The server is still hosted by us, we do not monitor or log inside your servers, and the rules in our terms apply to everything you run, including what you generate and publish. You are responsible for following the licence of the model you download.
Yes, if you follow the licence of the model. Choose a server whose GPU memory fits the model with some margin, install a serving tool such as Ollama or vLLM and secure the API.
Roughly 35 GB of weights at 4-bit, 70 GB at 8-bit and 140 GB at 16-bit, plus about 20% for working memory. By that estimate a 4-bit 70B model needs a 80 GB H100, not a 24 or 32 GB card.
The RTX 4090 server fits them at any precision in the table, and the cheapest is $109.25 a month. Larger models need more memory.
No. It means you choose and run the model on your own server. What you generate and publish is still covered by the law and by our terms.
No. Generate a login and pay in crypto. See no-KYC GPU servers.
Monthly from your prepaid balance, with up to 50% off on longer terms. See pricing.
Pick a GPU, pay in crypto, log in as root. No ID, no card.
