Run Ollama and Open WebUI on a GPU Server

Running a language model on your own server keeps your prompts and documents private and removes per-token fees. Ollama makes the model side simple, and Open WebUI gives you a ChatGPT-style interface on top. This guide installs both on a GPU server running Ubuntu.

What you need

• A GPU server. Model size decides the GPU. A 7 to 8 billion parameter model in 4-bit form needs roughly 5 to 6 GB of VRAM. A 70B model needs far more, typically 40 GB or more. See the private LLM hosting page for sized options.
• Ubuntu 22.04 or 24.04 with the NVIDIA driver installed. GPU plans from us can come with CUDA ready.
• Docker for Open WebUI. Follow how to install Docker if you need it.

Step 1: check the GPU

nvidia-smi

You should see your GPU, the driver version and the CUDA version. If the command is missing, install the NVIDIA driver for your card first.

Step 2: install Ollama

curl -fsSL https://ollama.com/install.sh | sh
ollama --version
systemctl status ollama

The installer sets up a systemd service that listens on 127.0.0.1:11434. Read the script before you run it if you prefer, since it is a piped install.

Step 3: pull and test a model

ollama pull llama3.1:8b
ollama run llama3.1:8b "Explain what a VPS is in two sentences."
ollama ps

ollama ps shows whether the model is loaded on the GPU. If it says CPU, the GPU is not being used and replies will be slow.

Step 4: add Open WebUI

Run Open WebUI in Docker and point it at Ollama on the host:

docker run -d --name open-webui --restart unless-stopped \
  --add-host=host.docker.internal:host-gateway \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  -v open-webui:/app/backend/data \
  -p 127.0.0.1:3000:8080 \
  ghcr.io/open-webui/open-webui:main

Binding to 127.0.0.1 means it is not exposed to the internet yet. Create the admin account the first time you open it.

Step 5: put HTTPS in front

Use Caddy for automatic certificates. Point a domain at the server, then:

sudo apt install -y caddy
echo 'chat.example.com {
  reverse_proxy 127.0.0.1:3000
}' | sudo tee /etc/caddy/Caddyfile
sudo systemctl reload caddy

In Open WebUI settings, disable open sign-ups so only you can create accounts.

Keep it private

• Never expose port 11434 (the Ollama API) to the internet. It has no login.
• Use the firewall to allow only SSH, 80 and 443. See securing a new VPS.
• Keep the models and chat history on the server's disk. They do not leave the machine.