Open WebUI + LiteLLM + Vertex AI (Claude Opus 4.7) on GCP

Architecture

User → Open WebUI (Cloud Run) → LiteLLM (Cloud Run) → Vertex AI → Claude Opus 4.7
              ↓
        GCS Bucket (persistence for Open WebUI data + LiteLLM config)

Step 1 — Enable APIs

Run in Cloud Shell (>_ icon in GCP console top bar):

gcloud config set project YOUR_PROJECT_ID

gcloud services enable \
  storage.googleapis.com \
  run.googleapis.com \
  aiplatform.googleapis.com \
  artifactregistry.googleapis.com \
  cloudbuild.googleapis.com

Step 2 — Enable Claude Opus 4.7 in Model Garden

Manual step — cannot be done via CLI.

1. Go to console.cloud.google.com/vertex-ai/model-garden
2. Search Claude
3. Open Claude Opus 4.7
4. Click Enable and accept Anthropic’s terms

Step 3 — Create GCS Bucket

gcloud storage buckets create gs://YOUR_BUCKET_NAME \
  --location=us-east1 \
  --uniform-bucket-level-access

Bucket names are globally unique across all of GCP. Use something like openwebui-data-YOUR_PROJECT_ID.

Step 4 — Create Service Account

gcloud iam service-accounts create openwebui-sa \
  --display-name="OpenWebUI SA"

Grant permissions:

# Vertex AI access (for LiteLLM)
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
  --member="serviceAccount:openwebui-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/aiplatform.user"

# GCS access (for Open WebUI persistence + LiteLLM config)
gcloud storage buckets add-iam-policy-binding gs://YOUR_BUCKET_NAME \
  --member="serviceAccount:openwebui-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/storage.objectAdmin"

Step 5 — Create and Upload LiteLLM Config

cat > config.yaml << 'EOF'
model_list:
  - model_name: claude-opus-4-7
    litellm_params:
      model: vertex_ai/claude-opus-4-7
      vertex_project: YOUR_PROJECT_ID
      vertex_location: us-east5

litellm_settings:
  drop_params: true
EOF

gcloud storage cp config.yaml gs://YOUR_BUCKET_NAME/litellm-config.yaml

Step 6 — Copy LiteLLM Image to Artifact Registry

Cloud Run GUI only accepts images from GCP’s own registries. Use Cloud Build to copy from ghcr.io.

First create the Artifact Registry repository:

gcloud artifacts repositories create openwebui-repo \
  --repository-format=docker \
  --location=us-east1

Then create the build config and submit:

cat > cloudbuild.yaml << 'EOF'
steps:
  - name: 'gcr.io/cloud-builders/docker'
    args: ['pull', 'ghcr.io/berriai/litellm:main-latest']
  - name: 'gcr.io/cloud-builders/docker'
    args: ['tag', 'ghcr.io/berriai/litellm:main-latest', 'us-east1-docker.pkg.dev/YOUR_PROJECT_ID/openwebui-repo/litellm:latest']
images:
  - 'us-east1-docker.pkg.dev/YOUR_PROJECT_ID/openwebui-repo/litellm:latest'
EOF

gcloud builds submit --no-source --config cloudbuild.yaml

Takes ~3-5 minutes. LiteLLM image will be at:

us-east1-docker.pkg.dev/YOUR_PROJECT_ID/openwebui-repo/litellm:latest

Step 7 — Deploy LiteLLM on Cloud Run (GUI)

Go to Cloud Run → Create Service.

Container settings:

Settings tab — Container command and arguments:

In the GUI, each argument is a separate tag. Press Enter after typing --config, then type the path and press Enter again.

Variables & Secrets tab: (none needed)

Volumes tab:

The GUI mounts the entire bucket — do not specify a path in bucket. The config file will be accessible at /mnt/litellm/litellm-config.yaml.

Service account: openwebui-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com

After deploy, copy the LiteLLM service URL — looks like https://litellm-xxxx-uc.a.run.app.

Step 8 — Deploy Open WebUI on Cloud Run (GUI)

Enter openwebui/open-webui as the container link.

Go to Cloud Run → Create Service.

Container settings:

Variables & Secrets tab:

Volumes tab:

Service account: openwebui-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com

Step 9 — Connect in Open WebUI

Open WebUI auto-connects to LiteLLM via the env vars on first boot.

1. Open your Open WebUI Cloud Run URL
2. Create an admin account on first launch
3. Select claude-opus-4-7 from the model dropdown
4. Start chatting

Quick Reference

APIs to enable

• storage.googleapis.com
• run.googleapis.com
• aiplatform.googleapis.com
• artifactregistry.googleapis.com
• cloudbuild.googleapis.com

Image URLs (after Cloud Build)

• LiteLLM: us-east1-docker.pkg.dev/YOUR_PROJECT_ID/openwebui-repo/litellm:latest
• Open WebUI: us-east1-docker.pkg.dev/YOUR_PROJECT_ID/openwebui-repo/open-webui:latest

Ports

• LiteLLM: 4000
• Open WebUI: 8080

GCS bucket layout

gs://YOUR_BUCKET_NAME/
├── litellm-config.yaml   ← LiteLLM config
└── (Open WebUI data auto-created under /app/backend/data mount)