Deploy Agentic AI on GKE with ADK and Vertex AI

This tutorial demonstrates how to deploy and manage containerized agentic AI/ML applications using Google Kubernetes Engine (GKE) with the Google Agent Development Kit (ADK) and Vertex AI.

Overview

Key Components

Google Agent Development Kit (ADK)

ADK is a flexible and modular framework for developing and deploying AI agents:

• Model-agnostic: Works with Gemini and other models
• Deployment-independent: Deploy anywhere
• Framework compatible: Integrates with other AI frameworks

Vertex AI Integration

When consuming LLMs through the Vertex AI API:

• Model inference occurs on Google's managed infrastructure
• No GPU/TPU quotas needed in your GKE cluster
• Access to Gemini 2.0 Flash with 1M token context window

Gemini 2.0 Flash

Feature	Capability
Speed	Higher than previous models
Context Window	1M tokens
Tool Use	Built-in support
Multimodal	Text, code, and more

Objectives

1. Set up your Google Cloud environment
2. Build a container image for your agent
3. Deploy the agent to a GKE cluster
4. Test your deployed agent

Prerequisites

Before you begin, ensure familiarity with:

• GKE and Kubernetes
• Containerization with Docker
• Vertex AI

Step-by-Step Deployment

Step 1: Prepare Environment

# Set environment variables
gcloud config set project PROJECT_ID
export GOOGLE_CLOUD_LOCATION=REGION
export PROJECT_ID=PROJECT_ID
export GOOGLE_CLOUD_PROJECT=$PROJECT_ID
export WORKLOAD_POOL=$PROJECT_ID.svc.id.goog
export PROJECT_NUMBER=$(gcloud projects describe --format json $PROJECT_ID | jq -r ".projectNumber")

Step 2: Clone Sample Project

git clone https://github.com/GoogleCloudPlatform/kubernetes-engine-samples.git
cd kubernetes-engine-samples/ai-ml/adk-vertex

Step 3: Create GKE Cluster

Autopilot

gcloud container clusters create-auto CLUSTER_NAME \
    --location=$GOOGLE_CLOUD_LOCATION \
    --project=$PROJECT_ID

Standard

gcloud container clusters create CLUSTER_NAME \
    --location=$GOOGLE_CLOUD_LOCATION \
    --project=$PROJECT_ID \
    --release-channel=stable \
    --num-nodes=1 \
    --machine-type=e2-medium \
    --workload-pool=$PROJECT_ID.svc.id.goog

Step 4: Create Artifact Registry

gcloud artifacts repositories create adk-repo \
    --repository-format=docker \
    --location=$GOOGLE_CLOUD_LOCATION \
    --project=$PROJECT_ID

Step 5: Build and Push Container

export IMAGE_URL="${GOOGLE_CLOUD_LOCATION}-docker.pkg.dev/${PROJECT_ID}/adk-repo/adk-agent:latest"

gcloud builds submit \
    --tag "$IMAGE_URL" \
    --project="$PROJECT_ID" \
    app

Step 6: Configure Workload Identity

# Create IAM service account
gcloud iam service-accounts create vertex-sa \
    --project=$PROJECT_ID

# Grant Vertex AI access
gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member "serviceAccount:vertex-sa@$PROJECT_ID.iam.gserviceaccount.com" \
    --role "roles/aiplatform.user"

# Create Kubernetes service account
kubectl create serviceaccount vertex-sa

# Annotate KSA
kubectl annotate serviceaccount vertex-sa \
    iam.gke.io/gcp-service-account=vertex-sa@$PROJECT_ID.iam.gserviceaccount.com

# Grant Workload Identity permissions
gcloud iam service-accounts add-iam-policy-binding vertex-sa@$PROJECT_ID.iam.gserviceaccount.com \
    --role roles/iam.workloadIdentityUser \
    --member "serviceAccount:$PROJECT_ID.svc.id.goog[default/vertex-sa]"

Step 7: Deploy the Agent

Create agent-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: adk-agent-deployment
  labels:
    app: adk-agent
spec:
  replicas: 1
  selector:
    matchLabels:
      app: adk-agent
  template:
    metadata:
      labels:
        app: adk-agent
    spec:
      serviceAccountName: vertex-sa
      containers:
      - name: adk-agent
        image: IMAGE_URL
        ports:
        - containerPort: 8000
        env:
        - name: GOOGLE_CLOUD_PROJECT_ID
          value: PROJECT_ID
        - name: GOOGLE_CLOUD_LOCATION
          value: REGION
        - name: GOOGLE_GENAI_USE_VERTEXAI
          value: "1"
        - name: PORT
          value: "8000"
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "1Gi"
            cpu: "1"

Apply the deployment:

kubectl apply -f agent-deployment.yaml

Step 8: Expose the Agent

LoadBalancer Service

apiVersion: v1
kind: Service
metadata:
  name: adk-agent-service
spec:
  selector:
    app: adk-agent
  type: LoadBalancer
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8000

Port Forward

POD_NAME=$(kubectl get pods -l app=adk-agent -o jsonpath='{.items[0].metadata.name}')
kubectl port-forward $POD_NAME 8000:8000

Testing Your Agent

Create a Session

curl -X POST AGENT_BASE_URL/apps/capital-agent/users/user-123/sessions/session-123

Send a Query

curl -X POST AGENT_BASE_URL/run \
  -H "Content-Type: application/json" \
  -d '{
    "appName": "capital-agent",
    "userId": "user-123",
    "sessionId": "session-123",
    "newMessage": {
      "role": "user",
      "parts": [{
        "text": "Hello, agent! What can you do for me?"
      }]
    }
  }'

Application Structure

File	Purpose
main.py	FastAPI application entry point
agent.py	ADK agent logic with Vertex AI
__init__.py	Python package initialization
requirements.txt	Python dependencies
Dockerfile	Container image definition

Clean Up

# Delete cluster
gcloud container clusters delete CLUSTER_NAME \
    --location=${GOOGLE_CLOUD_LOCATION} \
    --project=$PROJECT_ID

# Remove IAM binding
gcloud projects remove-iam-policy-binding $PROJECT_ID \
    --member "serviceAccount:vertex-sa@$PROJECT_ID.iam.gserviceaccount.com" \
    --role "roles/aiplatform.user"

# Delete service account
gcloud iam service-accounts delete vertex-sa@$PROJECT_ID.iam.gserviceaccount.com

# Delete Artifact Registry
gcloud artifacts repositories delete adk-repo \
    --location=$GOOGLE_CLOUD_LOCATION \
    --project=$PROJECT_ID

What's Next

• Configure Horizontal Pod Autoscaler (HPA) for dynamic scaling
• Set up Identity-Aware Proxy (IAP) for secure access
• Use Cloud Logging and Monitoring for insights
• Explore experimental samples in GKE AI Labs