Creating and Alerting on Logs-based Metrics (GSP091)
Creating and Alerting on Logs-based Metrics (GSP091)
🔎 What You Learn in This Lab
You learn how to:
✅ Create a Log-based Alert
✅ Create a Log-based Metric
✅ Create a Metrics-based Alert
✅ Deploy GKE + Application
✅ Generate errors to trigger alerts
✅ Monitor incidents in Cloud Monitoring
🧠 Important Concepts (Very Clear Understanding)
1️⃣ Cloud Logging
Collects logs from:
VM instances
GKE clusters
Applications
Google services
2️⃣ Log-Based Metrics
Two types:
| Type | Who Creates | Description |
|---|---|---|
| System-defined | Auto-created from logs | |
| User-defined | You | Custom filter-based metrics |
3️⃣ Alerts
Two types used in lab:
| Alert Type | Based On |
|---|---|
| Log-based Alert | Directly from log query |
| Metrics-based Alert | From a log-based metric |
🛠️ LAB PRACTICAL STEPS (Step-by-Step)
✅ Task 1: Deploy GKE Cluster
Set Zone
gcloud config set compute/zone us-central1-a
Set Project
export PROJECT_ID=$(gcloud info --format='value(config.project)')
Create Cluster
gcloud container clusters create gmp-cluster --num-nodes=1 --zone us-central1-a
Wait until:
STATUS: RUNNING
✅ Task 2: Create Log-Based Alert (VM Stop Alert)
Go to:
Search → Logs Explorer
Use Query:
resource.type="gce_instance"
protoPayload.methodName="v1.compute.instances.stop"
Click Create Log Alert
Fill:
Alert name →
stopped vmNotification frequency → 5 minutes
Auto close → 1 hour
Add Email notification
Save.
🧪 Test It
Go to:
Navigation → Compute Engine → VM Instances
Stop instance1
Then go to:
Monitoring → Alerting → See incident
You should see alert triggered.
✅ Task 3: Create Docker Repository
Create Artifact Registry Repository
gcloud artifacts repositories create docker-repo \
--repository-format=docker \
--location=us-central1 \
--description="Docker repository" \
--project=$PROJECT_ID
Load Pre-built Image
wget https://storage.googleapis.com/spls/gsp1024/flask_telemetry.zip
unzip flask_telemetry.zip
docker load -i flask_telemetry.tar
Tag Image
docker tag gcr.io/ops-demo-330920/flask_telemetry:61a2a7aabc7077ef474eb24f4b69faeab47deed9 \
us-central1-docker.pkg.dev/$PROJECT_ID/docker-repo/flask-telemetry:v1
Push Image
docker push us-central1-docker.pkg.dev/$PROJECT_ID/docker-repo/flask-telemetry:v1
✅ Task 4: Deploy App That Emits Metrics
Authenticate Cluster
gcloud container clusters get-credentials gmp-cluster
Create Namespace
kubectl create ns gmp-test
Download Setup
wget https://storage.googleapis.com/spls/gsp1024/gmp_prom_setup.zip
unzip gmp_prom_setup.zip
cd gmp_prom_setup
Edit Image Name
Open file:
nano flask_deployment.yaml
Replace:
<ARTIFACT REGISTRY IMAGE NAME>
With:
us-central1-docker.pkg.dev/$PROJECT_ID/docker-repo/flask-telemetry:v1
Save.
Deploy App
kubectl -n gmp-test apply -f flask_deployment.yaml
kubectl -n gmp-test apply -f flask_service.yaml
Get External IP
kubectl get services -n gmp-test
Check Metrics Endpoint
curl $(kubectl get services -n gmp-test -o jsonpath='{.items[*].status.loadBalancer.ingress[0].ip}')/metrics
You should see:
flask_exporter_info
✅ Task 5: Create Log-Based Metric
Go to:
Logs Explorer → Create Metric
Settings:
Metric Type → Counter
Name →
hello-app-error
Filter:
severity=ERROR
resource.labels.container_name="hello-app"
textPayload:"ERROR: 404 Error page not found"
Click Create.
✅ Task 6: Create Metrics-Based Alert
Go to:
Logging → Log-based Metrics
Find:
hello-app-error
Click:
More Actions → Create Alert
Change:
Rolling window → 2 min
Add notification
Name →
log based metric alert
Create Policy.
✅ Task 7: Generate Errors (Trigger Alert)
Run:
timeout 120 bash -c -- 'while true; do curl \((kubectl get services -n gmp-test -o jsonpath='{.items[*].status.loadBalancer.ingress[0].ip}')/error; sleep \)((RANDOM % 4)) ; done'
Check:
Logs:
Logs Explorer → Severity → Error
Monitoring:
Monitoring → Alerting → Incidents
You should see 2 alerts triggered.
🎯 What You Achieved
✔ Created GKE cluster
✔ Created log-based alert
✔ Created user-defined metric
✔ Created metric-based alert
✔ Generated error logs
✔ Triggered alert
✔ Viewed incident details
🧾 Interview Notes (Very Important)
What is Log-based Metric?
A metric created from log filters to count occurrences of specific log entries.
Difference Between Log Alert & Metric Alert?
| Log Alert | Metric Alert |
|---|---|
| Direct log match | Based on metric value |
| Instant detection | Aggregated window-based |
Why Use Log-based Metrics?
Track error frequency
Monitor application health
Trigger automated response
Detect security issues
