ClusterIP Service
Learn what a ClusterIP Service is in Kubernetes, how it works, and how to expose applications internally within the cluster for secure communication between Pods.
Table of Contents
- What is a ClusterIP Service?
- Why is ClusterIP Used?
- How Does ClusterIP Work?
- Where is ClusterIP Used?
- ClusterIP vs Other Service Types
- Creating a ClusterIP Service
- Real-World Example: Frontend ↔ Backend Communication
- Troubleshooting ClusterIP Pods — Real-Time Examples
- Best Practices
- Summary
What is a ClusterIP Service?
A ClusterIP is the default and most fundamental Service type in Kubernetes. It exposes a set of Pods on a stable, internal IP address that is only reachable within the cluster. No external traffic can reach a ClusterIP service directly — it is designed purely for internal pod-to-pod communication.
When you create a Service without specifying a type, Kubernetes automatically defaults to ClusterIP.
ClusterIP = Virtual IP Address (internal only) assigned to a Service
Key Characteristics
| Property | Value |
|---|---|
| Scope | Internal (cluster-only) |
| Default Type | Yes — default when type is omitted |
| Stable IP | Yes — persists even if backing Pods restart |
| DNS Name | Yes — <service-name>.<namespace>.svc.cluster.local |
| Load Balancing | Yes — distributes across matching Pods |
| External Access | ❌ Not directly accessible outside cluster |
The Problem ClusterIP Solves
Pods in Kubernetes are ephemeral. They get created, die, and are replaced constantly. Each new Pod gets a new IP address. If Service A needs to talk to Service B, hardcoding Pod IPs would break every time a Pod restarts.
ClusterIP solves this by acting as a stable virtual front door to a group of Pods, regardless of how many times the Pods behind it are replaced.
Without ClusterIP: With ClusterIP:
Pod A → Pod B (10.0.1.5) Pod A → ClusterIP (10.96.0.10)
Pod B dies, new IP! ↓
Pod A → ??? BROKEN kube-proxy routes to healthy Pods
Why is ClusterIP Used?
1. Service Discovery Inside the Cluster
Microservices architectures require services to find and talk to each other reliably. ClusterIP provides a stable DNS entry and IP that never changes, even if the Pods behind it are replaced.
2. Load Balancing Across Pods
ClusterIP distributes incoming traffic across all healthy Pods matching the Service’s label selector using iptables or IPVS rules managed by kube-proxy.
3. Decoupling Consumers from Producers
The frontend doesn’t need to know how many backend Pods exist or where they are. It just calls the ClusterIP service name, and Kubernetes handles routing.
4. Security Boundary
Since ClusterIP is not exposed externally, it naturally prevents external access to sensitive internal services such as databases, caches, internal APIs, and message queues.
5. Foundation for Other Service Types
Both NodePort and LoadBalancer services are built on top of ClusterIP. Every NodePort and LoadBalancer service also has an internal ClusterIP assigned.
How Does ClusterIP Work?
Understanding ClusterIP requires understanding three core components: kube-proxy, iptables/IPVS, and CoreDNS.
Architecture Overview
┌─────────────────────────────────────────────────────────────┐
│ Kubernetes Cluster │
│ │
│ ┌──────────┐ ┌─────────────────┐ │
│ │ Pod A │──────▶│ ClusterIP Svc │ │
│ │(consumer)│ │ 10.96.45.200 │ │
│ └──────────┘ │ port: 80 │ │
│ └────────┬────────┘ │
│ │ kube-proxy (iptables) │
│ ┌──────────┼──────────┐ │
│ ▼ ▼ ▼ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Pod B-1 │ │ Pod B-2 │ │ Pod B-3 │ │
│ │10.244.1.2│ │10.244.2.5│ │10.244.3.8│ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
Step-by-Step Traffic Flow
DNS Resolution
Pod A wants to reach Service B. It callshttp://backend-service(or the full FQDNbackend-service.default.svc.cluster.local).
CoreDNS resolves this to the ClusterIP, e.g.,10.96.45.200.Packet Interception by kube-proxy
The packet with destination10.96.45.200:80leaves Pod A and hits the node’s network interface.kube-proxyhas pre-programmediptablesrules that intercept this packet.DNAT (Destination NAT)
iptablesapplies a DNAT rule, randomly selecting one of the healthy backend Pod IPs (e.g.,10.244.2.5:8080) and rewrites the destination.Delivery
The packet is routed to the selected Pod directly via the cluster network (CNI plugin like Calico, Flannel, etc.).Response Path
The response is reverse-NATted back so Pod A sees the reply coming from10.96.45.200, maintaining the abstraction.
Under the Hood: iptables Rules
You can inspect the actual rules kube-proxy creates:
# SSH into a node, then:
sudo iptables -t nat -L KUBE-SERVICES -n | grep <service-ip>
# Example output:
# KUBE-SVC-XXXX tcp -- 0.0.0.0/0 10.96.45.200 tcp dpt:80
CoreDNS and Service Discovery
Every Service gets a DNS A record automatically:
<service-name>.<namespace>.svc.cluster.local
For a service named backend in namespace default:
backend.default.svc.cluster.local → 10.96.45.200
Pods in the same namespace can use just backend.
Pods in other namespaces must use backend.default or the full FQDN.
Where is ClusterIP Used?
ClusterIP is the backbone of internal Kubernetes communication. Here are the most common real-world use cases:
1. Database Services
App Pods → ClusterIP (postgres-service) → PostgreSQL Pods
You never want your database directly exposed. ClusterIP keeps it internal and provides a stable endpoint for your application.
2. Cache Layers (Redis, Memcached)
API Pods → ClusterIP (redis-service) → Redis Pods
Internal cache services are classic ClusterIP use cases.
3. Microservices Communication
order-service → ClusterIP (payment-service) → payment-service Pods
payment-service → ClusterIP (notification-service) → notification Pods
4. Internal APIs and gRPC Services
Backend microservices that should never be exposed publicly communicate via ClusterIP.
5. Message Broker Internal Endpoints
Producer Pods → ClusterIP (kafka-broker) → Kafka Pods
6. Monitoring and Metrics Collection
Prometheus scrapes metrics from other services via their ClusterIP addresses.
ClusterIP vs Other Service Types
| Feature | ClusterIP | NodePort | LoadBalancer | ExternalName |
|---|---|---|---|---|
| External Access | ❌ | ✅ (via Node IP) | ✅ (via LB IP) | ✅ (DNS redirect) |
| Internal Access | ✅ | ✅ | ✅ | ✅ |
| Use Case | Internal services | Dev/testing exposure | Production external | External DNS alias |
| Cost | Free | Free | Cloud LB cost | Free |
| Stable IP | ✅ Internal | ✅ Internal | ✅ External | N/A |
| Built on ClusterIP | — | ✅ Yes | ✅ Yes | N/A |
Creating a ClusterIP Service
Method 1: YAML Manifest (Recommended)
Step 1: Deploy a Sample Backend Application
# backend-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend
namespace: default
labels:
app: backend
spec:
replicas: 3
selector:
matchLabels:
app: backend
template:
metadata:
labels:
app: backend
spec:
containers:
- name: backend
image: nginx:1.25
ports:
- containerPort: 80
resources:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "200m"
memory: "256Mi"
Apply it:
kubectl apply -f backend-deployment.yaml
Step 2: Create the ClusterIP Service
# backend-clusterip-service.yaml
apiVersion: v1
kind: Service
metadata:
name: backend-service
namespace: default
labels:
app: backend
spec:
type: ClusterIP # This is also the default; can be omitted
selector:
app: backend # Must match Pod labels in the Deployment
ports:
- name: http
protocol: TCP
port: 80 # Port the Service listens on (inside cluster)
targetPort: 80 # Port the container is listening on
Apply it:
kubectl apply -f backend-clusterip-service.yaml
Step 3: Verify the Service
kubectl get svc backend-service
# Output:
# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
# backend-service ClusterIP 10.96.45.200 <none> 80/TCP 10s
Note: EXTERNAL-IP is <none> — confirming this is internal only.
Step 4: Inspect Service Details
kubectl describe svc backend-service
# Output:
# Name: backend-service
# Namespace: default
# Selector: app=backend
# Type: ClusterIP
# IP: 10.96.45.200
# Port: http 80/TCP
# TargetPort: 80/TCP
# Endpoints: 10.244.1.2:80,10.244.2.5:80,10.244.3.8:80
# Session Affinity: None
The Endpoints field shows the actual Pod IPs behind the service. This is crucial for troubleshooting.
Method 2: Imperative Command
# Expose an existing deployment as a ClusterIP service
kubectl expose deployment backend \
--name=backend-service \
--port=80 \
--target-port=80 \
--type=ClusterIP
Method 3: Headless ClusterIP (No Virtual IP)
A headless service (ClusterIP: None) is used when you need to directly address individual Pods (e.g., StatefulSets like databases):
apiVersion: v1
kind: Service
metadata:
name: backend-headless
spec:
clusterIP: None # Makes it headless — no virtual IP assigned
selector:
app: backend
ports:
- port: 80
targetPort: 80
With a headless service, DNS returns the individual Pod IPs instead of a single virtual IP.
Real-World Example: Frontend ↔ Backend Communication
Let’s walk through a complete, realistic scenario of a frontend web application communicating with a backend API using ClusterIP.
Scenario
We have:
- A React frontend serving static files via Nginx
- A Node.js backend API handling business logic
- A PostgreSQL database for data persistence
The frontend talks to the backend via ClusterIP. The backend talks to PostgreSQL via ClusterIP. Neither the backend API nor the database is exposed externally.
Architecture
Internet
│
▼
[LoadBalancer]
│
▼
[Frontend Service - NodePort/LB]
│
▼
[Frontend Pods - React/Nginx]
│
│ HTTP → backend-api-service (ClusterIP)
▼
[Backend API Pods - Node.js]
│
│ TCP → postgres-service (ClusterIP)
▼
[PostgreSQL Pods]
Full Manifests
# 1. Backend API Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend-api
namespace: production
spec:
replicas: 3
selector:
matchLabels:
app: backend-api
template:
metadata:
labels:
app: backend-api
spec:
containers:
- name: api
image: mycompany/backend-api:v2.1.0
ports:
- containerPort: 3000
env:
- name: DB_HOST
value: "postgres-service" # Calls the ClusterIP service by DNS name
- name: DB_PORT
value: "5432"
---
# 2. Backend ClusterIP Service
apiVersion: v1
kind: Service
metadata:
name: backend-api-service
namespace: production
spec:
type: ClusterIP
selector:
app: backend-api
ports:
- port: 3000
targetPort: 3000
---
# 3. PostgreSQL Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres
namespace: production
spec:
replicas: 1
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: postgres:15
ports:
- containerPort: 5432
env:
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgres-secret
key: password
---
# 4. PostgreSQL ClusterIP Service
apiVersion: v1
kind: Service
metadata:
name: postgres-service
namespace: production
spec:
type: ClusterIP
selector:
app: postgres
ports:
- port: 5432
targetPort: 5432
Testing Internal Connectivity
# Run a temporary debug pod to test connectivity
kubectl run debug-pod --image=busybox -it --rm --restart=Never -- sh
# Inside the pod, test DNS resolution
nslookup backend-api-service.production.svc.cluster.local
# Expected: returns 10.96.x.x (ClusterIP)
# Test HTTP connectivity
wget -qO- http://backend-api-service.production.svc.cluster.local:3000/health
# Expected: {"status": "ok"}
# Test from same namespace (short name works)
wget -qO- http://backend-api-service:3000/health
Troubleshooting ClusterIP Pods — Real-Time Examples
This section covers the most common issues with ClusterIP services and how to diagnose and fix them systematically.
Troubleshooting Framework
Symptom: Service unreachable
│
▼
Is the Service created?
kubectl get svc
│
├── No → Create the service
│
▼
Does the Service have Endpoints?
kubectl describe svc <name>
│
├── No endpoints → Label selector mismatch
│
▼
Are the Pods Running?
kubectl get pods -l app=<name>
│
├── Not Running → Pod crash/scheduling issue
│
▼
Can you reach the Pod directly?
kubectl exec -it <pod> -- curl localhost:<port>
│
├── No → App not listening on correct port
│
▼
Can another Pod reach the service?
kubectl run test --image=busybox -it --rm -- wget <service>
│
├── No → kube-proxy / CNI issue
Problem 1: “No Endpoints” — Label Selector Mismatch
Symptom:
kubectl describe svc backend-service
# Endpoints: <none>
Cause: The Service’s selector labels don’t match the Pod’s labels.
Real-Time Diagnosis:
# Check service selector
kubectl get svc backend-service -o jsonpath='{.spec.selector}'
# Output: {"app":"backend"}
# Check pod labels
kubectl get pods --show-labels
# NAME LABELS
# backend-6d4f9c-xk2p9 app=backend-api,version=v2 ← MISMATCH!
# The service looks for app=backend but pods have app=backend-api
Fix:
# Option A: Fix the Service selector
spec:
selector:
app: backend-api # Match the actual pod label
# Option B: Fix the Pod label
template:
metadata:
labels:
app: backend # Match the service selector
Apply and verify:
kubectl apply -f backend-clusterip-service.yaml
kubectl describe svc backend-service
# Endpoints: 10.244.1.2:80,10.244.2.5:80 ← Now showing endpoints!
Problem 2: Service Exists but Connection Refused
Symptom:
curl http://backend-service:80
# curl: (7) Failed to connect to backend-service port 80: Connection refused
Cause: The container is listening on a different port than what targetPort specifies.
Real-Time Diagnosis:
# Step 1: Check what port the service targets
kubectl get svc backend-service -o yaml | grep targetPort
# targetPort: 80
# Step 2: Check what port the container actually listens on
kubectl exec -it backend-6d4f9c-xk2p9 -- netstat -tlnp
# Or:
kubectl exec -it backend-6d4f9c-xk2p9 -- ss -tlnp
# Output:
# LISTEN 0 128 0.0.0.0:8080 ... ← App is on 8080, not 80!
# Step 3: Check container port definition in Deployment
kubectl get deployment backend -o jsonpath='{.spec.template.spec.containers[0].ports}'
# [{"containerPort":8080}]
Fix:
# Update the Service targetPort to match the actual container port
spec:
ports:
- port: 80 # Service port (what callers use)
targetPort: 8080 # Actual container port
kubectl apply -f backend-clusterip-service.yaml
# Verify
kubectl describe svc backend-service
# Port: 80/TCP → TargetPort: 8080/TCP
Problem 3: Pod CrashLoopBackOff Behind ClusterIP Service
Symptom:
kubectl get pods
# NAME READY STATUS RESTARTS AGE
# backend-6d4f9c-xk2p9 0/1 CrashLoopBackOff 5 3m
Even though the Service exists, it has no healthy Pods to route to.
Real-Time Diagnosis:
# Step 1: Check pod events
kubectl describe pod backend-6d4f9c-xk2p9
# Events:
# Warning BackOff 2m kubelet Back-off restarting failed container
# Step 2: Check logs (current)
kubectl logs backend-6d4f9c-xk2p9
# Step 3: Check logs (previous container, before crash)
kubectl logs backend-6d4f9c-xk2p9 --previous
# Output:
# Error: Cannot connect to database: postgres-service:5432 - ECONNREFUSED
# ← The backend can't reach the database!
# Step 4: Check if the database service exists
kubectl get svc postgres-service
# Error from server (NotFound): services "postgres-service" not found
# ← Database service was never created!
# Step 5: Check if postgres pods are running
kubectl get pods -l app=postgres
# No resources found.
# ← Database deployment is missing too!
Fix:
# Deploy the missing database and its service
kubectl apply -f postgres-deployment.yaml
kubectl apply -f postgres-service.yaml
# Wait for postgres to be ready
kubectl wait --for=condition=ready pod -l app=postgres --timeout=60s
# Restart the backend pods to pick up the new connectivity
kubectl rollout restart deployment/backend
# Verify all pods are running
kubectl get pods
# NAME READY STATUS RESTARTS AGE
# backend-7c9f8d-mn3k1 1/1 Running 0 30s
# postgres-5b6c7d-pq9r2 1/1 Running 0 45s
Problem 4: Intermittent Connection Failures (Readiness Probe Issue)
Symptom: Sometimes requests succeed, sometimes they fail with connection errors. Very inconsistent.
Cause: Pods are included in Service Endpoints before they are actually ready to serve traffic. No readiness probe is configured.
Real-Time Diagnosis:
# Check if readiness probes are configured
kubectl get deployment backend -o jsonpath='{.spec.template.spec.containers[0].readinessProbe}'
# null ← No readiness probe!
# Check endpoint churn
kubectl get endpoints backend-service -w
# NAME ENDPOINTS AGE
# backend-service 10.244.1.2:80,10.244.2.5:80,10.244.3.8:80 5m
# backend-service 10.244.1.2:80,10.244.2.5:80 5m10s
# backend-service 10.244.1.2:80,10.244.2.5:80,10.244.3.8:80 5m20s
# ← Endpoints are flapping: pods added/removed erratically
Fix:
# Add readiness and liveness probes to your Deployment
spec:
template:
spec:
containers:
- name: backend
image: mycompany/backend-api:v2.1.0
readinessProbe:
httpGet:
path: /health
port: 3000
initialDelaySeconds: 10 # Wait 10s before first check
periodSeconds: 5 # Check every 5s
failureThreshold: 3 # Mark unready after 3 failures
livenessProbe:
httpGet:
path: /health
port: 3000
initialDelaySeconds: 30
periodSeconds: 10
failureThreshold: 3
kubectl apply -f backend-deployment.yaml
kubectl rollout status deployment/backend
# Now pods are only added to Endpoints when /health returns 200
kubectl get endpoints backend-service
# NAME ENDPOINTS AGE
# backend-service 10.244.1.2:80,10.244.2.5:80 2m
# (only healthy, ready pods appear here)
Problem 5: DNS Resolution Failure
Symptom:
kubectl exec -it frontend-pod -- curl http://backend-service
# curl: (6) Could not resolve host: backend-service
Real-Time Diagnosis:
# Step 1: Check if CoreDNS is running
kubectl get pods -n kube-system -l k8s-app=kube-dns
# NAME READY STATUS RESTARTS AGE
# coredns-5d78c9869d-7xqn2 0/1 Pending 0 10m ← CoreDNS not running!
# Step 2: Describe the CoreDNS pod for events
kubectl describe pod coredns-5d78c9869d-7xqn2 -n kube-system
# Events:
# Warning FailedScheduling No nodes available to schedule pods
# Step 3: Try with full FQDN and IP to isolate DNS vs connectivity
kubectl exec -it frontend-pod -- nslookup backend-service.default.svc.cluster.local
# ;; connection timed out; no servers could be reached ← DNS server unreachable
# Step 4: Check the pod's DNS config
kubectl exec -it frontend-pod -- cat /etc/resolv.conf
# nameserver 10.96.0.10
# search default.svc.cluster.local svc.cluster.local cluster.local
# Step 5: Try reaching CoreDNS directly
kubectl exec -it frontend-pod -- nslookup backend-service 10.96.0.10
Fix (if CoreDNS is Pending):
# Check node taints that might block CoreDNS scheduling
kubectl describe nodes | grep Taint
# If control-plane taint is blocking (common in single-node setups):
kubectl taint nodes --all node-role.kubernetes.io/control-plane-
# Restart CoreDNS
kubectl rollout restart deployment/coredns -n kube-system
kubectl wait --for=condition=ready pod -l k8s-app=kube-dns -n kube-system --timeout=60s
# Re-test DNS
kubectl exec -it frontend-pod -- nslookup backend-service
Problem 6: Wrong Namespace — Cross-Namespace Communication Failure
Symptom: Service exists, pods are healthy, but the call fails.
Real-Time Diagnosis:
# Service is in "production" namespace
kubectl get svc -n production backend-api-service
# NAME TYPE CLUSTER-IP PORT(S)
# backend-api-service ClusterIP 10.96.45.200 3000/TCP
# But the frontend pod is in "staging" namespace
kubectl get pods -n staging -l app=frontend
# NAME READY STATUS RESTARTS
# frontend-abc123 1/1 Running 0
# Calling just "backend-api-service" from staging fails because
# it looks for the service in the "staging" namespace, not "production"
kubectl exec -it frontend-abc123 -n staging -- curl http://backend-api-service:3000
# curl: (6) Could not resolve host: backend-api-service
Fix:
# Use the full FQDN with namespace
kubectl exec -it frontend-abc123 -n staging -- \
curl http://backend-api-service.production.svc.cluster.local:3000
# {"status": "ok"} ← Works!
Or update the frontend configuration:
# In frontend deployment env vars:
env:
- name: API_URL
# Short name (only works within same namespace):
value: "http://backend-api-service:3000"
# Cross-namespace (always use full FQDN):
value: "http://backend-api-service.production.svc.cluster.local:3000"
Quick Troubleshooting Cheat Sheet
# === INSPECT ===
kubectl get svc <name> # Service basic info
kubectl describe svc <name> # Full details + Endpoints
kubectl get endpoints <name> # Raw endpoint list
kubectl get pods -l <selector> --show-labels # Check pod labels
# === CONNECTIVITY TEST ===
# Run a throwaway debug pod
kubectl run debug \
--image=nicolaka/netshoot \
-it --rm --restart=Never \
-- bash
# Inside debug pod:
nslookup <service-name> # DNS resolution
curl http://<service-name>:<port>/health # HTTP test
nc -zv <service-name> <port> # TCP port test
wget -qO- http://<service-name>:<port> # Alternative HTTP test
# === LOGS ===
kubectl logs <pod-name> # Current logs
kubectl logs <pod-name> --previous # Logs before last crash
kubectl logs -l app=<label> --all-containers # Logs for all pods with label
# === EVENTS ===
kubectl get events --sort-by='.lastTimestamp' # All recent events
kubectl describe pod <pod-name> # Pod-specific events
# === NETWORK DEBUG ===
kubectl exec -it <pod> -- netstat -tlnp # What ports container listens on
kubectl exec -it <pod> -- cat /etc/resolv.conf # DNS config inside pod
sudo iptables -t nat -L KUBE-SERVICES -n # iptables rules (on node)
Best Practices
1. Always Use Named Ports
# ✅ Good — named ports make targetPort references more resilient
ports:
- name: http
port: 80
targetPort: http # References the containerPort name, not number
# In Deployment:
ports:
- name: http
containerPort: 8080
2. Use Specific Label Selectors
# ❌ Too broad — could accidentally match unrelated pods
selector:
app: backend
# ✅ More specific — reduces risk of mismatches
selector:
app: backend
component: api
version: v2
3. Always Configure Readiness Probes
Never run production workloads without readiness probes. They prevent traffic from reaching unready Pods through the ClusterIP service.
4. Use Namespaces for Isolation
Keep services in dedicated namespaces and use full FQDNs for cross-namespace communication. This avoids accidental service name collisions.
5. Document Your Service Topology
For complex microservices, maintain a diagram showing which services communicate via which ClusterIP services. This dramatically speeds up debugging.
6. Monitor Endpoint Health
Set up alerts for Services with 0 endpoints:
# Quick check script
kubectl get endpoints -A | awk '$3 == "<none>" {print "⚠️ No endpoints:", $1, $2}'
Summary
| Concept | Key Takeaway |
|---|---|
| What is ClusterIP | A stable internal virtual IP for a group of Pods |
| Default type | Yes — used when type: is omitted in a Service spec |
| Scope | Internal cluster only — no external access |
| DNS | <svc>.<ns>.svc.cluster.local automatically created |
| How it works | kube-proxy uses iptables/IPVS to route & load balance |
| Common use | Databases, internal APIs, caches, microservice-to-microservice |
| Key troubleshooting | Check labels, endpoints, pod logs, readiness probes, and DNS |
ClusterIP is the foundational building block of Kubernetes networking. Mastering it — understanding how traffic flows, how DNS resolves service names, and how to diagnose connectivity issues — is essential before moving on to NodePort, LoadBalancer, or Ingress resources.
Next Up: 02 - NodePort Service → — Learn how to expose services outside the cluster using node-level ports.