networking - GCP Alias IP Not reachable

Question

I have an alias IP range is 10.7.0.0/16

Configured VPC-native cluster with secondary range, so my pods/service can have the alias IP range

In GKE, there is a service that is using an alias IP range

? k get svc
NAME             TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
hasura-service   NodePort   10.7.165.27   <none>        80:30891/TCP   2d21h

---
apiVersion: v1
kind: Service
metadata:
  name: sura-service
  namespace: sura
  annotations:
    cloud.google.com/neg: '{"ingress": true}'
spec:
  selector:
    app: sura
  ports:
  - port: 80
    targetPort: 8080
  type: NodePort

The other instances in Project are not able to communicate with 10.7.165.27:80, Those instances are in the same subnetwork 10.152.0.0/20

Do I need to configure anything else to be able to use connect with an alias range from VPC itself?

question from:https://stackoverflow.com/questions/65830072/gcp-alias-ip-not-reachable

Answer 1

The annotation cloud.google.com/neg: '{"ingress": true}' is for ClusterIP, not for NodePort, meaning that said service is going to be exposed in 2 ways:

1. Behind the scenes is going to create a standalone ClusterIP which would only be accessible from other pods in the same subnet at 10.7.165.27

2. Is going to create among all the GKE nodes a forwarding rule iptables in port 30891/tcp at your GKE nodes subnet

In your current situation, the annotation is going to be accepted but is not going to do anything with it, even when you create a ClusterIP behind the scenes (point "1"), is not going to take effect because it's not the way it was designed for.

With that said, you have multiple paths on here.

a) You can leave it at that, and communicate with your GKE nodes directly at port 30891/tcp, I would only use this as a last resort since I personally don't like to handle random (or manually) selected ports, which in the long run could impact in the handling.

b) You can also communicate directly to your GKE pod IP address (assuming there's no GCP Firewall rule blocking the connection), I would also not use this since a GKE pod IP could change any given time, so keeping track of it would require some manual work to do, also you wouldn't have any "balancing" in case you have multiple pods providing the same service.

c) You can implement the following ClusterIP yaml file instead of your current one:

apiVersion: "v1"
kind: "Service"
metadata:
  name: "sura-service"
  annotations:
    cloud.google.com/neg: '{"exposed_ports": {"80":{}}}'
  namespace: "sura"
spec:
  ports:
  - protocol: "TCP"
    port: 80
    targetPort: 8080
  selector:
    app: "sura"
  type: "ClusterIP"

This will create a NEG, and you would need to point your connections to the GCP NEG internal IP is going to create, I would personally use this among all others if you are in need to use a GCP NEG for any reason.

d) You can also deploy an Internal TCP Load Balancer if you don't need to use NEGs like this:

apiVersion: "v1"
kind: "Service"
metadata:
  name: "sura-l4"
  namespace: "sura"
  annotations:
    cloud.google.com/load-balancer-type: "Internal"
spec:
  ports:
  - protocol: "TCP"
    port: 80
    targetPort: 8080
  selector:
    app: "sura"
  type: "LoadBalancer"
  loadBalancerIP: ""

Depending on what you are trying to achieve, you can combine this with NEGs and / or Global Access.