服务无法与k8s集群中的Rabbitmq通信

服务无法与k8s集群中的Rabbitmq通信

问题描述:

我有k8s集群(1.16),在它的特定名称空间中有一个服务,以及一组有状态的Rabbitmq(图像:3.8.0-alpine),其中有3个副本,每个工人上都有一个副本,在不同的名称空间中. 我将服务配置为将Rabbitmq与amqp://user:password@rabbitmq-service.rabbitmq-namespace:5672

I have k8s cluster (1.16) with one service in it's specific namespace, and a stateful set of rabbitmq (image: 3.8.0-alpine) with 3 replicas, one on each of my workers, in a different namespace. I configured the service to connect rabbitmq with amqp://user:password@rabbitmq-service.rabbitmq-namespace:5672

我的服务日志中出现此错误:

I'm getting this error in the logs of my service:

Rabbit mq disconnected connect EHOSTUNREACH <rabbitmq_svc_ip>:5672

rabbitmq似乎工作正常.窗格中没有显示错误.

rabbitmq seems to be working fine. there are no errors shown in the pods.

我该怎么检查才能发现问题并与他们进行沟通?

What can I check to find the issue and make them communicate?

您询问了如何解决此问题,因此我将介绍如何解决此问题.

You asked for how to approach this issue, and so I am including how I would approach it.

我将描述您的服务,以查看您的Pod ip是否列为其后面的端点.

I would describe your service to see if your pod ips are listed as endpoints behind it.

kubectl describe service <rabbitmq-service>

如果未列出它们,那么我将查看您服务的选择器,以确保它与您的状态集匹配.如果列出了它们,那么我将再次检查您的服务端口是否确实是5672,然后我将在执行您的应用程序的Pod中执行并尝试在连接字符串中ping服务名称,以确保可以从以下位置访问该服务名称您应用的名称空间.我还将尝试对您的服务名称和端口进行telnet测试,以确保您的应用程序可以在该端口上连接到您的服务,并且我将通过telnet到您的Rabbitmq Pod IP和端口,以确保您的应用程序可以与它们连接.如果您的映像中没有telnet,我将其安装以进行故障排除.

If they are not listed, then I would look at your service's selector to make sure that it matches your statefulset. If they are listed, then I would double check that your Service Port is actually 5672, and then I would exec into your pod where your app is running and try to ping the service name in your connection string to make sure it's reachable from your app's namespace. I would also try to do telnet tests to your service name and port to ensure that your app can connect to your service on that port, and I would telnet to your rabbitmq pod ips and ports to ensure that your app can connect to them. If telnet isn't in your image, I would install it for troubleshooting purposes.

kubectl exec -it <app-pod> -n <namespace> /bin/bash

telnet <pod ip or service name> <port>