KeepAlived VRID mismatch
VRID : virtual_router_id
/var/log/messages에 아래 로그가 출력되기 시작함.
Keepalived_vrrp[xxxx]: ip address associated with VRID not present in received packet : 192.168.17.100 Keepalived_vrrp[xxxx]: one or more VIP associated with VRID mismatch actual MASTER advert Keepalived_vrrp[xxxx]: bogus VRRP packet received on eth0 !!! Keepalived_vrrp[xxxx]: VRRP_Instance(VI_1) Dropping received VRRP packet...
Openstack의 경우 컨트롤 노드의 /var/log/messages 에 아래처럼 로그가 출력됨
Jul 10 17:28:35 openstackctrl-02 docker[3922]: Mon Jul 10 17:28:35 2023: (kolla_internal_vip_51) ip address associated with VRID 51 not present in MASTER advert : 10.8.6.34 Jul 10 17:28:36 openstackctrl-02 docker[3922]: Mon Jul 10 17:28:36 2023: (kolla_internal_vip_51) Entering MASTER STATE Jul 10 17:28:36 openstackctrl-02 docker[3922]: Mon Jul 10 17:28:36 2023: (kolla_internal_vip_51) Master received advert from 10.8.6.33 with higher priority 3, ours 2 Jul 10 17:28:36 openstackctrl-02 docker[3922]: Mon Jul 10 17:28:36 2023: (kolla_internal_vip_51) Entering BACKUP STATE Jul 10 17:28:37 openstackctrl-02 docker[3922]: Mon Jul 10 17:28:37 2023: (kolla_internal_vip_51) ip address associated with VRID 51 not present in MASTER advert : 10.8.6.34 Jul 10 17:28:38 openstackctrl-02 docker[3922]: Mon Jul 10 17:28:38 2023: (kolla_internal_vip_51) ip address associated with VRID 51 not present in MASTER advert : 10.8.6.34 Jul 10 17:28:39 openstackctrl-02 docker[3922]: Mon Jul 10 17:28:39 2023: (kolla_internal_vip_51) ip address associated with VRID 51 not present in MASTER advert : 10.8.6.34
위 로그는 openstackctrl-02 에서 해당 오픈스택의 VIP인 10.8.6.34 VRID 51에 대한 MASTER의 advert를 제대로 수신하지 못했다는 로그이다.
하지만 분명 다른 컨트롤 노드에 해당 VIP가 동작하고 있는것이 확인된 경우 보통 같은 네트워크상에 동일한 VRID의 다른 VIP가 동작하고 있기 때문이다.
동일한 VRID 를 사용하는 KeepAlived 가 같은 네트워크에 존재하고 있어서 발생
(VIP 가 다르고 VRID 가 동일한 상황)
VIP 192.168.17.100 -> VRID 20 VIP 192.168.17.150 -> VRID 20
KeepAlived 가 동작하는 서버에서 tcpdump 를 이용해서 vrid 확인
# tcpdump -i <interface> vrrp 또는 # tcpdump -i <interface> host 224.0.0.18
# tcpdump -i eth0 host 224.0.0.18 10:59:10.697847 IP 192.168.17.100 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 20, prio 100, authtype simple, intvl 1s, length 20 10:59:10.698571 IP 192.168.17.150 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 20, prio 100, authtype simple, intvl 1s, length 20
Openstack의 경우
[root@openstackctrl-01]# tcpdump -i bond0.1006 host 224.0.0.18 dropped privs to tcpdump tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on bond0.1006, link-type EN10MB (Ethernet), snapshot length 262144 bytes 15:29:24.906726 IP openstackctrl-03 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 3, authtype simple, intvl 1s, length 24 15:29:25.906839 IP openstackctrl-03 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 3, authtype simple, intvl 1s, length 24 15:29:26.907203 IP openstackctrl-03 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 3, authtype simple, intvl 1s, length 24 15:29:26.907326 IP 10.8.6.100 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 1, authtype simple, intvl 1s, length 24 15:29:27.907617 IP 10.8.6.100 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 1, authtype simple, intvl 1s, length 24 15:29:27.907653 IP openstackctrl-03 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 3, authtype simple, intvl 1s, length 24 15:29:28.907970 IP openstackctrl-03 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 3, authtype simple, intvl 1s, length 24 15:29:28.908015 IP 10.8.6.100 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 1, authtype simple, intvl 1s, length 24 15:29:29.908275 IP 10.8.6.100 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 1, authtype simple, intvl 1s, length 24
위에서 보는것처럼 openstackctrl-03에서 vrid 51으로 vrrp를 advert하고 있는데 바로 아래쪽에 동일하게 10.8.6.100에서도 같은 vrid 51로 advert하는것을 알 수 있다.
즉 동일한 vrid 51로 서로 다른 VIP를 advert 하고 있어 이를 수신하는 다른 노드에서 오류가 발생하는 것이다.
IP Multicast address 참고 [https://en.wikipedia.org/wiki/Multicast_address#IPv4]224.0.0.18 Virtual Router Redundancy Protocol (VRRP)
keepalived.conf 내에 vrid 값을 중복되지 않는 값으로 변경 후 KeepAlived 데몬 restart
''virtual_router_id 21''
Openstack의 경우 kolla-ansible 배포 인 경우 globals.yml의 keepalived_virtual_router_id 값을 변경하여 재배포 한다.
keepalived_virtual_router_id: "51"
VRID 값은 0..255 사이의 값을 사용한다.