Why address on lo interface ?
Getting address on that interface is only used in case of DSR (direct server return) balancing, which haproxy does not do.
Haproxy is L3 and above, while DSR is L2.

Can you remove the lo:0 address entry from ALL servers (LB and mail servers) ?
In your case, VIP address is only on master haproxy node (one of two) with /24 mask (not on lo, and keepalived is handing that.

Also, configure the keepalived in the following manner, then retest :

Code:

vrrp_script check_haproxy {
script "/usr/bin/killall -0 haproxy" # be sure to check the availability of killall program or configure some other check, killall is cheap.
interval 2
weight 2
}
vrrp_instance VI_1 {
        state MASTER #
        interface <your network interface for VIP address>
        virtual_router_id 51
        priority 101
# VRRP VIP
virtual_ipaddress {
          10.2.0.4
}
authentication {
        auth_type PASS
        auth_pass <some password>
}

track_script {
        check_haproxy
}
}

Haproxy keeps monitoring accessibility of (mail) backend servers, and keepalived keeps monitoring if haproxy is up.
If that is what you need and i understood correctly.

Of course, you can add additional conditions to keepalived to execute failover of VIP address, after you confirm everything is working.

Hope that helps
Regards
Peasant.