iptables setup for two different lxc containers inside the same host
Hello out there. A month ago I started to deal with this problem and until now I couldn't cope with it. The quick story is that I'm trying to setup two different lxc containers inside the same host machine running debian linux. One of the containers is running a VPN server, while the second one is running a WEB server. Both of them are connected to the internet through a bridge (br0) interface on the host. In order to route the traffic between these two containers I used iptables.
Let's now see this procedure more closely and step by step.
SETUP THE FIRST CONTAINER (VPN)
@HOST [ /etc/network/interfaces ]
Code:
# interfaces(5) file used by ifup(8) and ifdown(8)
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet dhcp
# Bridge interface
auto br0
iface br0 inet static
address 192.168.1.1
netmask 255.255.255.0
broadcast 192.168.1.255
bridge_ports none
bridge_fd 2.0
bridge_maxwait 1
@HOST [ /var/lib/lxc/VPN/config ]
Code:
# Template used to create this container: /usr/share/lxc/templates/lxc-debian
# Parameters passed to the template: -r jessie
# For additional config options, please look at lxc.container.conf(5)
#lxc.network.type = empty
lxc.rootfs = /var/lib/lxc/VPN/rootfs
# Common configuration
lxc.include = /usr/share/lxc/config/debian.common.conf
# Container specific configuration
lxc.mount = /var/lib/lxc/VPN/fstab
lxc.utsname = VPN
lxc.arch = amd64
lxc.autodev = 1
lxc.kmsg = 0
lxc.start.auto = 1
lxc.network.type = veth
lxc.network.veth.pair = vethVPN
lxc.network.name = veth0
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:42:1d:a7
lxc.network.link = br0
## for openvpn
lxc.mount.entry = /dev/net dev/net none bind,create=dir
lxc.cgroup.devices.allow = c 10:200 rwm
@GUEST VPN [ /etc/network/interfaces ]
Code:
auto lo
iface lo inet loopback
#auto eth0
#iface eth0 inet dhcp
auto veth0
iface veth0 inet static
address 192.168.1.2
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 192.168.1.1
After some more configuration inside the container for the vpn server (not interesting for this post) i add the following iptables rules in the HOST machine:
Until that step everything is working as expected. VPN container can ping the outside world, can apt-get update correctly and VPN clients find their way to the outside world as expected.
The next step was to add another container for the WEB server.
@HOST [ /var/lib/lxc/WEB/config ]
Code:
# Template used to create this container: /usr/share/lxc/templates/lxc-debian
# Parameters passed to the template: -r jessie
# For additional config options, please look at lxc.container.conf(5)
#lxc.network.type = empty
lxc.rootfs = /var/lib/lxc/WEB/rootfs
# Common configuration
lxc.include = /usr/share/lxc/config/debian.common.conf
# Container specific configuration
lxc.mount = /var/lib/lxc/WEB/fstab
lxc.utsname = WEB
lxc.arch = amd64
lxc.autodev = 1
lxc.kmsg = 0
lxc.start.auto = 0
# Network config
lxc.network.type = veth
lxc.network.veth.pair = vethWEB
lxc.network.name = veth0
lxc.network.flags = up
lxc.network.hwaddr = 00:14:2e:42:1d:a7
lxc.network.link = br0
## for openvpn
lxc.mount.entry = /dev/net dev/net none bind,create=dir
lxc.cgroup.devices.allow = c 10:200 rwm
@GUEST [ /etc/network/interfaces ]
Code:
auto lo
iface lo inet loopback
auto veth0
iface veth0 inet static
address 192.168.1.4
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 192.168.1.1
For http traffic to be routed in the WEB container I add the following iptables rule
# Generated by iptables-save v1.4.21 on Wed May 17 08:13:33 2017
*nat
:PREROUTING ACCEPT [5132:301425]
:INPUT ACCEPT [5124:300824]
:OUTPUT ACCEPT [95:6546]
:POSTROUTING ACCEPT [55:3052]
-A PREROUTING -p udp -m udp --dport 1194 -j DNAT --to-destination 192.168.1.2:1194
-A PREROUTING -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.1.4:80
-A POSTROUTING -o eth0 -j MASQUERADE
COMMIT
# Completed on Wed May 17 08:13:33 2017
# Generated by iptables-save v1.4.21 on Wed May 17 08:13:33 2017
*filter
:INPUT ACCEPT [1227003:153631244]
:FORWARD ACCEPT [3344156:3204894200]
:OUTPUT ACCEPT [1377802:229812203]
COMMIT
This is where problems start.
1. containers can ping the outside world
Code:
PING www.google.com (216.58.208.228) 56(84) bytes of data.
64 bytes from par10s22-in-f228.1e100.net (216.58.208.228): icmp_seq=1 ttl=52 time=12.4 ms
2. containers can not apt-get update
Code:
Err http://http.debian.net jessie InRelease
Err http://http.debian.net jessie Release.gpg
Cannot initiate the connection to http.debian.net:80 (2605:bc80:3010:b00:0:deb:166:202). - connect (101: Network is unreachable) [IP: 2605:bc80:3010:b00:0:deb:166:202 80]
Reading package lists... Done
W: Failed to fetch http://http.debian.net/debian/dists/jessie/InRelease
W: Failed to fetch http://http.debian.net/debian/dists/jessie/Release.gpg Cannot initiate the connection to http.debian.net:80 (2605:bc80:3010:b00:0:deb:166:202). - connect (101: Network is unreachable) [IP: 2605:bc80:3010:b00:0:deb:166:202 80]
W: Some index files failed to download. They have been ignored, or old ones used instead.
3. vpn clients doesn't access internet properly. Some web sites doesn't load at all while others work perfectly.
It seems that there is a conflict in the http protocol traffic. If I delete the later iptables rule (for the WEB container) , container regain the ability for the apt-get update and vpn clients can access all the web sites. In that way the drawback is that I cannot access the web server from the outside world.
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.8.44.198 0.0.0.0 UG 0 0 0 eth0
10.8.44.198 0.0.0.0 255.255.255.254 U 0 0 0 eth0
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br0
brctl show @ HOST
Code:
bridge name bridge id STP enabled interfaces
br0 8000.fecd0340b8ca no vethVPN
vethWEB
Any idea/hint on how to fix this routing problem will be very thankful because I cannot think anything else to try and my mind is going to burn out.
Thank you.
P.S Something that I just realized is that the error from the apt-get update command show that it tries to communicate with ipv6 protocol and not ipv4. Is that weird ?
Hello,
The host on which i work is a remote machine behind a proxy.
I have installed the jdk like that and it works well: (I access the remote machine via ssh)
http_proxy=http://proxy:3128 && https_proxy=http://proxy:3128
sudo apt-get install openjdk-7-jre-headless
:~# java... (1 Reply)
Hello,
Please, I try to do ping 10.0.3.8 (ip of LXC container) from VMB but it didn't work ! Have you an idea please ?
Here is an explanation of what I want do: (red arrow)
http://imgur.com/2IzJvXO
imgur: the simple image sharer
Thanks a lot.
Best Regards. (0 Replies)
Hi,
We have Oracle Connection parameters set up in file name "TESTDB" at location /abc/etc.When I try to run my shell script it does not connect to Oracle database. Please let me know how "TESTDB" file can be called inside script.
####################### Setting the directories... (2 Replies)
Hello,
Greetings!!
I have a server with 3 TB of disk space and 12 GB RAM and a i7 processor.
What I did thus far is to install Oracle Enterprise Linux (OEL 5.7)as the host system and install Oracle Virtual box and created 3 VM's. Installed OEL 5.7 on one of the VM, working on installing... (1 Reply)
Hi All,
I am trying setup a remote printer on a solaris 10 server. The printer is online and working fine is solaris 8. I have added the pritner to /etc/hosts file and /etc/printers.conf ... I need to know followings:
1. How do I find the default pritner port on the old solaris machine... (0 Replies)
Hey all,
I have a script that I use for some automated installs. Unfortunately for the script to work the server that it's running from needs to have host-key authentication setup to the target server. If it isn't setup beforehand and the script is executed the install partially completes and... (1 Reply)
I do a ssh to remote host(A1) from local host(L1). I then ssh to another remote(A2) from A1.
When I do a who -m from A2, I see the "connected from" as "A1".
=> who -m
userid pts/2 2010-03-27 08:47 (A1)
I want to identify who is the local host who initiated the connection to... (3 Replies)
HI all,
I have setup IPTables firewall/Router and my home network, with address space 192.168.10.XXX
Form my private network hosts, i can ping the gateway ( 192.168.10.101 ) , but the reverse is not happening.
Can someone help me as of what i need to do, so that i can ping my private... (1 Reply)
I know that IBM's official stance is that NIM does not work on etherchannel environment, but has anyone able to get around it?
I'm working on a p5-590 LPAR system, and the NIM master and clients are all on the same frame.
Any help is appreciated. (1 Reply)
I am running Redhat 7.2. I am using a router with dhcp setup for my computer. When logging in, I get the error "dhcppc1 not found". My router assigned that host name to my pc for dhcp. It says to modify the /etc/hosts file to keep from having any problems. Everything seems to be working ok... (2 Replies)