iptables setup for two different lxc containers inside the same host


 
Thread Tools Search this Thread
Special Forums IP Networking iptables setup for two different lxc containers inside the same host
# 1  
Old 05-17-2017
Debian iptables setup for two different lxc containers inside the same host

Hello out there. A month ago I started to deal with this problem and until now I couldn't cope with it. The quick story is that I'm trying to setup two different lxc containers inside the same host machine running debian linux. One of the containers is running a VPN server, while the second one is running a WEB server. Both of them are connected to the internet through a bridge (br0) interface on the host. In order to route the traffic between these two containers I used iptables.

Let's now see this procedure more closely and step by step.

SETUP THE FIRST CONTAINER (VPN)

@HOST [ /etc/network/interfaces ]

Code:
# interfaces(5) file used by ifup(8) and ifdown(8)
# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp

# Bridge interface
auto br0
iface br0 inet static
address 192.168.1.1
netmask 255.255.255.0
broadcast 192.168.1.255
bridge_ports none
bridge_fd 2.0
bridge_maxwait 1

@HOST [ /var/lib/lxc/VPN/config ]

Code:
# Template used to create this container: /usr/share/lxc/templates/lxc-debian
# Parameters passed to the template: -r jessie
# For additional config options, please look at lxc.container.conf(5)
#lxc.network.type = empty
lxc.rootfs = /var/lib/lxc/VPN/rootfs

# Common configuration
lxc.include = /usr/share/lxc/config/debian.common.conf

# Container specific configuration
lxc.mount = /var/lib/lxc/VPN/fstab
lxc.utsname = VPN
lxc.arch = amd64
lxc.autodev = 1
lxc.kmsg = 0
lxc.start.auto = 1

lxc.network.type = veth
lxc.network.veth.pair = vethVPN
lxc.network.name = veth0
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:42:1d:a7
lxc.network.link = br0

## for openvpn
lxc.mount.entry = /dev/net dev/net none bind,create=dir
lxc.cgroup.devices.allow = c 10:200 rwm


@GUEST VPN [ /etc/network/interfaces ]

Code:
auto lo
iface lo inet loopback

#auto eth0
#iface eth0 inet dhcp

auto veth0
iface veth0 inet static
address 192.168.1.2
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 192.168.1.1

After some more configuration inside the container for the vpn server (not interesting for this post) i add the following iptables rules in the HOST machine:

Code:
# Generated by iptables-save v1.4.21 on Fri Apr 28 16:07:58 2017
*filter
:INPUT ACCEPT [1189211:150089991]
:FORWARD ACCEPT [902865:826112449]
:OUTPUT ACCEPT [1324099:212970374]
COMMIT
# Completed on Fri Apr 28 16:07:58 2017
# Generated by iptables-save v1.4.21 on Fri Apr 28 16:07:58 2017
*nat
:PREROUTING ACCEPT [36:1998]
:INPUT ACCEPT [17:858]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A PREROUTING -p udp -m udp --dport 1194 -j DNAT --to-destination 192.168.1.2:1194
-A POSTROUTING -o eth0 -j MASQUERADE
COMMIT

Until that step everything is working as expected. VPN container can ping the outside world, can apt-get update correctly and VPN clients find their way to the outside world as expected.

The next step was to add another container for the WEB server.

@HOST [ /var/lib/lxc/WEB/config ]

Code:
# Template used to create this container: /usr/share/lxc/templates/lxc-debian
# Parameters passed to the template: -r jessie
# For additional config options, please look at lxc.container.conf(5)
#lxc.network.type = empty
lxc.rootfs = /var/lib/lxc/WEB/rootfs

# Common configuration
lxc.include = /usr/share/lxc/config/debian.common.conf

# Container specific configuration
lxc.mount = /var/lib/lxc/WEB/fstab
lxc.utsname = WEB
lxc.arch = amd64
lxc.autodev = 1
lxc.kmsg = 0
lxc.start.auto = 0

# Network config
lxc.network.type = veth
lxc.network.veth.pair = vethWEB
lxc.network.name = veth0
lxc.network.flags = up
lxc.network.hwaddr = 00:14:2e:42:1d:a7
lxc.network.link = br0

## for openvpn
lxc.mount.entry = /dev/net dev/net none bind,create=dir
lxc.cgroup.devices.allow = c 10:200 rwm

@GUEST [ /etc/network/interfaces ]

Code:
auto lo
iface lo inet loopback

auto veth0
iface veth0 inet static
address 192.168.1.4
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 192.168.1.1

For http traffic to be routed in the WEB container I add the following iptables rule

Code:
iptables -t nat -A PREROUTING -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.1.4:80

and thus now the final rules are :

Code:
# Generated by iptables-save v1.4.21 on Wed May 17 08:13:33 2017
*nat
:PREROUTING ACCEPT [5132:301425]
:INPUT ACCEPT [5124:300824]
:OUTPUT ACCEPT [95:6546]
:POSTROUTING ACCEPT [55:3052]
-A PREROUTING -p udp -m udp --dport 1194 -j DNAT --to-destination 192.168.1.2:1194
-A PREROUTING -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.1.4:80
-A POSTROUTING -o eth0 -j MASQUERADE
COMMIT
# Completed on Wed May 17 08:13:33 2017
# Generated by iptables-save v1.4.21 on Wed May 17 08:13:33 2017
*filter
:INPUT ACCEPT [1227003:153631244]
:FORWARD ACCEPT [3344156:3204894200]
:OUTPUT ACCEPT [1377802:229812203]
COMMIT

This is where problems start.

1. containers can ping the outside world
Code:
PING www.google.com (216.58.208.228) 56(84) bytes of data.
64 bytes from par10s22-in-f228.1e100.net (216.58.208.228): icmp_seq=1 ttl=52 time=12.4 ms

2. containers can not apt-get update
Code:
Err http://http.debian.net jessie InRelease         
  
Err http://http.debian.net jessie Release.gpg       
  Cannot initiate the connection to http.debian.net:80 (2605:bc80:3010:b00:0:deb:166:202). - connect (101: Network is unreachable) [IP: 2605:bc80:3010:b00:0:deb:166:202 80]
Reading package lists... Done
W: Failed to fetch http://http.debian.net/debian/dists/jessie/InRelease  

W: Failed to fetch http://http.debian.net/debian/dists/jessie/Release.gpg  Cannot initiate the connection to http.debian.net:80 (2605:bc80:3010:b00:0:deb:166:202). - connect (101: Network is unreachable) [IP: 2605:bc80:3010:b00:0:deb:166:202 80]

W: Some index files failed to download. They have been ignored, or old ones used instead.

3. vpn clients doesn't access internet properly. Some web sites doesn't load at all while others work perfectly.

It seems that there is a conflict in the http protocol traffic. If I delete the later iptables rule (for the WEB container) , container regain the ability for the apt-get update and vpn clients can access all the web sites. In that way the drawback is that I cannot access the web server from the outside world.

Finally some network info:

ifconfig @ HOST

Code:
br0       Link encap:Ethernet  HWaddr fe:cd:03:40:b8:ca  
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::2024:18ff:febf:2d13/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1939159 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1432506 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1631320323 (1.5 GiB)  TX bytes:1595149625 (1.4 GiB)

eth0      Link encap:Ethernet  HWaddr de:2b:44:3f:a0:03  
          inet addr:10.8.44.199  Bcast:10.255.255.255  Mask:255.255.255.254
          inet6 addr: 2001:bc8:4700:2300::9:1107/127 Scope:Global
          inet6 addr: fe80::dc2b:44ff:fe3f:a003/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2694541 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3348824 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1823342534 (1.6 GiB)  TX bytes:1909806405 (1.7 GiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:3 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:204 (204.0 B)  TX bytes:204 (204.0 B)

vethVPN   Link encap:Ethernet  HWaddr fe:cd:03:40:b8:ca  
          inet6 addr: fe80::fccd:3ff:fe40:b8ca/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1932533 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1423250 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1657633998 (1.5 GiB)  TX bytes:1582663302 (1.4 GiB)

vethWEB   Link encap:Ethernet  HWaddr fe:d2:6e:95:19:46  
          inet6 addr: fe80::fcd2:6eff:fe95:1946/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:937 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1273 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:135281 (132.1 KiB)  TX bytes:103741 (101.3 KiB)


route -n @ HOST

Code:
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.8.44.198     0.0.0.0         UG    0      0        0 eth0
10.8.44.198     0.0.0.0         255.255.255.254 U     0      0        0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 br0

brctl show @ HOST

Code:
bridge name	bridge id		STP enabled	interfaces
br0		8000.fecd0340b8ca	no		vethVPN
							vethWEB


Any idea/hint on how to fix this routing problem will be very thankful because I cannot think anything else to try and my mind is going to burn out.

Thank you.

P.S Something that I just realized is that the error from the apt-get update command show that it tries to communicate with ipv6 protocol and not ipv4. Is that weird ?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Proxy Server

Installation JDK in the LXC container - Host behind Proxy

Hello, The host on which i work is a remote machine behind a proxy. I have installed the jdk like that and it works well: (I access the remote machine via ssh) http_proxy=http://proxy:3128 && https_proxy=http://proxy:3128 sudo apt-get install openjdk-7-jre-headless :~# java... (1 Reply)
Discussion started by: chercheur111
1 Replies

2. Ubuntu

Ping a container LXC

Hello, Please, I try to do ping 10.0.3.8 (ip of LXC container) from VMB but it didn't work ! Have you an idea please ? Here is an explanation of what I want do: (red arrow) http://imgur.com/2IzJvXO imgur: the simple image sharer Thanks a lot. Best Regards. (0 Replies)
Discussion started by: chercheur111
0 Replies

3. Shell Programming and Scripting

How to setup Oracle connection inside shell script?

Hi, We have Oracle Connection parameters set up in file name "TESTDB" at location /abc/etc.When I try to run my shell script it does not connect to Oracle database. Please let me know how "TESTDB" file can be called inside script. ####################### Setting the directories... (2 Replies)
Discussion started by: sandy162
2 Replies

4. Red Hat

Static IP Address setup for vm as well as the host system

Hello, Greetings!! I have a server with 3 TB of disk space and 12 GB RAM and a i7 processor. What I did thus far is to install Oracle Enterprise Linux (OEL 5.7)as the host system and install Oracle Virtual box and created 3 VM's. Installed OEL 5.7 on one of the VM, working on installing... (1 Reply)
Discussion started by: rparavastu
1 Replies

5. IP Networking

Remote printer setup on solaris 10 host

Hi All, I am trying setup a remote printer on a solaris 10 server. The printer is online and working fine is solaris 8. I have added the pritner to /etc/hosts file and /etc/printers.conf ... I need to know followings: 1. How do I find the default pritner port on the old solaris machine... (0 Replies)
Discussion started by: mohullah
0 Replies

6. Shell Programming and Scripting

Script to check if host key authentication is setup

Hey all, I have a script that I use for some automated installs. Unfortunately for the script to work the server that it's running from needs to have host-key authentication setup to the target server. If it isn't setup beforehand and the script is executed the install partially completes and... (1 Reply)
Discussion started by: Rike255
1 Replies

7. UNIX for Advanced & Expert Users

Help! How to find the local host after few ssh hops to remote host???

I do a ssh to remote host(A1) from local host(L1). I then ssh to another remote(A2) from A1. When I do a who -m from A2, I see the "connected from" as "A1". => who -m userid pts/2 2010-03-27 08:47 (A1) I want to identify who is the local host who initiated the connection to... (3 Replies)
Discussion started by: gomes1333
3 Replies

8. Cybersecurity

Help needed in IPTables firewall/router setup - Linux

HI all, I have setup IPTables firewall/Router and my home network, with address space 192.168.10.XXX Form my private network hosts, i can ping the gateway ( 192.168.10.101 ) , but the reverse is not happening. Can someone help me as of what i need to do, so that i can ping my private... (1 Reply)
Discussion started by: chandan_m
1 Replies

9. AIX

NIM server setup on etherchannel setup environment

I know that IBM's official stance is that NIM does not work on etherchannel environment, but has anyone able to get around it? I'm working on a p5-590 LPAR system, and the NIM master and clients are all on the same frame. Any help is appreciated. (1 Reply)
Discussion started by: pdtak
1 Replies

10. UNIX for Dummies Questions & Answers

local host setup

I am running Redhat 7.2. I am using a router with dhcp setup for my computer. When logging in, I get the error "dhcppc1 not found". My router assigned that host name to my pc for dhcp. It says to modify the /etc/hosts file to keep from having any problems. Everything seems to be working ok... (2 Replies)
Discussion started by: jeremiebarber
2 Replies
Login or Register to Ask a Question