Hi,
I have weird issue relating to usage of ER-X as openVPN client and now after a few frustrating days investigation as alone, I need your eyes as help to find out that I don't see to cause problem. Issue is that I have two ER-X routers (ouluGW and kamppiGW) with almost identical configurations (two different physical locations), both are connecting as openVPN client to same openVPN server (Linux server as third location), but LAN machines (subnet: 192.168.10.0/24) behind of second ER-X (kamppiGW) are not able to reach subnet behind of openVPN server, but LAN machines (subnet: 192.168.35.10/24) behind of "ouluGW" ER-X are able to reach that subnet via openVPN server.
Image as description of network's topolocy:
![aatmanet2.png aatmanet2.png]()
In image seen IPSEC site-to-site tunnel between ERXs is working.
Here is configuration, routing etc details as background information of setup:
- ER-X configuration of ouluGW (with that ER-X all is working correctly), please see attachment file with name: oulugw_ERX.txt
- ER-X configuration of kamppiGW (LAN machines behind of this router at 192.168.10.0/24 subnet are not able to reach 192.168.25.0/24 subnet via openVPN server), please see attachment file with name: kamppigw_ERX.txt
- Routes at ouluGW relating to vtun0 interface:
S *> 192.168.20.0/24 [1/0] via 192.168.20.13, vtun0
C *> 192.168.20.13/32 is directly connected, vtun0
C *> 192.168.20.14/32 is directly connected, vtun0
S *> 192.168.25.0/24 [1/0] via 192.168.20.13, vtun0
- Routes at kamppiGW relating to vtun0 interface:
S *> 192.168.20.0/24 [1/0] via 192.168.20.17, vtun0
C *> 192.168.20.17/32 is directly connected, vtun0
C *> 192.168.20.18/32 is directly connected, vtun0
S *> 192.168.25.0/24 [1/0] via 192.168.20.17, vtun0
- OpenVPN server (APOLLO) has "192.168.20.0/24" as subnet where from ER-Xs as VPN clients get IP-address and "192.168.25.0/24" as subnet where this "APOLLO" server is providing some network services what LAN machines behind of "ouluGW" and "kamppiGW" routers are utilizing.
-- Relating routes at APOLLO server:
192.168.20.2 dev tun0 proto kernel scope link src 192.168.20.1
192.168.10.0/24 via 192.168.20.2 dev tun0
192.168.20.0/24 via 192.168.20.2 dev tun0
192.168.35.0/24 via 192.168.20.2 dev tun0
192.168.25.0/24 dev eth0 proto kernel scope link src 192.168.25.10
-- Configuration of openVPN server:
port 1194
proto udp
dev tun
ca ca.crt
cert apollo.crt
key apollo.key
dh dh1024.pem
server 192.168.20.0 255.255.255.0
ifconfig-pool-persist "/etc/openvpn/ipp.txt" 10
client-config-dir "/etc/openvpn/ccd"
route 192.168.10.0 255.255.255.0
route 192.168.35.0 255.255.255.0
client-to-client
keepalive 10 60
tls-auth ta.key
cipher DES-EDE3-CBC
comp-lzo
max-clients 10
user nobody
group nogroup
persist-key
persist-tun
status "/etc/openvpn/openvpn-status.log"
management /var/run/openvpn.mgmt unix
log "/var/log/openvpn.log"
log-append "/var/log/openvpn_append.log"
verb 6
plugin /usr/lib/openvpn/openvpn-plugin-auth-pam.so login
-- Both VPN clients are using same configuration where only differences are inlined KEY and CERT details per client:
client
dev tun
proto udp
remote x.x.x.x 1194
resolv-retry infinite
remote-random
nobind
tun-mtu 1500
tun-mtu-extra 32
mssfix 1450
persist-key
persist-tun
ping 15
ping-restart 0
ping-timer-rem
reneg-sec 86400
remote-cert-tls server
auth-user-pass /config/openvpn/apollo_vpnauth.txt
comp-lzo
verb 4
route-nopull
fast-io
cipher DES-EDE3-CBC
keysize 192<ca></ca>
key-direction 1
<tls-auth></tls-auth><key></key><cert></cert>
-- OpenVPN server has CCD file per client to define IP and subnet what is existing behind of client (as information for openVPN server):
ouluGW:
ifconfig-push 192.168.20.14 192.168.20.13
iroute 192.168.35.0 255.255.255.0
kamppiGW:
fconfig-push 192.168.20.18 192.168.20.17
iroute 192.168.10.0 255.255.255.0
Okey, if I start from working setup (ouluGW):
- When I ping from LAN machine at "192.168.35.0/24" subnet to IP-address at "192.168.25.0/24" (for example: 192.168.25.10) at APOLLO server, I get response correctly as is able to seen from below tcpdumps, packages from LAN machine at IP-addres 192.168.35.102 to 192.168.25.10 goes correctly and return correctly:
# TCPDUMP from workstation at 192.168.35.0/24 network point of view looking for eth0 interface:
13:21:35.152225 IP 192.168.35.102 > 192.168.25.10: ICMP echo request, id 6044, seq 1, length 64
13:21:35.210125 IP 192.168.25.10 > 192.168.35.102: ICMP echo reply, id 6044, seq 1, length 64
# TCPDUMP from APOLLO's point of view looking for tun0 interface:
14:18:41.386466 IP 192.168.35.102 > 192.168.25.10: ICMP echo request, id 6031, seq 1, length 64
14:18:41.386494 IP 192.168.25.10 > 192.168.35.102: ICMP echo reply, id 6031, seq 1, length 64
- When I ping from ER-X (ouluGW with VPN IP: 192.168.20.14) to 192.168.25.10 at APOLLO, packages leave correctly via "vtun0" interface at ER-X and "tun0" at APOLLO get then in and return route is also OK:
#TCPDUMP from ouluGW point of view for vtun0 interface:
15:24:16.860084 IP 192.168.20.14 > 192.168.25.10: ICMP echo request, id 11320, seq 1, length 64
15:24:16.917294 IP 192.168.25.10 > 192.168.20.14: ICMP echo reply, id 11320, seq 1, length 64
# TCPDUMP from APOLLO's point of view looking for tun0 interface:
14:19:54.441171 IP 192.168.20.14 > 192.168.25.10: ICMP echo request, id 10983, seq 1, length 64
14:19:54.441199 IP 192.168.25.10 > 192.168.20.14: ICMP echo reply, id 10983, seq 1, length 64
- I can ping from APOLLO to "192.168.35.0/24" subnet behind of "ouluGW" ER-X as well as "ouluGW" itself:
# TCPDUMP from APOLLO's point of view looking:
18:22:58.927844 IP 192.168.20.1 > 192.168.35.102: ICMP echo request, id 10296, seq 11, length 64
18:22:58.985811 IP 192.168.35.102 > 192.168.20.1: ICMP echo reply, id 10296, seq 11, length 64
# TCPDUMP from ouluGW point of view looking:
19:25:29.870730 IP 192.168.20.1 > 192.168.35.102: ICMP echo request, id 10310, seq 1, length 64
19:25:29.871248 IP 192.168.35.102 > 192.168.20.1: ICMP echo reply, id 10310, seq 1, length 64
# TCPDUMP from 192.168.35.0/24 subnet machine point of view looking:
17:26:41.724282 IP 192.168.20.1 > 192.168.35.102: ICMP echo request, id 10311, seq 1, length 64
17:26:41.724306 IP 192.168.35.102 > 192.168.20.1: ICMP echo reply, id 10311, seq 1, length 64
Then what doesn't for some reason will works and I just cannot understand and see, why (kamppiGW):
- When I ping from LAN machine at "192.168.10.0/24" subnet to IP-address at "192.168.25.0/24" subnet at APOLLO server, I can see packages to leave via vtun0 interface of ER-X and arriving to APOLLO via tun0 interface as ICMP request, but ICMP reply won't go out from APOLLO:
# Ping from 192.168.10.51 to 192.168.25.10 from APOLLO's point of view looking:
09:30:14.871439 IP 192.168.10.51 > 192.168.25.10: ICMP echo request, id 1, seq 380, length 40
09:30:19.599280 IP 192.168.10.51 > 192.168.25.10: ICMP echo request, id 1, seq 381, length 40
# From kamppiGW point of view looking:
10:32:27.384204 IP 192.168.10.51 > 192.168.25.10: ICMP echo request, id 1, seq 388, length 40
10:32:32.063683 IP 192.168.10.51 > 192.168.25.10: ICMP echo request, id 1, seq 389, length 40
- But I am able to ping APOLLO from kamppiGW and able to ping that LAN machine at "192.168.10.0/24" subnet from APOLLO:
# ping from APOLLO to LAN machine from APOLLO's point of view:
10:06:10.769927 IP 192.168.20.1 > 192.168.10.51: ICMP echo request, id 8541, seq 14, length 64
10:06:10.828172 IP 192.168.10.51 > 192.168.20.1: ICMP echo reply, id 8541, seq 14, length 64
# ping from APOLLO to LAN machine from kamppiGW point of view:
11:04:41.121912 IP 192.168.20.1 > 192.168.10.51: ICMP echo request, id 8489, seq 17, length 64
11:04:41.123171 IP 192.168.10.51 > 192.168.20.1: ICMP echo reply, id 8489, seq 17, length 64
I was able to see packages at LAN machine with Wireshark (this one is windows 10 machine)!
# Ping from APOLLO to kamppiGW from apollo's point of view:
09:33:53.298899 IP 192.168.20.10 > 192.168.25.18: ICMP echo request, id 11629, seq 1, length 64
09:33:53.298931 IP 192.168.25.18 > 192.168.20.10: ICMP echo reply, id 11629, seq 1, length 64
# Ping from APOLLO to kamppiGW from kamppiGW's point of view:
10:35:33.842667 IP 192.168.20.10 > 192.168.25.18: ICMP echo request, id 11702, seq 1, length 64
10:35:33.897732 IP 192.168.25.18 > 192.168.20.10: ICMP echo reply, id 11702, seq 1, length 64
- There is no any firewall rules to block/reject traffic IN or OUT at APOLLO server relating to subnet 192.168.10.0/24.
So, any ideas what is reason, why LAN machine at 192.168.10.0/24 subnet is not able to reach subnet 192.168.25.0/24?