I have an Avaya 5610SW Remote VPN H.323 IP phone that I have not been able to get working with a new EdgeRouter PoE 5 in a home office. The EdgeRouter is quite a step up from the consumer router we had been using. We are running EdgeMax v1.9.0. The phone appears to establish to the IPSec tunnel with the server, but gets stuck at "Discover x.x.x.x". The consumer routers had a VPN Passthrough option that was enabled, but I have not been able to replicate this option with the EdgeRouter. The issue does not appear to be port related because I do not see any dropped packets in the syslog after enabling logging for all rules and default actions. I also tried setting the default action to accept on the default firewall rules created by the wizards. The sip and h323 modules have also been disabled as suggested in other posts and I am now wondering if the issue could be NAT related, but not sure what the proper configuration should be. I tried creating a source nat exclude rule for traffic between the phone's IP and the server, but then the VPN stopped working on the phone. I have tried both the Basic Setup and WAN+2LAN2 wizards for the initial setup. Does anyone have Avaya H.323 VPN phones working with an EdgeRouter at a remote site?
When I view the tcpdump traffic using wireshark I see the following:
Source Destination Protocol Info
<Eth0IP>:2070 <ServerIP>:500 ISAKMP Aggressive
<ServerIP>:500 <Eth0IP>:2070 ISAKMP Aggressive
<Eth0IP>:4500 <ServerIP>:4500 ISAKMP Aggressive
<ServerIP>:4500 <Eth0IP>:4500 ESP ESP (SPI=0xb14f9f64)
<Eth0IP>:4500 <ServerIP>:4500 UDPENCAP NAT-keepalive
<ServerIP>:4500 <Eth0IP>:4500 ESP ESP (SPI=0xb14f9f64)
<Eth0IP>:4500 <ServerIP>:4500 UDPENCAP NAT-keepalive
<Eth0IP>:4500 <ServerIP>:4500 ESP ESP (SPI=0x651f84dd)
<Eth0IP>:4500 <ServerIP>:4500 ESP ESP (SPI=0x651f84dd)
<ServerIP>:4500 <Eth0IP>:4500 UDPENCAP NAT-keepalive
<Eth0IP>:4500 <ServerIP>:4500 ESP ESP (SPI=0x651f84dd)
<Eth0IP>:4500 <ServerIP>:4500 ESP ESP (SPI=0x651f84dd)
<Eth0IP>:4500 <ServerIP>:4500 ESP ESP (SPI=0x651f84dd)
<Eth0IP>:4500 <ServerIP>:4500 ESP ESP (SPI=0x651f84dd)
<Eth0IP>:4500 <ServerIP>:4500 ESP ESP (SPI=0x651f84dd)
<Eth0IP>:4500 <ServerIP>:4500 ESP ESP (SPI=0x651f84dd)
<Eth0IP>:4500 <ServerIP>:4500 ESP ESP (SPI=0x651f84dd)
<ServerIP>:4500 <Eth0IP>:4500 UDPENCAP NAT-keepalive
<Eth0IP>:4500 <ServerIP>:4500 ESP ESP (SPI=0x651f84dd)
<Eth0IP>:4500 <ServerIP>:4500 ESP ESP (SPI=0x651f84dd)
<Eth0IP>:4500 <ServerIP>:4500 ESP ESP (SPI=0x651f84dd)
<ServerIP>:4500 <Eth0IP>:4500 UDPENCAP NAT-keepalive
<Eth0> is 192.168.x.x obtained via DHCP from the ISP (with the source nat exclude rule enabled this IP is the phone's IP from Eth2)
<ServerIP> is a public IP address of the server