Hi everybody!
Before I spew out walls of text this is the overall problem:
I've got a strange performance issue that I can't pinpoint. Randomly, VLAN traffic out to the internet is very poor. Pings time-out almost 50%. It only seems to do this sometimes. I've seen perfect pings out to the internet for extended time periods, then very bad pings out to the internet for extended time periods. Only the traffic from VLAN is affected. From the same physical workstation pings not on specific VLAN (eg default traffic, VLAN 1) are perfect with no loss at the same time as VLAN pings show huge loss. How can this be?
I am very new to Ubiquiti these are the first products I've purchased, so I have no experience with the configuration options.
What the heck am I even doing? Now let me explain further. Walls of text incoming.
I work with Windows servers for a living. At home I have a workstation upon which I run virtualized test environments, sometimes with dozens of virtual machines (yes, I have a lot of RAM).
The workstation has an Intel i350-T2 dual port NIC. One of the NIC ports is for the OS, the other NIC port is dedicated to Hyper-V.
I have an EdgeRouter Lite with eth0=WAN0, eth1=WAN1, eth2=LAN.
I set up load-balancing of WAN0+WAN1 such that WAN1 is 'failover only'.
The EdgeRouter has a bunch of VLANS configured eth2.2, eth2.10, eth2.21, eth2.22, eth2.23.
I have added firewall modify rules to send eth2.2, eth2.21, eth2.22 and eth2.23 out WAN1 with no failover.
All normal traffic (untagged) and eth2.10 will go out WAN0 and if that fails it will go out WAN1.
The EdgeRouter eth2 (LAN) is connected to a ToughSwitch on Port1 which is a trunk port.
The ToughSwitch is then uplinked to a second ToughSwitch, both connected on Port8 which are trunk ports.
On the second ToughSwitch the access port the Hyper-V dedicated NIC is plugged into is also a trunk port.
On the workstation, OS traffic uses NIC Port 1 which is not tagged for any VLAN and will route out WAN0.
If I set a virtual machine NIC as VLAN10 the traffic will also route out WAN0.
I have created a fresh virtual machine for testing. 4 vCPU, 4GB RAM, 1 NIC.
Running pings from the host and the guest simultaneously show vastly different results. Host never drops a packet, while the guest drops almost half.
If I swap the virtual machine to another VLAN to test WAN1 the same problem occurs.
Bizzarely, it's not always like this. The first time it happened I dicked with the EdgeRouter and ToughSwitches for a while and suddenly things were great for a few days, so I thought I'd "fixed" it.
But today the issue is back again and I don't see anything worthy of change.