Anyone available that can glance at this dump to see if this looks like a hardware issue or something else? I believe the version is 1.9.1.1 but this crash has been happening randomly for awhile, finally got a console cable on it to catch the crash. This is an ERPro that is consistently moving around 700mbps or so.
Welcome to EdgeOS Router ttyS0
By logging in, accessing, or using the Ubiquiti product, you
acknowledge that you have read and understood the Ubiquiti
License Agreement (available in the Web UI at, by default,
http://192.168.1.1) and agree to be bound by its terms.
Router login: Unhandled kernel unaligned access[#1]:
CPU: 0 PID: 862 Comm: kworker/0:0 Tainted: P O 3.10.20-UBNT #1
Workqueue: events flow_cache_gc_task
task: 8000000089393600 ti: 800000007e56c000 task.ti: 800000007e56c000
$ 0 : 0000000000000000 0000000050109ce1 0000000000000000 ffffffffc04b3380
$ 4 : ffffffffc0080a20 ffffffffffff8100 0010000000000008 0010000000000000
$ 8 : 0000000050109ce3 0000000000000000 0000000000000010 8000000002c703c8
$12 : 0000000000800010 0000000000000010 0000000000000000 0000000000000000
$16 : 0010000000000000 800000007e56fb50 ffffffffdce80000 ffffffffc03dddfc
$20 : 0000000000000000 ffffffffc03dde1c 0000000000000037 8000000002c70b18
$24 : ffffffffdce80000 0000000000000000
$28 : 800000007e56c000 800000007e56faf0 0000000000000001 0010000000000008
Hi : 0000000000000005
Lo : 333333333333333b
epc : ffffffffc0080a38 do_ade+0x778/0xe10
Tainted: P O
ra : 0010000000000008 0x10000000000008
Status: 50109ce3 KX SX UX KERNEL EXL IE
Cause : 00800010
BadVA : 0010000000000000
PrId : 000d9301 (Cavium Octeon II)
Modules linked in: authenc xfrm4_mode_transport sha256_generic xfrm6_mode_tunnel deflate zlib_deflate xfrm_user xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 ip_vti xfrm4_mode_tunnel ip_tunnel 8021q garp stp llc xt_multiport xt_nat xt_policy xt_tcpudp xt_limit xt_LOG xt_comment xt_set xt_conntrack ip_set_bitmap_port ip6table_mangle ip6table_filter ip6table_raw ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 iptable_mangle xt_CT iptable_raw nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_h323 nf_conntrack_h323 nf_nat_sip nf_conntrack_sip nf_nat_proto_gre nf_nat_tftp nf_nat_ftp nf_nat nf_conntrack_tftp nf_conntrack_ftp nf_conntrack ip_set_hash_net ip_set nfnetlink iptable_filter ip_tables x_tables cvm_ipsec_kame(O) ipv6 imq cavium_ip_offload(PO) ubnt_nf_app(PO) tdts(PO) octeon_rng rng_core octeon_ethernet mdio_octeon of_mdio ethernet_mem octeon_common ubnt_platform(PO) libphy
Process kworker/0:0 (pid: 862, threadinfo=800000007e56c000, task=8000000089393600, tls=0000000000000000)
Stack : 0000000000000000 8000000002c70b18 ffffffffc0630000 ffffffffc0630000
80000000830b02d0 800000008c871338 ffffffffc05a0000 8000000002c74400
0000000000000000 0000000000000000 0000000000000000 ffffffffc0078f20
0000000000000000 0000000050109ce1 ffffffffc05a0000 ffffffffc0620000
0010000000000000 0000000000000000 ffffffffc059a428 0010000000000000
ffffffffc06203c8 8000000042650000 8000000002c703c8 8000000002c703c8
800000008c2ac2f8 000000000000cfd7 000000000000cfd8 0000000000000000
80000000830b02d0 800000008c871338 ffffffffc05a0000 8000000002c74400
0000000000000000 0000000000000000 0000000000000000 8000000002c70b18
0000000000000000 0000000000000010 0000000000800400 ffffffffc016dce4
...
Call Trace:
[<ffffffffc0080a38>] do_ade+0x778/0xe10
[<ffffffffc0078f20>] ret_from_exception+0x0/0xc
[<ffffffffc03dddfc>] flow_cache_gc_task+0x7c/0xc0
[<ffffffffc00c370c>] process_one_work+0x26c/0x4a8
[<ffffffffc00c3e7c>] worker_thread+0x144/0x480
[<ffffffffc00cbc0c>] kthread+0xa4/0xb0
[<ffffffffc0078f70>] ret_from_kernel_thread+0x14/0x1c
Code: 00d91024 1440ff40 00000000 <6a160000> 6e160007 24030000 1460ff78 00000000 0802023f
---[ end trace 392c274b7abff8da ]---
[sched_delayed] sched: RT throttling activated
Fatal exception: panic in 5 seconds[
[[[[[[ REMOVED LOG ENTRIES AS THEY CONTAIN PUBLIC IPS, ALL JUST FIREWALL DROP LOGS, WILL SEND TO UBNT DEVS IF NEEDED ]]]]]]]
Kernel panic - not syncing: Fatal exception
Rebooting in 60 seconds..
*** NMI Watchdog interrupt on Core 0x00 ***
$0 0x0000000000000000 at 0x0000000050109ce0
v0 0x0002b53beb120ff8 v1 0xffffffffc0630000
a0 0x00000000000f4240 a1 0x00000000000003e8
a2 0x0002b53beb215238 a3 0x0000000000000001
a4 0x0002b53beb213d3e a5 0x0000000000000001
a6 0x0000000000000003 a7 0xffffffffc0760000
t0 0x0000000000000018 t1 0xffffffffc0760000
t2 0x000000000000003c t3 0x000000000000ea60
s0 0x0000000000001324 s1 0x0000000000000010
s2 0x0000000000001388 s3 0x0000000000000001
s4 0xffffffffc0750000 s5 0x00000000000003e8
s6 0xffffffffc0750000 s7 0x8000000002c70b18
t8 0x0000000000000001 t9 0x0000000000000000
k0 0x000000007fb83a90 k1 0x800000008942ffe0
gp 0x800000007e56c000 sp 0x800000007e56f840
s8 0x0000000000000001 ra 0xffffffffc049b5f0
err_epc 0xffffffffc000788c epc 0x00000000774e8504
status 0x0000000050589ce4 cause 0x0000000040808c20
sum0 0x0004000100008000 en0 0x0900200500008000
*** Chip soft reset soon ***
Jumping to start of image at address 0xbfca0000
U-Boot 2012.04.01 (UBNT Build ID: 4670715-g7c4b1d0) (Build time: May 27 2014 - 11:19:05)
Skipping PCIe port 0 BIST, in EP mode, can't tell if clocked.
Skipping PCIe port 1 BIST, reset not done. (port not configured)
BIST check passed.
UBNT_E200 r1:0, r2:17, serial #: 802AA88FAB7A
MPR 13-00317-17
Core clock: 1000 MHz, IO clock: 600 MHz, DDR clock: 533 MHz (1066 Mhz DDR)
Base DRAM address used by u-boot: 0x8f800000, size: 0x800000
DRAM: 2 GiB
Clearing DRAM...... done
Flash: 8 MiB
Net: octeth0, octeth1, octeth2, octeth3, octeth4, octeth5, octeth6, octeth7
MMC: Octeon MMC/SD0: 0
USB: USB EHCI 1.00
scanning bus for devices... 1 USB Device(s) found
Type the command 'usb start' to scan for USB storage devices.
Hit any key to stop autoboot: 0
reading vmlinux.64
6435976 bytes read
argv[2]: numcores=2
argv[3]: endbootargs
Allocating memory for mapped kernel segment, alignment: 0x400000
Allocated memory for ELF segment: addr: 0x400000, size 0x77cd00
## Loading big-endian Linux kernel with entry point: 0x80000000008995d0 ...
Bootloader: Done loading app on coremask: 0x3
Starting cores 0x3
Linux version 3.10.20-UBNT (root@ubnt-builder2) (gcc version 4.7.0 (Cavium Inc. Version: SDK_3_1_0_p2 build 34) ) #1 SMP Sat Apr 22 06:58:28 PDT 2017
CVMSEG size: 2 cache lines (256 bytes)
Cavium Inc. SDK-3.1
bootconsole [early0] enabled
CPU revision is: 000d9301 (Cavium Octeon II)
Checking for the multiply/shift bug... no.
Checking for the daddiu bug... no.
Determined physical RAM map:
memory: 000000000e400000 @ 0000000000c00000 (usable)
memory: 0000000000c00000 @ 000000000f200000 (usable)
memory: 000000006f800000 @ 0000000020000000 (usable)
memory: 00000000005e2000 @ 0000000000400000 (usable)
memory: 000000000004e000 @ 00000000009e2000 (usable after init)
Wasting 57344 bytes for tracking 1024 unused pages
Using passed Device Tree <8000000000080000>.
software IO TLB [mem 0x02c0a000-0x02c4a000] (0MB) mapped at [8000000002c0a000-8000000002c49fff]
Zone ranges:
DMA32 [mem 0x00400000-0xefffffff]
Normal empty
Movable zone start for each node
Early memory node ranges
node 0: [mem 0x00400000-0x00a2ffff]
node 0: [mem 0x00c00000-0x0effffff]
node 0: [mem 0x0f200000-0x0fdfffff]
node 0: [mem 0x20000000-0x8f7fffff]
Primary instruction cache 37kB, virtually tagged, 37 way, 8 sets, linesize 128 bytes.
Primary data cache 32kB, 32-way, 8 sets, linesize 128 bytes.
Secondary unified cache 1024kB, 16-way, 512 sets, linesize 128 bytes.
PERCPU: Embedded 10 pages/cpu @8000000002c6f000 s11904 r8192 d20864 u40960
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 512622
Kernel command line: root=/dev/mmcblk0p2 rootdelay=10 rw rootsqimg=squashfs.img rootsqwdir=w mtdparts=phys_mapped_flash:640k(boot0),640k(boot1),64k(eeprom) console=ttyS0,115200
PID hash table entries: 4096 (order: 3, 32768 bytes)
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Memory: 2040272k/2078912k available (4760k kernel code, 38640k reserved, 1261k data, 312k init, 0k highmem)
Hierarchical RCU implementation.
Additional per-CPU info printed with stalls.
NR_IRQS:255
Calibrating delay loop (skipped) preset value.. 2000.00 BogoMIPS (lpj=10000000)
pid_max: default: 32768 minimum: 501
Security Framework initialized
Mount-cache hash table entries: 256
Checking for the daddi bug... no.
SMP: Booting CPU01 (CoreId 1)...
CPU revision is: 000d9301 (Cavium Octeon II)
Brought up 2 CPUs
NET: Registered protocol family 16
PTP Clock: Using sclk reference at 600000000 Hz
bio: create slab <bio-0> at 0
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
Switching to clocksource OCTEON_CVMCOUNT
NET: Registered protocol family 2
TCP established hash table entries: 16384 (order: 6, 262144 bytes)
TCP bind hash table entries: 16384 (order: 6, 262144 bytes)
TCP: Hash tables configured (established 16384 bind 16384)
TCP: reno registered
UDP hash table entries: 1024 (order: 3, 32768 bytes)
UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
NET: Registered protocol family 1
octeon_pci_console: Console not created.
/proc/octeon_perf: Octeon performance counter interface loaded
HugeTLB registered 2 MB page size, pre-allocated 0 pages
squashfs: version 4.0 (2009/01/31) Phillip Lougher
Registering unionfs 2.5.13 (for 3.10.34)
msgmni has been set to 3984
io scheduler noop registered
io scheduler cfq registered (default)
Serial: 8250/16550 driver, 6 ports, IRQ sharing disabled
1180000000800.serial: ttyS0 at MMIO 0x1180000000800 (irq = 34) is a OCTEON
console [ttyS0] enabled, bootconsole disabled
console [ttyS0] enabled, bootconsole disabled
1180000000c00.serial: ttyS1 at MMIO 0x1180000000c00 (irq = 35) is a OCTEON
loop: module loaded
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
octeon-ehci 16f0000000000.ehci: Octeon EHCI
octeon-ehci 16f0000000000.ehci: new USB bus registered, assigned bus number 1
octeon-ehci 16f0000000000.ehci: irq 56, io mem 0x16f0000000000
octeon-ehci 16f0000000000.ehci: USB 2.0 started, EHCI 1.00
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
octeon-ohci 16f0000000400.ohci: Octeon OHCI
octeon-ohci 16f0000000400.ohci: new USB bus registered, assigned bus number 2
octeon-ohci 16f0000000400.ohci: irq 56, io mem 0x16f0000000400
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 2 ports detected
i2c-octeon 1180000001000.i2c: version 2.5
i2c-octeon 1180000001200.i2c: version 2.5
octeon_wdt: Initial granularity 5 Sec
TCP: cubic registered
NET: Registered protocol family 17
NET: Registered protocol family 15
Bootbus flash: Setting flash for 8MB flash at 0x1f400000
phys_mapped_flash: Found 1 x16 devices at 0x0 in 8-bit bank. Manufacturer ID 0x0000c2 Chip ID 0x0000c9
Amd/Fujitsu Extended Query Table at 0x0040
Amd/Fujitsu Extended Query version 1.1.
phys_mapped_flash: Swapping erase regions for top-boot CFI table.
number of CFI chips: 1
3 cmdlinepart partitions found on MTD device phys_mapped_flash
Creating 3 MTD partitions on "phys_mapped_flash":
0x000000000000-0x0000000a0000 : "boot0"
0x0000000a0000-0x000000140000 : "boot1"
mmc0: BKOPS_EN bit is not set
0x000000140000-0x000000150000 : "eeprom"
mmc0: new high speed DDR MMC card at address 0001
Waiting 10sec before mounting root device...
mmcblk0: mmc0:0001 SEM04G 3.68 GiB
mmcblk0boot0: mmc0:0001 SEM04G partition 1 2.00 MiB
mmcblk0boot1: mmc0:0001 SEM04G partition 2 2.00 MiB
mmcblk0rpmb: mmc0:0001 SEM04G partition 3 2.00 MiB
mmcblk0: p1 p2
mmcblk0boot1: unknown partition table
mmcblk0boot0: unknown partition table
kjournald starting. Commit interval 3 seconds
EXT3-fs (mmcblk0p2): using internal journal
EXT3-fs (mmcblk0p2): recovery complete
EXT3-fs (mmcblk0p2): mounted filesystem with journal data mode
VFS: Mounted root (unionfs filesystem) on device 0:11.
Freeing unused kernel memory: 312K (ffffffffc05e2000 - ffffffffc0630000)
Algorithmics/MIPS FPU Emulator v1.5
INIT: version 2.88 booting
INIT: Entering runlevel: 2
Starting network plug daemon: netplugd.
[ ok ] Starting routing daemon: rib nsm ribd.
[ ok ] Starting EdgeOS router: migrate rl-system configure.
Welcome to EdgeOS Router ttyS0
By logging in, accessing, or using the Ubiquiti product, you
acknowledge that you have read and understood the Ubiquiti
License Agreement (available in the Web UI at, by default,
http://192.168.1.1) and agree to be bound by its terms.
Router login: