ksoftirqd consumes >80% CPU on embedded platform

we're designing SOHO router开发者_开发问答 based on MIPS processor, wired up with 24-ports switch. The CPU runs NAT (configured with iptables), iptables rules, dhcp etc. it doesn't have any H/W acceleration for these functions. When testing NAT in full-mesh mode (i.e. one WAN port and others are LAN port), we observe the significant system's slowdown, especially console responds very slowly, and there is also packets loss.

The 'top' shows that ksoftirqd consumes over 80% of CPU.

What can be the reason of such behaviour? Does the Linux NAT run in userland?

ksoftirqds are kernel threads driving ... soft IRQs, things like TIMER_SOFTIRQ, SCSI_SOFTIRQ, TASKLET_SOFTIRQ, and what's relevant to your case, NET_TX_SOFTIRQ and NET_RX_SOFTIRQ. These are implemented in bottom halves of the kernel, as deffered work from top halves - the actual interrupt handlers in the device drivers where latency is critical.

Actual interrupt handler, or hardware IRQ, for a network card is concerned with getting data to/from the device as quickly as possible. It doesn't know anything about NAT and other TCP/IP processing. It knows about its bus handling (say PCI), its card specifics (ring buffers, control/config registers), DMA, and a bit about Ethernet. It hands/receives packets (skbufs to be exact) through queues to/from bottom half.

Take a look at the ethtool(8) if you haven't yet. See if you can tune the hardware/drivers to do checksum/segmentation offloading, etc. I don't have any suggestions on the NAT front, I don't use it.

Hope this helps a bit.

Edit:

As mentioned in the comments, check the NIC hardware for interrupt mitigation and the supporting driver for NAPI support.

ksoftirqd is the IRQ handler. You may check /proc/interrupts to see which IRQ is under load.

The CPU is overloaded: use a stronger model, or use simplier iptables rules. Linux NAT run in kernel space, ksoftirqd is in kernel space.