| View previous topic :: View next topic |
| Author |
Message |
tazinblack
Veteran
Joined: 23 Jan 2005
Posts: 1146
Location: Baden / Germany
|
 Posted: Wed Aug 14, 2013 10:20 am Post subject: packet drops under vmware with newer kernels |
 |
|
Hi together,
I have an issue with some virtual gentoo machines running on vmware esxi 5.1 using newer kernels.
I run i.e icinga on these vms which sends and receives a lot of small network packets as I think.
Problem is that aI have a lot of dropped incoming packets:
| Code: | enp2s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet <some ip> netmask 255.255.255.0 broadcast <some bc adress>
inet6 <some mac address> prefixlen 64 scopeid 0x20<link>
ether <some mac address> txqueuelen 1000 (Ethernet)
RX packets 16357 bytes 1872630 (1.7 MiB)
RX errors 0 dropped 545 overruns 0 frame 0
TX packets 17015 bytes 7781044 (7.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
|
As you can see I have 545 dropped RX packets and this VM is only running for about 5 minutes.
This number increases every minute. After some days I have millions of them.
I also have very poor check latency in icinga. It does not matter if I use e1000 driver based network
interface or vmxnet3. It's the same behavior on both of them.
Looks like this is with kernels greater 3.2 On machines using old 2.6 kernels I do not have this issue.
I also have a ubuntu vm using 3.2.0.29 kernel which also drops packages but there I have only about 5600 over 239 days uptime and that I think may be normal.
Another suse sles11 with kernel 2.6.27.19-5-default does not drop any packet.
I tried to extent RX buffer with ethtool -G enp2s0 rx 4096 so that I have these values configured:
| Code: | ethtool -g enp2s0
Ring parameters for enp2s0:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 256 |
But this does not change anything. I also can not see dropped packets on the virtual or any physical switch.
Would be great if anyone out there could give me a hint how to fix this. 
_________________
Gruß / Regards
tazinblack
_______________________________________________________
what's the point in being grown up if you can't be childish sometimes |
|
| Back to top |
|
 |
Neon Dude
n00b
Joined: 13 Jun 2004
Posts: 15
Location: London
|
| Posted: Thu Aug 15, 2013 8:10 pm Post subject: |
|
|
Hi,
So dropped packets can take quite some debugging, but you may want to try experimenting with the net.core.netdev_max_backlog sysctl setting - try bumping it up and see if this has any impact. It may also require net.core.rmem_max/net.core.wmem_max to be increased too.
Good luck! |
|
| Back to top |
|
|
tazinblack
Veteran
Joined: 23 Jan 2005
Posts: 1146
Location: Baden / Germany
|
| Posted: Fri Aug 16, 2013 6:18 am Post subject: |
|
|
Good idea, thank you!
The defaults were
| Code: | sysctl net.core
net.core.dev_weight = 64
net.core.message_burst = 10
net.core.message_cost = 5
net.core.netdev_budget = 300
net.core.netdev_max_backlog = 1000
net.core.netdev_tstamp_prequeue = 1
net.core.optmem_max = 20480
net.core.rmem_default = 212992
net.core.rmem_max = 212992
net.core.rps_sock_flow_entries = 0
net.core.somaxconn = 128
net.core.warnings = 1
net.core.wmem_default = 212992
net.core.wmem_max = 212992
net.core.xfrm_acq_expires = 30
net.core.xfrm_aevent_etime = 10
net.core.xfrm_aevent_rseqth = 2
net.core.xfrm_larval_drop = 1
|
And I changed them to
| Code: | sysctl net.core
net.core.dev_weight = 64
net.core.message_burst = 10
net.core.message_cost = 5
net.core.netdev_budget = 300
net.core.netdev_max_backlog = 10000
net.core.netdev_tstamp_prequeue = 1
net.core.optmem_max = 20480
net.core.rmem_default = 212992
net.core.rmem_max = 425984
net.core.rps_sock_flow_entries = 0
net.core.somaxconn = 128
net.core.warnings = 1
net.core.wmem_default = 212992
net.core.wmem_max = 425984
net.core.xfrm_acq_expires = 30
net.core.xfrm_aevent_etime = 10
net.core.xfrm_aevent_rseqth = 2
net.core.xfrm_larval_drop = 1
|
I also tried to adjust settings recommended here http://gnet.soe.ucsc.edu/doku.php?id=tcptuning and here http://www.speedguide.net/articles/linux-tweaking-121.
Unfortunately this does not have any effect.
I have round about 1 dropped packed per second and nothing I try seems to help 
_________________
Gruß / Regards
tazinblack
_______________________________________________________
what's the point in being grown up if you can't be childish sometimes |
|
| Back to top |
|
|
tazinblack
Veteran
Joined: 23 Jan 2005
Posts: 1146
Location: Baden / Germany
|
| Posted: Thu Sep 19, 2013 10:59 am Post subject: |
|
|
isn't there anything new that can help?
_________________
Gruß / Regards
tazinblack
_______________________________________________________
what's the point in being grown up if you can't be childish sometimes |
|
| Back to top |
|
|
|
Networking & Security
