| View previous topic :: View next topic |
| Author |
Message |
doublehp
Guru
Joined: 11 Apr 2005
Posts: 473
Location: FRANCE
|
 Posted: Sun Feb 16, 2014 11:33 pm Post subject: ip_conntrack always at max |
 |
|
Hello. Small issue on one server.
| Code: | leon:~# cat /proc/sys/net/ipv4/netfilter/ip_conntrack_max
12216
leon:~#
leon:~# wc -l /proc/net/ip_conntrack
11548 /proc/net/ip_conntrack
leon:~# cat /proc/net/ip_conntrack | grep 192.168.248 | wc -l
11710
leon:~# wc -l /proc/net/ip_conntrack
12098 /proc/net/ip_conntrack
leon:~#
leon:~# head /proc/net/ip_conntrack | grep 192.168.248
tcp 6 430749 ESTABLISHED src=192.168.246.208 dst=192.168.248.35 sport=36567 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.35 dst=192.168.246.208 sport=80 dport=36567 packets=0 bytes=0 mark=0 secmark=0 use=1
icmp 1 6 src=192.168.246.208 dst=192.168.248.193 type=8 code=0 id=26384 packets=1 bytes=84 [UNREPLIED] src=192.168.248.193 dst=192.168.246.208 type=0 code=0 id=26384 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 424149 ESTABLISHED src=192.168.246.208 dst=192.168.248.167 sport=59210 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.167 dst=192.168.246.208 sport=80 dport=59210 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 424149 ESTABLISHED src=192.168.246.208 dst=192.168.248.46 sport=59210 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.46 dst=192.168.246.208 sport=80 dport=59210 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 414549 ESTABLISHED src=192.168.246.208 dst=192.168.248.57 sport=44526 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.57 dst=192.168.246.208 sport=80 dport=44526 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 413949 ESTABLISHED src=192.168.246.208 dst=192.168.248.152 sport=42318 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.152 dst=192.168.246.208 sport=80 dport=42318 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 412148 ESTABLISHED src=192.168.246.208 dst=192.168.248.67 sport=54206 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.67 dst=192.168.246.208 sport=80 dport=54206 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 407349 ESTABLISHED src=192.168.246.208 dst=192.168.248.164 sport=53543 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.164 dst=192.168.246.208 sport=80 dport=53543 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 407349 ESTABLISHED src=192.168.246.208 dst=192.168.248.4 sport=53543 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.4 dst=192.168.246.208 sport=80 dport=53543 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 404350 ESTABLISHED src=192.168.246.208 dst=192.168.248.237 sport=49887 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.237 dst=192.168.246.208 sport=80 dport=49887 packets=0 bytes=0 mark=0 secmark=0 use=1
leon:~#
leon:~# tail /proc/net/ip_conntrack | grep 192.168.248
tcp 6 419334 ESTABLISHED src=192.168.246.208 dst=192.168.248.125 sport=62524 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.125 dst=192.168.246.208 sport=80 dport=62524 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 415735 ESTABLISHED src=192.168.246.208 dst=192.168.248.172 sport=34188 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.172 dst=192.168.246.208 sport=80 dport=34188 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 415134 ESTABLISHED src=192.168.246.208 dst=192.168.248.198 sport=50938 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.198 dst=192.168.246.208 sport=80 dport=50938 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 414535 ESTABLISHED src=192.168.246.208 dst=192.168.248.81 sport=44526 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.81 dst=192.168.246.208 sport=80 dport=44526 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 413934 ESTABLISHED src=192.168.246.208 dst=192.168.248.8 sport=42318 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.8 dst=192.168.246.208 sport=80 dport=42318 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 413335 ESTABLISHED src=192.168.246.208 dst=192.168.248.241 sport=38613 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.241 dst=192.168.246.208 sport=80 dport=38613 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 404934 ESTABLISHED src=192.168.246.208 dst=192.168.248.116 sport=49532 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.116 dst=192.168.246.208 sport=80 dport=49532 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 403134 ESTABLISHED src=192.168.246.208 dst=192.168.248.111 sport=58170 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.111 dst=192.168.246.208 sport=80 dport=58170 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 402534 ESTABLISHED src=192.168.246.208 dst=192.168.248.25 sport=54812 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.25 dst=192.168.246.208 sport=80 dport=54812 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp 6 402534 ESTABLISHED src=192.168.246.208 dst=192.168.248.203 sport=54812 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.248.203 dst=192.168.246.208 sport=80 dport=54812 packets=0 bytes=0 mark=0 secmark=0 use=1
leon:~#
leon:~# cat /etc/tayga/tayga.conf
# http://www.litech.org/tayga/
# http://www.litech.org/tayga/faq.html
# http://ipvsix.me/?tag=tayga
# http://priv.nu/projects/ndppd/
tun-device nat64
ipv4-addr 192.168.248.1 #(this is TAYGA's IPv4 address, not your router's address)
prefix 2a01:a:x:y:z::/96 #(replace with an unused /96 prefix from your site's address range)
dynamic-pool 192.168.248.0/24
data-dir /var/db/tayga
leon:~#
leon:~# ifconfig nat64
nat64 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:192.168.246.208 P-t-P:192.168.246.208 Mask:255.255.255.255
inet6 addr: 2a01:a:x:y::208/128 Scope:Global
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:66015 errors:0 dropped:0 overruns:0 frame:0
TX packets:66350 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:5619776 (5.3 MiB) TX bytes:3809248 (3.6 MiB)
leon:~#
|
Now, a small bit of a Munin plugin:
| Code: | if [ -f /proc/sys/net/ipv4/ip_conntrack_max ] ; then
read MAX </proc/sys/net/ipv4/ip_conntrack_max
elif [ -f /proc/sys/net/ipv4/netfilter/ip_conntrack_max ]; then
read MAX < /proc/sys/net/ipv4/netfilter/ip_conntrack_max
fi
if [ -n "$MAX" ]; then
echo total.warning `expr $MAX \* 8 / 10`
echo total.critical `expr $MAX \* 9 / 10`
fi |
The issue is that, since several years, the graph fw_conntrack.html is always in warning state. I could not see it, because the total field was not graphed (had graph=no in the plugin; I changed it to yes, so that now the graph has a color in the over-all view). I just receive an email about the critical state from time to time, but, when I visited the page, it was not red, and there was no Total field ... so I could not see any value, neither any critical setting.
For thos who do not understand munin: /proc/net/ip_conntrack always report a very high connection count, very close to the max set in /proc/sys/net/ipv4/netfilter/ip_conntrack_max .
1: what happens when the number of connections reaches the max allowed by kernel ? are connection refused ? are oldest connection closed ?
I first thought that connexions were opened by a home made script, that often performs requests, and may not always close them cleanly.
2: could this be a taiga bug ?
The SRC ip seems to always be the taiga IP. The host has many other interfaces, and IPs; all in 192.168.246.0/24, but different from 192.168.246.208.
The destinations seems to all be in 192.168.248.0/24 ... but there is not any physical host using any of those IPs any where in my LAN.
I have several scanners installed on the host that probes all possible IPs in my network. The scanner lists the interfaces, and the routes defined, and then, for each local network, will try to ping all IPs. I also have several nmap around.
3: could those lines be due to my scanners ? My scanner could see the nat64 interface, see it's IP, compute a netmask, and then scan 192.168.248.0/24 , but I am not sure it could explain this issue.
4: is it possible to track back which process initiated the request ? when the request was done (how old it is). What is the classic timeout for unreplied requests ?
I find strange that the requests are tcp with dport=80; this does not sound like my ping scanner (but it could be one of the nmap ones).
Because of the 192.168.248.* part, it has to be taiga related. But maybe it's just taiga doing mess on it's own.
After each reboot, Munin shows that the connection count comes down to 0. Then grows with time, linearly. Then the count stays flat around 11k (between 80% and 90% of the kernel max) for as long as the machine stays up. The growth from 0 to 12k after reboot takes between 7h00 and 7h15.
http://djlab.com/2009/12/sysctl-and-ip_conntrack_max-optimization/
It's a small computer, with very limited ressources. I am not sure increasing the max would be a good idea. It's a home server; it does not have so many clients connecting. The services installed are not even accessible via any public google research; it does not respond to any public domain name.
[IMG]http://imagizer.imageshack.us/v2/xq90/850/gyel.png[/IMG]
Thank you.
_________________
DEMAINE Benoît-Pierre (aka DoubleHP ) http://www.demaine.info/
>o_/ Coin coin coin \_o<
to contact me (MSN,ICQ, JABBER, Skype ... ) http://benoit.demaine.info/contact.png |
|
| Back to top |
|
 |
doublehp
Guru
Joined: 11 Apr 2005
Posts: 473
Location: FRANCE
|
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Networking & Security
