Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
OpenSwan 2.4.7 and NATed Win XP: PAYLOAD_MALFORMED [SOLVED]
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
VinzC
Watchman
Watchman


Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood

PostPosted: Mon Jan 01, 2007 5:25 am    Post subject: OpenSwan 2.4.7 and NATed Win XP: PAYLOAD_MALFORMED [SOLVED] Reply with quote

Hi.

I've unmasked (keyword file) and installed OpenSwan 2.4.7 and l2tpd-0.70_pre20031121. The VPN server is behind a router with a firewall that allows forwarding only TCP and UPD ports; UDP ports 500, 4500 and 1701 have been forwarded to the OpenSwan server. My own XP machine is behind a Gentoo Linux NAT. Both machines have dynamic IP adresses that may change in less than 24 hours.

When I try to make a connection to the VPN server from my XP machine, I can see these messages on the server:
Code:
Jan  1 05:47:46 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #12: sending notification PAYLOAD_MALFORMED to 62.197.xxx.yyy:4500
Jan  1 05:47:49 serenity pluto[6728]: ERROR: asynchronous network error report on eth0 (sport=4500) for message to 62.197.xxx.yyy port 4500, complainant 192.168.1.5: No route to host [errno 113, origin ICMP type 3 code 1 (not authenticated)]
Jan  1 05:47:49 serenity pluto[6728]: ERROR: asynchronous network error report on eth0 (sport=4500) for message to 62.197.xxx.yyy port 4500, complainant 192.168.1.5: No route to host [errno 113, origin ICMP type 3 code 1 (not authenticated)]
Jan  1 05:47:54 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #12: next payload type of ISAKMP Hash Payload has an unknown value: 120
Jan  1 05:47:54 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #12: malformed payload in packet
Jan  1 05:47:54 serenity pluto[6728]: | payload malformed after IV
Jan  1 05:47:54 serenity pluto[6728]: |   30 5e ec b5  a5 3b 39 e2
Jan  1 05:47:54 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #12: sending notification PAYLOAD_MALFORMED to 62.197.xxx.yyy:4500
Jan  1 05:48:10 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #12: next payload type of ISAKMP Hash Payload has an unknown value: 120
Jan  1 05:48:10 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #12: malformed payload in packet
Jan  1 05:48:10 serenity pluto[6728]: | payload malformed after IV
Jan  1 05:48:10 serenity pluto[6728]: |   30 5e ec b5  a5 3b 39 e2
Jan  1 05:48:10 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #12: sending notification PAYLOAD_MALFORMED to 62.197.xxx.yyy:4500
Jan  1 05:48:12 serenity pluto[6728]: ERROR: asynchronous network error report on eth0 (sport=4500) for message to 62.197.xxx.yyy port 4500, complainant 192.168.1.5: No route to host [errno 113, origin ICMP type 3 code 1 (not authenticated)]
Jan  1 05:48:12 serenity pluto[6728]: ERROR: asynchronous network error report on eth0 (sport=4500) for message to 62.197.xxx.yyy port 4500, complainant 192.168.1.5: No route to host [errno 113, origin ICMP type 3 code 1 (not authenticated)]
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #9: received Delete SA(0xfdc36d06) payload: deleting IPSEC State #10
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #9: received and ignored informational message
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #7: received Delete SA(0x30d63173) payload: deleting IPSEC State #8
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #7: received and ignored informational message
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #5: received Delete SA(0x9c8943e3) payload: deleting IPSEC State #6
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #5: received and ignored informational message
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #3: received Delete SA(0x0a342f7a) payload: deleting IPSEC State #4
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #3: received and ignored informational message
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #1: received Delete SA(0x39fe8292) payload: deleting IPSEC State #2
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #1: received and ignored informational message
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #11: received Delete SA payload: deleting ISAKMP State #11
Jan  1 05:48:12 serenity pluto[6728]: packet from 62.197.xxx.yyy:4500: received and ignored informational message
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #9: received Delete SA payload: deleting ISAKMP State #9
Jan  1 05:48:12 serenity pluto[6728]: packet from 62.197.xxx.yyy:4500: received and ignored informational message
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #7: received Delete SA payload: deleting ISAKMP State #7
Jan  1 05:48:12 serenity pluto[6728]: packet from 62.197.xxx.yyy:4500: received and ignored informational message
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #5: received Delete SA payload: deleting ISAKMP State #5
Jan  1 05:48:12 serenity pluto[6728]: packet from 62.197.xxx.yyy:4500: received and ignored informational message
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #3: received Delete SA payload: deleting ISAKMP State #3
Jan  1 05:48:12 serenity pluto[6728]: packet from 62.197.xxx.yyy:4500: received and ignored informational message
Jan  1 05:48:12 serenity pluto[6728]: "roadwarrior-xp"[2] 62.197.xxx.yyy #1: received Delete SA payload: deleting ISAKMP State #1
Jan  1 05:48:12 serenity pluto[6728]: packet from 62.197.xxx.yyy:4500: received and ignored informational message
Jan  1 05:48:15 serenity pluto[6728]: ERROR: asynchronous network error report on eth0 (sport=4500) for message to 62.197.xxx.yyy port 4500, complainant 192.168.1.5: No route to host [errno 113, origin ICMP type 3 code 1 (not authenticated)]
Jan  1 05:48:15 serenity pluto[6728]: ERROR: asynchronous network error report on eth0 (sport=4500) for message to 62.197.xxx.yyy port 4500, complainant 192.168.1.5: No route to host [errno 113, origin ICMP type 3 code 1 (not authenticated)]
Jan  1 05:48:15 serenity pluto[6728]: ERROR: asynchronous network error report on eth0 (sport=4500) for message to 62.197.xxx.yyy port 4500, complainant 192.168.1.5: No route to host [errno 113, origin ICMP type 3 code 1 (not authenticated)]

I had OpenSwan 2.4.4 before and other error messages, which exhibited a problem with the server being behind a NAT. The masked version of OpenSwan fixed these problems (actually version 2.4.5 but it's not in portage). 62.197.xxx.yyy is my public IP address, i.e. the one of my Gentoo Linux gateway at home, which happens to do NAT. 192.168.1.5 is the private IP address of the remote OpenSwan server (it has only one NIC since it's inside a private LAN).

Here's the server's ipsec.conf:
/etc/ipsec/ipsec.conf:
version 2.0     # conforms to second version of ipsec.conf specification
config setup
        interfaces=%defaultroute
        klipsdebug=none
        plutodebug=none
        overridemtu=1410
        nat_traversal=yes
        virtual_private=%v4:10.0.0.0/8,%v4:172.16.0.0/12,%v4:192.168.0.0/16,%v4:!192.168.1.0/24
conn %default
        keyingtries=3
        compress=no
        disablearrivalcheck=no
        keyexchange=ike
        ikelifetime=240m
        keylife=60m
conn roadwarrior-xp
        leftprotoport=17/1701
        rightprotoport=17/%any
        rekey=no
        also=roadwarrior
conn roadwarrior
        authby=secret
        pfs=no
        type=tunnel
        left=%defaultroute
        right=%any
        rightsubnet=vhost:%no,%priv
        auto=add
include /etc/ipsec/ipsec.d/examples/no_oe.conf

The only wrong thing I understand is that there can't be anything at 62.197.xxx.yyy:500 nor 62.197.xxx.yyy:500 since this is my home Gentoo Linux server public address and my XP machine (the VPN client) is behind it. So if I understand correctly the VPN server tries to send responses to a port that doesn't exist. Can anybody help me find out what's wrong?
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!


Last edited by VinzC on Tue Jan 02, 2007 11:03 am; edited 2 times in total
Back to top
View user's profile Send private message
VinzC
Watchman
Watchman


Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood

PostPosted: Mon Jan 01, 2007 1:17 pm    Post subject: Reply with quote

As I'm using Windows XP SP2 and a NAT'ed VPN server, there is a registry patch to apply ( http://www.jacco2.dds.nl/networking/openswan-l2tp.html#serverNATed ). I also switched to net-dialup/xl2tpd-1.1.06 and disabled Data compression in the XP VPN client connection properties. I'm still receiving error messages but it looks like the connection process goes a little further:
Code:
Jan  1 14:11:58 serenity pluto[10568]: packet from 62.197.xxx.yyy:500: ignoring Vendor ID payload [MS NT5 ISAKMPOAKLEY 00000004]
Jan  1 14:11:58 serenity pluto[10568]: packet from 62.197.xxx.yyy:500: ignoring Vendor ID payload [FRAGMENTATION]
Jan  1 14:11:58 serenity pluto[10568]: packet from 62.197.xxx.yyy:500: received Vendor ID payload [draft-ietf-ipsec-nat-t-ike-02_n] method set to=106
Jan  1 14:11:58 serenity pluto[10568]: packet from 62.197.xxx.yyy:500: ignoring Vendor ID payload [Vid-Initial-Contact]
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #48: responding to Main Mode from unknown peer 62.197.xxx.yyy
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #48: transition from state STATE_MAIN_R0 to state STATE_MAIN_R1
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #48: STATE_MAIN_R1: sent MR1, expecting MI2
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #48: NAT-Traversal: Result using draft-ietf-ipsec-nat-t-ike-02/03: both are NATed
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #48: transition from state STATE_MAIN_R1 to state STATE_MAIN_R2
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #48: STATE_MAIN_R2: sent MR2, expecting MI3
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #48: Main mode peer ID is ID_FQDN: '@game01'
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #48: I did not send a certificate because I do not have one.
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #48: transition from state STATE_MAIN_R2 to state STATE_MAIN_R3
Jan  1 14:11:58 serenity pluto[10568]: | NAT-T: new mapping 62.197.xxx.yyy:500/4500)
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #48: STATE_MAIN_R3: sent MR3, ISAKMP SA established {auth=OAKLEY_PRESHARED_KEY ci
pher=oakley_3des_cbc_192 prf=oakley_sha group=modp2048}
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #49: responding to Quick Mode {msgid:b322070a}
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #49: transition from state STATE_QUICK_R0 to state STATE_QUICK_R1
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #49: STATE_QUICK_R1: sent QR1, inbound IPsec SA installed, expecting QI2
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #49: transition from state STATE_QUICK_R1 to state STATE_QUICK_R2
Jan  1 14:11:58 serenity pluto[10568]: "roadwarrior-xp"[2] 62.197.xxx.yyy #49: STATE_QUICK_R2: IPsec SA established {ESP=>0x748f1449 <0x94eb6794 xfrm=3DES_
0-HMAC_MD5 NATD=62.197.xxx.yyy:4500 DPD=none}
Jan  1 14:12:00 serenity xl2tpd[10431]: control_finish: Peer requested tunnel 2 twice, ignoring second one.
Jan  1 14:12:01 serenity xl2tpd[10431]: control_finish: Peer requested tunnel 2 twice, ignoring second one.
Jan  1 14:12:03 serenity pluto[10568]: ERROR: asynchronous network error report on eth0 (sport=4500) for message to 62.197.xxx.yyy port 4500, complainant 1
92.168.1.5: No route to host [errno 113, origin ICMP type 3 code 1 (not authenticated)]
Jan  1 14:12:03 serenity pluto[10568]: ERROR: asynchronous network error report on eth0 (sport=4500) for message to 62.197.xxx.yyy port 4500, complainant 1
92.168.1.5: No route to host [errno 113, origin ICMP type 3 code 1 (not authenticated)]
Jan  1 14:12:03 serenity pluto[10568]: ERROR: asynchronous network error report on eth0 (sport=4500) for message to 62.197.xxx.yyy port 4500, complainant 1
92.168.1.5: No route to host [errno 113, origin ICMP type 3 code 1 (not authenticated)]
Jan  1 14:12:05 serenity xl2tpd[10431]: control_finish: Peer requested tunnel 2 twice, ignoring second one.
Jan  1 14:12:05 serenity xl2tpd[10431]: Maximum retries exceeded for tunnel 52425.  Closing.
Jan  1 14:12:05 serenity xl2tpd[10431]: Connection 2 closed to 62.197.xxx.yyy, port 1701 (Timeout)

_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!
Back to top
View user's profile Send private message
VinzC
Watchman
Watchman


Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood

PostPosted: Mon Jan 01, 2007 3:13 pm    Post subject: Reply with quote

Problem almost solved: I added the following line to the road-warrior section /etc/ipsec/ipsec.conf:
Code:
leftnexthop=<LAN ip address of the internet router>

Now the remote server is inaccessible once I close the VPN connection 8O . I must get to the remote site to check what's wrong. Anyway I can make a succesfull connection now.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!
Back to top
View user's profile Send private message
VinzC
Watchman
Watchman


Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood

PostPosted: Tue Jan 02, 2007 11:04 am    Post subject: Reply with quote

Problem solved. The server happened to be inaccessible due to something else, pure coïncidence.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum