linux to windows ipsec

[Juridian]

Do any of you have experience setting up linux to windows ipsec connections?

More specifically configuring a windows 2k3 box to talk to a red hat racoon ike daemon?

[hidoa]

Are you talking client-server communications or client-client communications??

[Juridian]

I don't believe client vs server matters one bit when you are enabling ipsec communications. It matters more whether it's host to host, host to gateway, or gateway to gateway. I'm simply doing host to host communications.

[Maestr0]

Depends what you are trying to do. Windows comes with L2TP/IPSEC, so if you want to use L2TP its fairly straight forward from the client after you create your X.509 cert and your server supports L2TP, but if you want to ditch L2TP then you need to use an IPSEC client on the Windows machine that doesnt use L2TP (http://vpn.ebootis.de/) there are also L2TP clients (and server) for linux if you are connecting a linux client to a Windows server (http://sourceforge.net/projects/l2tpd)


-Maestr0

[Juridian]

Here is where our communications are getting screwed. You are talking about host (client) to gateway (server) vpn. You do not however have to use a client server model using ipsec.

I'm simply working on a server isolation scheme of sorts. This is more of a host to host setup with ipsec handling secure communications and doing packet filtering.

I've been having issues getting the two ike daemons to do a proper handshake and get past phase 1, I really just need to take more time and finish reading the manual to configure and troubleshoot things properly.

I'll write it up when I'm done and post it.

l2tp? I don't need no stinking l2tp...

[hidoa]

client-client, i meant host to host, sorry for the confusion. I knew you weren't doing host to gateway because of the two OSs..but wasn't sure if it was client/server vpn via ipsec

I would like to see what you find out...

[sec_ware]

Hi

I am a bit late, but I need to test what I write. And time is precious
In general, I can recommend two readers: on securityfocus about the
windows part[1], Spenneberg about the linux part[2].

Let us not make this more complicated than it needs to be. Two hosts,
isolating setup. We use transport mode with preshared keys as a beginning.


Linux-part (192.168.1.13)


The policy (SPD) is manipulated with setkey. Let's create
in /etc/racoon/setkey.conf

#!/usr/sbin/setkey -f

flush;
spdflush;

spdadd 192.168.1.13 192.168.1.16 any -P out ipsec
        esp/transport//require
        ah/transport//require; 

spdadd 192.168.1.16 192.168.1.13 any -P in ipsec
        esp/transport//require
        ah/transport//require;

I want to enable a communication .1.13 <-> .1.16, transport,
both encryption and authentication (no replay). Start with
setkey -f /etc/racoon/setkey.conf and check with setkey -DP

Next step: racoon.

We need to specify the kind of encryption. I use pre_shared_key,
3des and md5 - for simplicity.
racoon.conf in /etc/racoon/racoon.conf

path pre_shared_key "/etc/racoon/psk.txt";
#path certificate "/etc/racoon/certs";

remote 192.168.1.16 {
        exchange_mode main;
        proposal {
                encryption_algorithm 3des;
                hash_algorithm md5;
                authentication_method pre_shared_key;
                dh_group modp1024;
        }
}

However, for phase 2, we need the SA information. Let us not be
specific and use, also for simplicity DH group 2

sainfo anonymous
{
pfs_group 2;
encryption_algorithm 3des;
authentication_algorithm hmac_md5;
compression_algorithm deflate;
}

Create a file /etc/racoon/psk.txt

192.168.1.16    PresharedKey

Start racoon with racoon -F -f /etc/racoon/racoon.conf,
use -d for debugging.


Windows-part (192.168.1.16)


MMC->SnapIn->[3]

Create a new policy (no tunnel!), make sure to use the correct
encryptions MD5 and 3DES, and take the correct group: DH2 (Tab: General).
Make a new rule (for simplicity: all traffic, all protocols, ...) between
.1.13 and .1.16. (mirrored).

There is one important point: The Filter Action should have
Session key perfect forward secrecy enabled, otherwise
there is a problem getting phase 2! I would not accept or allow anything
related to unsecured communication.

Authentication: PreSharedKey. Type the same "passphrase" as in
/etc/racoon/psk.txt

Assign. Done.



Now, this is the first step to enable a IPSec transport, which
is fine for your purpose, I think. Check the traffic with a sniffer
and eventually, turn of the PreSharedKey-mode and use a certificate.


Cheers



[1] http://online.securityfocus.com/infocus/1519
http://online.securityfocus.com/infocus/1526
http://online.securityfocus.com/infocus/1528
[2] http://www.ipsec-howto.org/ipsec-howto.pdf
[3] http://www.antionline.com/showthread...hreadid=262801