IPSec between Linux and the NetScreen

VPNs are a fine thing, especially if you are not at home and you need to access your beloved network. I am using VPN connections since I have implemented them at my Pfsense using openvpn.

Since I got my NetScreen/Juniper SSG5 here, I cannot use Openvpn anymore – So I was forced to dig into IPSec stuff. My laptop will be the roadwarrior connecting to my home. That’s why I have to do it the DialUp way, which doesn’t really include a modem. DialUp connections are connections, initiated by the client.

Our environment:

  • Netscreen:
  • Roadwarrioir:
  • Private Network:

User & Keys:

  • User: myuser
  • IKE ID: user@host.com
  • Group: dialupusers
  • Preshared Key: secret

It’s best to start with the Netscreen box. As I don’t like waiting, I just connect to it using a ssh client. As a side effect I don’t have to wait for the websites to load, but that’s just a matter of taste. For keeping stuff readable here, I am splitting the setup into parts.

Users & Groups

set user "myuser" ike-id u-fqdn "user@host.com" share-limit 1
set user "myuser" type ike
set user "myuser" "enable"
set user-group "dialupusers" user "myuser"

Auto-IKE VPN and Policy

To be able to work behind firewalls, NAT-Traversal should be enabled. If our roadwarrior isn’t sitting behind a firewall using NAT, it just won’t be used. At least it doesn’t harm.

set ike gateway "dialupvpn" dialup "dialupusers" Aggr outgoing-interface "ethernet0/0" preshare "secret" proposal "pre-g2-3des-sha"
set ike gateway "dialupvpn" nat-traversal
set ike gateway "dialupvpn" udp-checksum
set vpn "dialupvpn" gateway "dialupvpn" no-replay tunnel idletime 0 proposal "g2-esp-3des-sha"
set address "Trust" ""
set policy top from "Untrust" to "Trust" "Dial-Up VPN" "" "ANY" Tunnel vpn "dialupvpn" log

Believe it or not, but that’s all folks – nothing more to do on the NetScreen except for ‘save’ and ‘quit’ for signing off. Our next stop is the linux box, having the ipsec tools already installed. The Gentoo Package would be net-firewall/ipsec-tools.

Regarding the kernel, please make sure, the following options are available – either as a module or compiled into:

  • PF_KEY sockets (CONFIG_NET_KEY)
  • AH transformation (CONFIG_INET_AH)
  • ESP transformation (CONFIG_INET_ESP)
  • IPCOMP transformation (CONFIG_INET_IPCOMP)
  • DES, SHA1, (etc) from Crypto API

Our pre-shared key (PSK) used in the NetScreen config has to be filled into /etc/racoon/psk.txt to be assigned to a vpn connection.

# ipv4/ipv6 adresses        abc123

Regarding target hosts we’d like to connect to, they have to be defined in the racoon.conf file…

# Pre-shared key
path pre_shared_key "/etc/racoon/psk.txt";

# Remote host
exchange_mode aggressive;
my_identifier user_fqdn "user@host.com";
proposal {
encryption_algorithm 3des;
hash_algorithm sha1;
authentication_method pre_shared_key;
dh_group modp1024;

# A sample sainfo section
# Create one for each subnet you want to access, etc.
sainfo address any address any
pfs_group modp1024;
encryption_algorithm 3des;
authentication_algorithm hmac_sha1;
compression_algorithm deflate;

… which are used in the ipsec.conf to tell the kernel where to send the packets to.

#!/usr/sbin/setkey -f


# outbound
spdadd any
    -P out ipsec esp/tunnel/;

# inbound
spdadd any
    -P in ipsec esp/tunnel/;

Now save the config files and start the connection. But don’t be shocked if it doesn’t work at the first packet – that’s a common ‘problem’ as authentication has to be done first. But if 15 seconds or more have passed without success, it’s time to have a look at the logfiles.


6 thoughts on “IPSec between Linux and the NetScreen”

  • for the lines with spdadd, can I use fqdn like example.domain.com instead of ip addresses, because i have a dynamic IP setup and i use dyndns to update the hostnameIP mapping. everytime the router resets or line drops, i have to redo the configuration. I have searched a lot on the web, but there’s no documentation telling whether it is possible or not.

    Thank you in advance for your time and efforts.

    Diabolic Preacher
    As Is

  • I don’t think that hostnames will work. I had the same problem with different networks I am in – but I solved it with a little script that writes the spdadd lines for me.

    In your case it could be a script that writes /etc/ipsec.conf using a template.

  • helo,

    I did everything you teach on this page, but I am receiving a message on racoon.log

    eth1[500] (Address already in use).

    do you known what is it?

    thanks anyway

  • thanks for the quick reposnse, but I dont have another ipsec running, I have only on another site with a nat, but I have openvpn on the same server is that a problem?

Leave a Reply

Your email address will not be published. Required fields are marked *