Exploring Windows 2003 Security: IP Security
January 9, 2004

More Secure Master Key (During the IKE Negotiation Stage)

As described previously, one of the tasks performed during the first phase of IKE negotiation is the generation of the IKE master key, which, in turn, is used to secure communication during the second phase (and frequently is re-used to generate a new session key, when its expiration interval passes). Starting with Windows Server 2003, you can increase security by specifying a longer key (2048 bit) to be used by the Diffie-Hellman algorithm, based on which a master key is created.

Diffie-Hellman parameters are configurable from the Key Exchange Security Methods dialog box. (They are specified using a separate setting for each of the security methods listed there.) This dialog box is reached by first clicking on the Settings ... button on the General tab of the IPSec policy Properties dialog box and then clicking on the Methods... button on the Key Exchange Settings dialog box.

More-Flexible Authentication Methods

As already explained, three authentication methods are available when establishing an IPSec session. Starting with Windows 2003, for certificate-based authentication, you can enable the certificate-to-account mapping option, as long as you operate in Active Directory environment. To turn this option on, launch the IP Security Policies MMC snap-in, double-click on the policy to be configured, choose a rule from the IPSec Policy properties dialog box to be modified (listed on the Rules tab), select the Authentication Methods tab of Edit Rule Properties dialog box, and, finally, edit the certificate authentication method (by checking on the checkbox labeled "Enable certificate to account mapping"). Once this setting is enabled, you can take advantage of user rights (such as Access this computer from the network or Deny access to this computer from the network) assigned via Group Policies to target computers to control which ones will be able to establish IPSec session.

Network Address Translation Awareness

One of the most significant problems with IPSec implementation in Windows 2000 (typically used for VPN L2TP/IPSec connections) was its incompatibility with Network Address Translation (NAT). This meant that two computers could not communicate using IPSec if their traffic was passing a NAT router. Although this could be circumvented by applying PPTP tunneling (with its inherent encryption methods) rather than IPSec/L2TP, or by setting a tunnel between an IPSec client and a firewall (before a NAT device was reached), these workarounds frequently compromised overall security.

Microsoft resolved this problem in Windows 2003. Its implementation of L2TP/IPSec is no longer based on proprietary IPSec encapsulation mechanism but rather on IETF RFP standards, incorporating NAT Transparency (NAT-T). Changes have also been made to RRAS NAT implementation. In addition, recently published updates to Windows XP and Windows 2000 L2TP/IPSec NAT-T VPN client software, as well as VPN client software for older versions of Windows (downloadable from the Microsoft Web site) allow connecting to Windows 2003 VPN server using L2TP/IPSec residing behind a NAT device from practically any version of Windows.

However, the NAT device will likely need to be configured to allow the passing of L2TP (UDP ports 500 and 1701), NAT-T (UDP port 4500), and ESP (IP protocol 50) traffic (specifics would depend on whether the NAT device resides on the client side, the server side, or both). Obviously, there must also be some way of authenticating the IPSec session (typically via certificate or preshared key).

For more information about the update, refer to Microsoft Knowledge Base article 818043.

Tutorial courtesy of ServerWatch.com.