Introduction

VPNs, like many tools in technology, offer both advantages and drawbacks. A significant downside is the latency and reduced network performance they can sometimes cause. Over the years, various VPN implementations have been developed to tackle these challenges and improve overall efficiency.

This article explores several state-of-the-art VPN implementations to assess the extent of performance degradation they experience. It is based on a 2019 study conducted by Thomas Fischer at the University of Skövde. The research question is as follows:

How does the performance differ between state-of-the-art VPN solutions under stable versus unreliable network conditions?

What are Virtual Private Networks (VPNs)

A Virtual Private Network (VPN) is a technology used to extend a private network over the internet, allowing authorized users (typically authenticated with a shared secret such as a password) to access it as if they were directly connected. This makes these 'authorized users' appear to be connected to the private network, even though they may be thousands of miles and numerous router hops away.

VPN Tunnel

VPN connection is achieved using secure, encrypted 'tunnels' called VPN tunnels. They are referred to as tunnels because they are established between two endpoints. When two routers are configured for tunneling, it becomes possible to send encrypted packets that might not be supported over the general internet between them.
When set up correctly, these tunnels provide:

• Privacy: By not exposing the private network to the public internet.

• Security: By encrypting the packets shared through the tunnels. Anyone who gains access to the encrypted packets will not be able to read them. Only authorized users can decrypt and access the data.

• Data integrity: Through mechanisms that detect if the data has been tampered with during transport.

Any network running such tunnel is running a Virtual Private Network or VPN.

Types of VPN Tunneling

• Host-to-network (Remote access) tunneling: Allows individual users to securely connect to a private network. This is commonly used by remote workers who need access to private company files.

• Site-to-site tunneling: Connects entire networks, often used by companies with offices in different locations.

• Split tunneling: Routes only specific traffic through the VPN, while other traffic is sent directly to the internet. This is useful for balancing security and speed.

• Full tunneling: Routes all traffic through the VPN, providing maximum privacy and security.

Side Note: VPNs provide security between two endpoints, but they do not secure the endpoints themselves. This poses a significant security risk because every endpoint with access to your VPN also has access to your private network. Additional security measures are necessary to ensure endpoint security and protect the network from potential vulnerabilities.

Different VPN Solutions

There are numerous VPN solutions available, but three stand out as the most popular and were used for this study:

• IPSec (IP Security Protocol)

• OpenVPN

• WireGuard

IPsec (IP Security Protocol)

IPsec was standardized by the Internet Engineering Task Force (IETF) in 1995. Originally developed for IPv6, it has also been widely implemented for IPv4. IPsec is an IETF-approved end-to-end authentication and encryption system. Almost all major VPN vendors include at least an IPsec compatibility mode in their products. Additionally, operating systems like Linux, Solaris, HP-UX, and AIX provide native kernel support for IPsec.

In tunnel mode, IPsec encrypts the transport layer header, which includes source and destination port numbers. However, this approach often conflicts with how most firewalls operate. To address this issue, most modern implementations default to transport mode, where only the payloads of packets (the data being transported) are encrypted.

OpenVPN

OpenVPN has become the de facto standard for VPNs today, with over 50 million downloads since its release in 2001 (OpenVPN, 2019). It uses SSL/TLS for key exchange and encryption. OpenVPN is open-source, which contributes to its security through the extensive scrutiny it undergoes, as anyone can review the code. In 2017, an independent review of OpenVPN was conducted by Cryptography Engineering (Hopkins and Green, 2019), and the results found no major vulnerabilities.

OpenVPN supports both TCP and UDP, with UDP as the default. While UDP is faster, it does not perform error correction like TCP does. OpenVPN is fully functional on Windows, macOS, and Linux, and offers a wide range of ciphers and encryption methods to choose from.

WireGuard

WireGuard is a new VPN solution designed to replace two of the most widely used VPN technologies: OpenVPN and IPSec (Donenfeld, 2018). It claims to offer better performance than OpenVPN while avoiding the complexity of IPSec. Originally written for Linux systems, WireGuard is now available on multiple platforms. Like OpenVPN and IPSec, it is open-source, allowing anyone with the knowledge to audit the code. One of its design goals is to offer a straightforward configuration, similar to SSH, using asymmetric key cryptography. Currently, WireGuard supports only UDP (Donenfeld, 2018).

Some other VPN solutions include: PPTP (Point-to-Point Tunneling Protocol), SSTP (Secure Socket Tunneling Protocol), and OpenSSH (Open Secure Shell).

Network Performance Metrics

Here are the key metrics used in this study when testing networks: throughput, latency, and packet loss:

• Throughput: This refers to how much data is sent from one point to another during a specific time frame. Throughput is typically measured in bits per second (bps). It is influenced by the entire infrastructure of the channel, including the physical medium (such as cabling) and computational power, among other factors.

• Latency: Latency is the time it takes to transmit a packet in one direction (e.g., from client to server). In VPN testing, latency is expressed as a time value, usually measured in milliseconds (ms).

• Packet loss: This metric refers to how many packets are “lost,” meaning they did not arrive from the source to the destination. Packet loss can occur due to network congestion, among other reasons. It is measured as a percentage of packets lost relative to the total packets sent.

Unreliability Handling

Unreliability handling refers to how a network manages disturbances, which can be either artificial or natural, such as high latency or packet loss. These disturbances can be quantified by limiting the aforementioned metrics.

For example, consider a remote worker with an unreliable network connection. The unreliability of a network could stem from fluctuating connections, such as cellular networks on a moving train traveling through areas with poor infrastructure or through mountains. It could also involve a remote worker located far from the office, requiring access to files stored across the globe. This would introduce delays far beyond the typical packet transfer delay expected when working with files stored on a server in the same building.

Therefore, it is crucial to investigate how different VPN solutions perform on unreliable networks.

Tools for Measurement

iPerf

The tool used to measure network performance is iPerf3 (https://iperf.fr), version 3.1.3. The reason for choosing iPerf3 is that it effectively tests throughput and provides data on the number of packets transferred, including those that did not arrive. Since iPerf is available for Windows, Linux, and macOS, it is well-suited for this experiment, which is being conducted across all three operating systems.

Operating systems

The test was performed on 3 different operating systems: Windows 10, Linux Ubuntu, and macOS.

pfSense router

The router used in this experiment is a pfSense software router running FreeBSD 11.2-RELEASE-p10 (pfSense 2.4.4-RELEASE-p3). The reason for choosing this router is its capability to shape traffic directly using a tool called dummynet, which is built into pfSense. With dummynet, it is possible to introduce network unreliability aspects (such as delay and packet loss) directly between the VPN nodes. This approach eliminates the need for using dedicated tools or software on the individual nodes themselves, simplifying the setup and ensuring more accurate control over network conditions for the experiment.

Steps for the Experiment

To accurately answer the research question—How does the performance differ between state-of-the-art VPN solutions under stable versus unreliable network conditions?—the following steps were taken:

1. Identify the VPN solutions to experiment on, control traffic, determine which metrics to use, choose the tools, and decide what data to collect

2. Test the Network in the Experimental Setup Without Any VPN Solution to Identify a Baseline Performance

3. Configure and Test VPN Solutions on Three Different Operating Systems with Network Unreliability Conditions

4. Analyze and Compare the Results to Identify Performance Differences Between the VPN Solutions

Important aspects of the default configuration for each VPN that could impact the perform

As mentioned earlier, the three VPN solutions selected for this test are OpenVPN, WireGuard, and IPSec. It is important to note that the default configurations of these VPN solutions were used. For all three VPN solutions, default settings were maintained wherever possible, rather than standardizing the settings across network protocols, cryptographic algorithms, or compression choices.

After setting up the three VPN solutions on three different operating systems, below are some important aspects of the default configuration for each VPN that could impact the perform:

• Encryption Algorithm: The performance of a VPN is significantly affected by the encryption algorithm used. Different algorithms have varying computational complexities, which can impact both the encryption/decryption speed and overall VPN performance.

• Compression: Compression of the payload is another factor. By default, all the tested VPN solutions have payload compression disabled. Enabling compression can reduce the size of the data being transmitted, potentially improving performance, especially in environments with high latency or limited bandwidth.

• Multi-threading: Multi-threading specifies whether the encryption and decryption processes can be distributed across multiple CPU cores. This feature allows parallel encryption on multiple cores, which could increase the speed of encryption and decryption, leading to better overall performance for the VPN.

Testing

The image below illustrates the flow of the testing process:

The testing procedure was structured as follows:

1. Initial Test (No VPN): The first round of testing was conducted without any VPN enabled. This provided a baseline for network performance.

2. Test with VPN Solutions: After establishing the baseline, each VPN solution (OpenVPN, WireGuard, and IPSec) was enabled and tested sequentially. During these tests, the performance metrics (throughput, latency, and packet loss) were measured with the VPNs active.

3. Test with Unreliability #1: Traffic shaping was applied to introduce a 400ms delay (Unreliability #1), and the VPN solutions were tested again under these conditions.

4. Test with Unreliability #2: In the final round, the traffic shaping tool introduced 1% packet loss (Unreliability #2), and the VPNs were tested once more.

This process was repeated for all three VPN solutions (OpenVPN, WireGuard, and IPSec).

Results

The results presented are values from when iPerf sends the packets to the server and server receives and presents the values. All the results presented are the mean values of the 50 tests per case. The tests are on 36 different cases. A table of all 36 individual cases can be seen in table 3.:

Observation

To reiterate, the research question for this paper is: 'How does performance differ between state-of-the-art VPN solutions under stable versus unreliable network conditions?' A definitive conclusion is that each implementation has its own advantages and disadvantages. Some VPN solutions perform better on certain operating systems. The most impactful conclusions drawn from the results are presented below.

• The best-performing VPN solution for macOS, if the network is reliable, was IPSec, followed by WireGuard, with OpenVPN performing the worst

• We can see that with any of the unreliability variables in effect, all VPNs and operating systems had reduced throughput compared to no unreliability

• Linux is fastest with the baseline and no unreliability at 943.5Mbits/sec

• All VPNs except OpenVPN perform best in Linux during the delay unreliability when comparing the other OSs under delay

• All VPNs perform best in Linux while experiencing packet loss

• OpenVPN is not the top performer in any test

• Linux performs best in all unreliability tests except one, the unreliability #1 – delay

Conclusion

Based on the results, a few recommendations for OS and VPN combinations emerged when using the default configuration. They can be seen in the Table below. The VPN in bold with a blue background is the overall recommended option for the specific network situation, as it had the best performance. The recommendations are as follows: