Linux kernel code vs FreeBSD kernel code

Linux driver code contains some serious garbage. I heard this refrain, but I did not realize how bad it was until I looked at it myself. Here is just one example.

Device drivers typically read static memory, typically known as EEPROM or ROM, from the chip to identify version, hard-coded information, device capabilities, etc. These values are used throughout execution of the driver. The reading process is among the first things when the device is attached and powered on.

In the case of FreeBSD, after the kernel reads the ROM, it uses a struct pointer with all the variables pre-populated, and points it at the ROM blob data stored in memory. For example:

struct r88e_rom {
	uint8_t		reserved1[16];
	uint8_t		cck_tx_pwr[R88E_GROUP_2G];
	uint8_t		ht40_tx_pwr[R88E_GROUP_2G - 1];
	uint8_t		tx_pwr_diff;
	uint8_t		reserved2[156];
	uint8_t		channel_plan;
	uint8_t		crystalcap;
#define R88E_ROM_CRYSTALCAP_DEF		0x20

	uint8_t		thermal_meter;
	uint8_t		reserved3[6];
	uint8_t		rf_board_opt;
	uint8_t		rf_feature_opt;
	uint8_t		rf_bt_opt;
	uint8_t		version;
	uint8_t		customer_id;
	uint8_t		reserved4[3];
	uint8_t		rf_ant_opt;
	uint8_t		reserved5[6];
	uint16_t	vid;
	uint16_t	pid;
	uint8_t		usb_opt;
	uint8_t		reserved6[2];
	uint8_t		macaddr[IEEE80211_ADDR_LEN];
	uint8_t		reserved7[2];
	uint8_t		string[33];	/* "realtek 802.11n NIC" */
	uint8_t		reserved8[256];
} __packed;

_Static_assert(sizeof(struct r88e_rom) == R88E_EFUSE_MAP_LEN,
    "R88E_EFUSE_MAP_LEN must be equal to sizeof(struct r88e_rom)!");

Notice the assertion at the bottom, which ensures that the ROM struct’s size equals a pre-defined length. The code will fail to compile if this assertion is not valid. Later, the kernel will instantiate a struct pointer and point it to the ROM, stored in the variable buf, as follows:

struct r88e_rom *rom = (struct r88e_rom *)buf;

Now, rom->channel_plan is set to the correct value. Simple.

Unfortunately, this is not how the same code is written on Linux. As mentioned, the Linux driver also begins by reading the ROM blob and storing it in a value called hwinfo. But rather than creating an equivalent struct pointer, the Linux code uses offset values of the ROM on an as-needed basis. For example, the driver reads the channel_plan as follows:

rtlefuse->eeprom_version = *(u16 *)&hwinfo[params[7]];

In this example, params[7] comes from a list of ROM offsets values set in the previous calling function. (That alone made tracing difficult.) The rtlefuse->eeprom_version is now the same as FreeBSD’s rom->version. This manual process repeats for every variable in the ROM.

While that may be just annoying and require a negligible bit more CPU power, this is not be a problem if it was done all in one place. But instead, the driver reads from the hwinfo blob on a seemingly as-needed during execution. And because these as-needed instances are during normal execution, the driver reads-in the same static value from hwinfo every a simple WiFi function occurs, such as changing the channel.

Okay, but even that might not be too difficult…right? Here’s the real kicker.

Sometimes, the driver works by using incrementing offsets from the ROM blob. For example, consider at read_power_value_fromprom (in drivers/net/wireless/realtek/rtlwifi/hw.c). It initializes eeaddr as a u32 (uint32_t), then assigns it with the offset value EEPROM_TX_PWR_INX. So far so good. But then, rather than using new offsets for every successive value, it increments the eeaddr value in multiple doubly-nested for-loops. Here is a simplified version of the code:

for (rfpath = 0 ; rfpath < MAX_RF_PATH ; rfpath++) {
		/*2.4G default value*/
		for (group = 0 ; group < MAX_CHNL_GROUP_24G; group++) { pwrinfo24g->index_cck_base[rfpath][group] =
			  hwinfo[eeaddr++];
			if (pwrinfo24g->index_cck_base[rfpath][group] == 0xFF)
				pwrinfo24g->index_cck_base[rfpath][group] =
				  0x2D;
		}
}

Notice the line hwinfo[eeaddr++]! Merely reading in that variable changes the offset. Its the Heisenberg Uncertainty Principle equivalent of code. This is a cleaned-up version of the 188-line function. The actual function has 6 nested for-loops, some with if-statements, each incrementing the eeaddr parameter as they go along.

Why would anyone do it this way? You are needlessly using up the CPU, making the code difficult to follow, repeatedly reading in static values and making any minor modifications and re-ordering or re-structuring will essentially break the entire function.

And perhaps the worst offender is when 20 functions deep you are not even working with hwinfo anymore. You are working to a pointer to hwinfo that has been incremented God-knows where, with their own offsets that are near impossible to track down.

In my efforts to port this driver to FreeBSD, I literally resorted to printing out the entire ROM, manually finding the memory, and backing into the equivalent offset. Other bizarre code: I have seen if-conditions that are impossible to reach, misplaced code that should go in the previous function, code that does bits of a tasks, while another function does the entire task – so repeat code, unnecessarily repeated code, etc.

How does this make it into the Linux Kernel?

To be fair, this does not appear to be the fault of Larry Finger, who maintains this driver. This is the fault of Realtek, for vomiting this terrible driver in the first place, providing absolutely zero documentation and refusing to respond to any contact attempts.

I hope my FreeBSD port is cleaner and more performant!

Switched from Ubuntu-based to Fedora

tl;dr: Fedora’s debugging packages work, Ubuntu’s are out of date.

Linux = Linux = Linux, whether Arch or Slackware or Ubuntu or OpenSUSE or Linux from scratch as I once did (before there were instructions!). Unless and until the kernel forks and someone decides to modify the syscall table, they all use the same basic syscalls, they typically share the same basic libraries and core utilities, etc. They’re all the same.

Why did I use Ubuntu-based distributions? (Note: Not Debian) Because Ubuntu came pre-configured with all the things I did not care to learn or manually configure: ACPI, firmware, X11, a pretty WM theme, etc. I did not particularly care whether I was running Mint, Elementary, Ubuntu MATE or basic Ubuntu (except Unity…nah). As long as it did not do strange things like remove /sbin/ifconfig or have a radically different file structure than I was used to. I felt at home with knowing where the standard file paths were, and knew how to administer my machine. Their package repository was pretty solid. It had almost everything I wanted – and what little was not on it was typically available in Debian-package format. The broader Linux community effectively standardized on this package format. This is crucial. Debian’s apt and FreeBSD’s pkg are in my muscle-memory at this point.

Literally one thing pushed me over: Ubuntu’s SystemTap was broken. Utterly broken!

I got into OS-level programming, specifically, porting a Linux WiFi driver to FreeBSD. I wanted to use SystemTap, Linux’s answer to DTrace, to help understand what is going on during live execution. But SystemTap does not work on Ubuntu – at least currently.

But wait, I thought Linux = Linux = Linux and programs from 20 years ago will still work. Why does SystemTap fail?

SystemTap works by producing C code for a kernel module, compiling it and loading it into memory. Sometime ago, the kernel team changed the get_user_pages() kernel API call. This meant that any code compiled against the old function definition failed. I encountered this in the professional space when the VMWare kernel modules failed to build and I hacked it until it worked. (They think I’m a wizard now). I was on Kernel 4.10 but the version of systemtap Ubuntu used was nearly 2 years old. This meant no one from the Ubuntu team was using it.

I submitted a bug report and installed Fedora 26.

SystemTap was developed by Red Hat and was trivial to get working under Fedora. And while not every single package is available (Bitcoin, Steam thus far), there is enough that moving over was trivial. Also, they come in Cinnamon, which I prefer, with a pretty theme. And it provides a clean terminal out-of-the-box. Which I need. (I would rather use stock XFCE if their terminal was clean than fully-loaded CentOS with an ugly terminal)

dnf took a little getting used to, but a hop-over from apt. So whatever on that front….

I would be willing to try OpenSUSE again, but the latest time I did, they got rid of /sbin/ifconfig for /sbin/ip, which is unacceptable. Silly, perhaps…Does it come in Cinammon? What does it offer? Are the packages as clean and up to date? I may never know, unless another business-need arises. I do not care to run any of these “hardware” distributions, like Arch. I paid my Linux dues around kernel 2.2 on Slackware and its time to move on from that.

Thoughts?

But look, if you’re 99% of the Linux world, any specific distribution is trivial. Pick one and go with it. Unless you’re doing very specific tasks like me, it really does not matter what you use. So stop Distro Hoping!

Custom Kernel Modules for Chromebook

Note: I wrote this about a year and a half ago, but I refer to it all the time. Hopefully the instructions have not changed too much! Enjoy!

I recently purchased a Chromebook. It’s great, it symbolizes the direction the PC market should head – inexpensive, low-powered ARM processor, defense in depth resistance to malware and simple for non-technical users. And with crouton, it functions quite cleanly as a Debian-based workstation.

With its simplicity and low price, there are certain key features that are lacking in the stripped down Linux kernel that can make it frustrating for a power-user. Unfortunately, Chromium addons have not or cannot satisfy some tasks that require kernel-level functionality. Even in crouton, you may find your ability limited to the user-space. Those looking for casual additions, recompiling the kernel may seem like daunting over-kill. Instead, compiling and inserting a single module may serve as an apt alternative. In this guide, I will explain how to compile a custom kernel module to add additional functionality to your Chromebook and how to circumvent the built-in security mechanisms that prevent you from adding into the kernel-space.. This guide is specifically written for an ARM-based CPU using kernel 3.10.18 for the CIFS (SMB) module, but can be trivially ported to any other architecture, kernel and module.

Compiling the Kernel Module

As mentioned, Chromium OS is a stripped down version of Linux. Therefore, you should be able to compile and dynamically link kernel modules from the stock kernel into Chromium.

Per Google’s documentation, you must compile the kernel and modules on an x86_64 CPU, even if you will be compiling an ARM or 32-bit x86 module. This is possible thanks to GNU C Compiler’s cross-platform capability. The documentation also specifies using Ubuntu, but it worked just fine on my Debian 8 workstation.

If you have not already done so, install git, subversion and perform the basic configurations:

sudo apt-get install git subversion
git config --global user.email “name@domain.tld”
git config --global user.name "Your Name"

Google manages its various git repositories with wrapper depot_tools, a custom git wrapper. You can clone the associated git repository and set your PATH environmental variable to include the wrapper scripts as follows.

git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git

export PATH=`pwd`/depot_tools:"$PATH"

Next, make a directory where your Chromium OS build will reside, download the Chromium source, and synchronize it to the latest updates. This take around 30 minutes to complete.

mkdir chromiumos && cd chromiumos
repo init -u https://chromium.googlesource.com/chromiumos/manifest.git
repo sync

Once completed, you will need to download the cross-platform SDK environment, build the dependencies and enter a chroot(1) environment. This will take another 30 minutes.

cros_sdk

Now that you are inside the chroot(1) environment, you need to specify the hardware configuration for your Chromebook device, either x86-generic, amd64-generic or arm-generic. You can determine your architecture by running uname -m on your Chromebook. For my ARM-based CPU, I did the following:

export BOARD=arm-generic

Now you must prepare the core packages associated with your board.

./setup_board --board=${BOARD}
./build_packages --board=${BOARD}

Change directory to ~/trunk/src/third_party/kernel/ and then to whichever subdirectory is associated with your kernel (ie, v3.10 for 3.10.18). You can determine your kernel version by running uname -r on your Chromebook.

Next, we will need to tell the kernel which hardware platform you are on and start with the base configuration of the kernel. A list the options of base configurations by running find ./chromeos/config. In my case I am using NVIDIA’s Tegra motherboard, which is ./chromeos/config/armel/chromeos-tegra.flavour.config, so I use chromeos-tegra as follows:

./chromeos/scripts/prepareconfig chromeos-tegra

If you are compiling for a non x86_64 CPU, set the architecture and compiler settings as follows:

export ARCH=arm
export CROSS_COMPILE=armv7a-cros-linux-gnueabi-

This next portion is the same as compiling any other kernel module. Configure the kernel by running make menuconfig

Select whichever controls you would like to install and save. Once completed you will have a .config file that corresponds to your hardware. Since we are only compiling the kernel modules, you can either run make modules to compile all kernel modules, or make fs/cifs/cifs.ko to build only a specific module. I prefer the former because your module may require other dependencies in other modules, such as with crypto/md4.ko for cifs. You can verify that the file was built for the right architecture by running file fs/cifs/cifs.ko. Great! On to inserting the module!

ChromiumOS’s Security Mechanisms

ChromeOS is the official signed release of ChromiumOS, which is what you run in developer mode. Even in developer mode, Google implemented multiple defensive mechanisms to slow down a would-be attacker from gaining access the underlying system. To protect the kernel, Google utilized the Linux Security Module (LSM), which validates files from the root partition against a list of cryptographic hash values stored in the kernel, thereby preventing an attacker from loading a malicious kernel modules. In effect, the only way to insert a kernel module is to have it stored on the root partition. But by default, the root partition is set to read-only, so you cannot simply move a file to the root partition and load it.

Therefore, we must disable the root partition verification running the following script.

sudo /usr/share/vboot/bin/make_dev_ssd.sh --remove_rootfs_verification --partitions 4

Now, reboot the machine and from ChromiumOS remount the root partition to be read-writeable, as follows:

sudo mount -o remount,rw /

From here, you should be able to simply insert the kernel module with insmod. Now, you can install
Enjoy!

FreeBSD and Linux Remote Dual Booting

The following is a quick and dirty guide on how to setup remote dual booting for FreeBSD (12.0-CURRENT) and Linux (Ubuntu 16.04). Granted, this method is slightly a hack, but it works and suits my needs.

Why remote dual-booting? I am currently developing a FreeBSD kernel module for a PCIe card. The device is supported on Linux and I am using the Linux implementation as documentation. As such, I find myself frequently rebooting into Linux to look printk() outputs, or booting into FreeBSD to test kernel code. This device is located at my house, and I typically work on it during my downtime at work.

Why not use Grub? I would have preferred Grub! But for whatever reason, Grub failed to install on FreeBSD. I do not know why, but even a very minimalistic attempt gave a non-descriptive error message.

efibootmgr? Any change I made with efibootmgr failed to survive a reboot. This is apparently a known problem. Also, this tool only exists on Linux, as FreeBSD does not seem to have an efibootmgr equivalent.

Ugh, so what do I do???

The solution I came up with was to manually swap EFI files on the EFI partition no an as-needed basis.

First, I went into the BIOS and disabled legacy BIOS booting, enabled EFI booting, and disabled secure booting.

Then, I installed Ubuntu. I had to manually create the partition tables, since by default the installer would consume the entire disk. However, this does not automatically create the EFI partition. So, you must manually create one. I set mine to 200MBs as the first partition. After installation, I booted up, mounted the /dev/sda1. I found that ubuntu had created /EFI/ubuntu/grubx64.efi and other related files. Great!

Next, I installed FreeBSD and while manually setting up the partition tables, FreeBSD auto-created an EFI partition. One already exists, so I safely deleted it, and proceeded with the rest of the install. Right before rebooting, I mounted /dev/ada0p1 (sda1 on Linux) as /boot.local/ and /dev/da0p1 as /boot.installer/. I then copied /boot.installer/EFI/BOOT/BOOTX64.EFI too /boot.local/EFI/BOOT/EFIBOOT/BOOTX64.EFI (I think I had to re-create EFI/BOOT, I’m forgetting off-hand). Then I rebooted.

When I rebooted the machine, Ubuntu still came up. This is because Ubuntu edits the EFI boot order and places ubuntu as the first partition. Ordinarily you should be able to use efibootmgr here to boot into FreeBSD and use the non-existent FreeBSD equivalent to boot back, but with the lack of that option, I mounted the EFI partition (/dev/sda1) as /boot/efi, and when I wanted boot into FreeBSD, I renamed /boot/efi/EFI/ubuntu/grubx64.efi to ubuntu.efi and then copied /boot/efi/EFI/BOOT/BOOTX64.EFI to /boot/efi/EFI/ubuntu/grubx64.efi. When I rebooted, FreeBSD came back up! Then on the FreeBSD side, I mounted /dev/sda1 to /boot/efi and did copied /boot/efi/EFI/ubuntu/ubuntu.efi to /boot/efi/EFI/ubuntu/grubx64.efi.

And that’s it! I can now remotely boot back and forth between the two systems.

Ugly? Yes. But it does the job.

Linux could fix this problem by debugging their efibootmgr utility and FreeBSD could fix this by having an efibootmgr equivalent at all.

Thoughts?

Easytether running on ARMv7l Chromebook

Easytether is a proprietary application, so you can’t just apt-get it on a chroot’ed crouton environment. Additionally, I am not running an x86 or x64 CPU, so most of these packages will not work.

Crouton mostly comes for Debian, so I downloaded the Ubuntu package. I installed the package ‘alien’. Then unzipped the deb file as follows:

alien easytether_0.8.5-1_armhf.deb

This will 3 tar.xz files, mostly importantly data.tar.xz. Unzip it with the following command:

tar xvfJ data.tar.xz

Now go into the newly created usr/bin directory and execute the easytether-usb command as root and it should work.

LibreSSL: The Secure OpenSSL Alternative

I originally published the following article with the InfoSec Institute, but I figured I would re-publish it on my personal blog.

Perhaps the most devastating vulnerability in recent years was OpenSSL’s Heartbleed exposure. This is just the latest in a series of major vulnerabilities affecting a linchpin security software package. Why does this continue to happen? What are the solutions? Moreover, could LibreSSL be an industry-level replacement to OpenSSL?

OpenSSL was initially developed in 1995 as a commercial-grade, open-source and fully featured toolkit for implementing Secure Socket Layer (SSL) and Transport Layer Security (TLS), and as a general-purpose cryptographic software library. OpenSSL was adopted by various operating systems and major applications as their primary cryptographic library, including all main Linux distributions, BSD variants, Solaris and is heavily utilized by Windows. However, perhaps due to its success, rapid adaption and diverse implementation, the quality of the code eroded. What should be mature code with a relatively simple and concise objective has grown to become an array of tangentially related features, patchwork of “quick fixes” and excessive backwards compatibility and portability to the detriment of the security of product.

Enter LibreSSL. LibreSSL began as a fork of OpenSSL 1.0.1g. Developed by the OpenBSD team, LibreSSL is designed to be a drop-in replacement of OpenSSL. Its stated goals are code modernization, security and software development best practice. In the past 18 months, the code has made impressive strides in said goals. It has also made controversial decisions, including removing widely used features and jettisoning oft-used government standards. Could this software package replace OpenSSL?

This paper seeks to compare OpenSSL and LibreSSL as the main encryption library for production environments by:

  • Direction and Progress
  • Security
  • Performance
  • Compatibility and Portability
  • Functionality

Code Modernization and Cleanup

A primary critique of OpenSSL is that the code-sacrificed industry best practices, code review and remediation, and structured development in favor of rapid portability and functionality. Specifically, they argued the following problems:

  • Multiple implementations of a task, depending on the OS, architecture or compiler, resulting in multiple points of failure.
  • Unorthodox implementations to accommodate failures in Operating Systems, architectures or compilers.
  • Code unreadability, such as a labyrinth of C preprocessor statements.
  • Custom implementation of standard C library calls, such as printf() or memory management.
  • A patchwork of “quick-fixes” rather than solving fundamental design flaws.

Modernization

OpenSSL aims for portability across all systems, including those that do not support standard C functions. It accomplishes this by creating an abstraction layer and reimplementing APIs that should be in provided by the OS. This resulted in a non-standard OpenSSL C Library variant, distinct from standard C that does not undergo the same level of public scrutiny, maturity and secure best practice.

LibreSSL tackles these issues assuming that the code is designed for a modern, POSIX compliant OS on a modern architecture using a compiler that conforms to standard C. Namely, LibreSSL assumes that the target system is OpenBSD. In cases where an OpenBSD-specific C call does not exist, LibreSSL ports only that function and maintains the same symbol, as not to create an abstraction layer.

Additionally, this assumption afforded LibreSSL developers the freedom to remove antiquated, CPU-specific, OS-specific, compiler-specific or otherwise non-standard code. One example of this is the get_ip() function which parses an IP address string and returns each octet in a character array. Rather than utilizing gethostbyname(), which became part of the GNU C Library 1997, OpenSSL chose this implementation because Visual C 1.52c, dated 1995, contained an unresolved DLL error with the sscanf() function. To date, LibreSSL removed 140,000 lines of code, or 23%.

Memory Management

OpenSSL insisted on portability on every platform, irrespective of its capabilities. This became problematic when the OS or the C Library was unable to meet cryptographic requirements. Examples included:

  • Operating systems that did not contain the malloc(3) function
  • Unacceptable latency when allocating or freeing memory
  • Inability to allocate 64-bit objects

OpenSSL solved these problems by developing an internal memory management system, distinct from the OS’s. This layer was a fixed Last-in-First-out stack that assigned memory to requesting objects, but never sanitized or freed used memory back to the OS. While this assisted in portability, it created numerous challenges, namely:

  • OpenSSL code maintenance and readability became extremely difficult.
  • Common debuggers, such as Valgrind, are designed to analyze fixed-buffers assigned to individual variables. Abstracting memory management from the OS made buffer overflow detection virtually impossible.
  • The LIFO memory architecture virtually ensured that exposed memory contains sensitive information, as evident by the Heartbleed vulnerability.
  • Creating memory leaks that could not be detected by garbage collectors.

LibreSSL simplified the process by shifting the responsibility of memory management back to the OS. To this end, LibreSSL entirely removed the OpenSSL memory management layer, simplified the higher-level macros, and utilized POSIX C libraries. Developers comment that this revealed thousands of buffer overflow errors, which were subsequently corrected.

Ticket Remediation

The OpenSSL Software Foundation (OSF) appeared slow or uninterested in remediating documented issues. As a metric, upon LibreSSL’s initial release in April 2014, OpenSSL’s bug ticket system contained 1104 open tickets with the earliest dating back to April 2002. To date, 115 item from that list still unremediated. The stagnation of ticket remediation exacerbated disinterest in community contribution. As part of its goal, LibreSSL remediated all open tickets and shared newly vulnerability newly identified vulnerabilities to the OpenSSL team.

Security

Security is a central goal for LibreSSL, which they aim to achieve by increasing code readability and review, removing insecure functionality and memory sanitization.

Code Readability

As discussed, LibreSSL assumes a level of competence of the underlying platform. This afforded LibreSSL developers the freedom to remove large swaths of confusing or unwieldy code, such as nested preprocessor statements for specific OSs or architectures. Though this may not immediately result in enhanced security, it has allow outside developers to review the code with greater ease and contribute to security and overall quality.

Functionality Removal

With the disclosure of the POODLE vulnerability, security analysts deemed SSLv3 insecure and not for use in production environments. However, many environments still require SSLv3 for legacy purposes and are unable to migrate to TLSv1.1. To this end, OpenSSL maintains backwards compatibility with SSLv3 and all applications compiled for SSLv3 are still operational. Conversely, LibreSSL 2.2.2 entirely removed SSLv3 support. Additionally, LibreSSL removed weak or broken cipher suites, including the Secure Remote Password (SRP), Pre-Shared Key (PSK) and all Export ciphers. LibreSSL added support for ChaCha, GOST and IDEA ciphers. While LibreSSL’s decisions provide technically enhanced security, this may be negatively impactful in environments that require legacy SSLv3 support.

Randomization

Random number generation is a critical aspect of cryptography. Without proper seeding, an attacker is able to predict private encryption keys.

OpenSSL relies on the operating system to generate random numbers. On UNIX and Unix-like systems, this typically means reading from /dev/urandom or /dev/random. However, in the event that OS is unable to generate random numbers, OpenSSL provides the RAND_add(3SSL) and RAND_seed(3SSL) API calls, allowing users to seed the PRNG function.

LibreSSL shifts the responsibility of random number generation entirely onto the OS. LibreSSL vestigially maintains the function for binary compatibility, but removed all associated code. It is worth noting that due to the lack of seeding, the initial release of LibreSSL contained a critical flaw in the PRNG function, where two forked processes contained the same PRNG seed and thus generated the same random data. This vulnerability was quickly patched.

Memory Sanitization

Memory sanitization is a central feature in LibreSSL that is lacking in OpenSSL. Prior to the deallocation of objects, LibreSSL explicitly zeros out memory using OpenBSD’s explicit_bzero(3) function. This proactively reduces the impact of memory exposure in the event of a future vulnerability or an unprivileged process that gains control of a tainted memory segment. The LibreSSL team created portable a module for OSs that do not have these OpenBSD-specific API calls.

Common Vulnerability Enumerations

Since LibreSSL is a fork of OpenSSL 1.0.1g, it is subject to many of the same issues and vulnerabilities that affect many of OpenSSL. From the date of LibreSSL’s initial release in April 2014, to today, there are 45 CVEs that affect OpenSSL. Of those, only twenty-four affected LibreSSL, with another one vulnerability only affects LibreSSL. This is demonstrable proof that LibreSSL has made strides in security.

FIPS 140-2 Compliance

The OSF sought to make OpenSSL FIPS 140-2 compliance. Consequently, OSF created the OpenSSL FIPS Object Module, which provides an API to invoke FIPS approved cryptographic functions. This is particularly important for Federal IT systems that are obligated to comply with FIPS 140-2. Conversely, LibreSSL removed the FIPS compliant module from its code, arguing that the FIPS 140-2 uses weak or broken ciphers and is detrimental to security. LibreSSL is neither FIPS 140-2 compliant nor is it a goal of the project. This is potentially impactful for risk-averse government IT systems that are required to comply with FIPS 140-2.

Portability

As discussed, LibreSSL was written specifically for OpenBSD and the OpenBSD team ported LibreSSL for modern OSs that conform to POSIX standards. Consequently, legacy systems or systems that do not conform to POSIX are unable to run LibreSSL.

Application Support

LibreSSL aims to be a drop-in replacement to OpenSSL with no changes to existing applications. LibreSSL achieves compatibility by maintaining exposed API calls, even if their functionality is nullified. However, many applications, such as Apache Web Server, must be minorly patched before it will link with LibreSSL. This is mostly due to removed features or removing unnecessarily exposed interfaces.

OS support

The LibreSSL team ported it to numerous operating systems, including Linux, Windows, FreeBSD, OS X, and Solaris. It is worth noting that outside of OpenBSD, no operating system has pre-compiled package-level support for LibreSSL. Thus, all binaries and applications must be manually compiled and manually upgraded. LibreSSL explicitly removed support for Classic Mac OS, 16-bit Windows, DOS, VAS, and OS/2.

Performance

Performance tests were performed on a stock Ubuntu 14.04 LTS install running kernel 3.19.0-28-generic using Apache 2.4.16. Both virtual machines were running on a single core from an Intel(R) Core(TM) i7-4712MQ CPU @ 2.30GHz CPU with 2 gigabytes of RAM. The tests were running OpenSSL 1.0.1g. and LibreSSL 2.3.0, respectively. The test utilized ApacheBench version 2.3 with 100,000 requests per test for a payload of 45-bytes.

In terms of speed, OpenSSL outperforms or is at best comparable to LibreSSL across all tested ciphers. The performance impact is attributed to the explicit memory sanitization operations within LibreSSL that are not present in OpenSSL and optimization features that were disabled in LibreSSL.

Conclusions

Both LibreSSL and OpenSSL have advantages and disadvantages, depending on a given application, requirement or tolerance.

From a purely security perspective, LibreSSL is the clear winner, by proactively addressing security concerns, disabling broken ciphers and protocols, and building security into the design. OpenSSL employs a reactive approach and fails to address adequately security in design. In addition, while they still share many of the same vulnerabilities, one can anticipate that LibreSSL will be subject to a diminishing number of shared vulnerabilities and less overall going forward.

However, this approach also has a downside. LibreSSL’s focus on security means that it is not backwards compatible with deprecated ciphers or protocols and therefore will not function in legacy environments. Additionally, LibreSSL’s stated refusal to comply with FIPS 140-2 effectively guarantees government systems and enterprise-level OSs will never utilize it.

OpenSSL outperforms LibreSSL in portability. This is not due to a failure in LibreSSL, but is instead a testament to OpenSSL’s successful adaptation. Since no major operating system currently supports LibreSSL, LibreSSL and dependent applications must be manually compiled and installed, rather than utilizing the package management systems in FreeBSD, Debian, Ubuntu or Redhat.

In terms of performance, OpenSSL exceeds LibreSSL across all ciphers. As stated, this is attributable to LibreSSL’s explicit memory sanitization operations. The performance may be negligible in low-availability applications, but can be impactful in large, latency-sensitive production environments.

Both LibreSSL and OpenSSL have their strengths and weaknesses. OpenBSD has demonstrated that LibreSSL is ready for production environments, but has yet to be widely deployed. Ultimately, the choice between LibreSSL and OpenSSL is a variant of the perennial question of security versus functionality.

Differences between Mint and Ubuntu

I looked into the differences between Mint and Ubuntu to see which was best for me. I watched tons of videos, reviews, comparisons, ran them both for months, etc. Here’s what I learned…

They’re the same damn thing. No really, they are identical. The only differences are what software comes pre-installed and some user interface prettiness. Otherwise, no difference in the underlying system at all. Literally no difference.

Next question?