Surviving the frequency of open source vulnerabilities

There is no denying that one of the appeals of Linux is the Do-It-Yourself (DIY) spirit and RYO approach. The most obvious advantages of Linux are easy availability of its source code, freedom from licensing and redistribution costs, the vast amount of system software, middleware, and application software freely available, and the power and flexibility that a multitasking operating system brings to device software.

The Roll Your Own developer environment

Let’s review the 'roll your own' basic requirements.  A Linux developer platform consists of three basic elements: Linux kernel, Root file system, and a Toolchain to produce application software that runs on the target hardware. Rolling your own Linux platform means that you must obtain, build, and support each of these components yourself.  

One hurdle in any roll-your-own Linux platform development project is getting the necessary tools to build system software, application software, and the Linux kernel for your target embedded device. Many developers use a set of tools based on the GNU Compiler Collection, which requires two other software packages: a C library used by the compiler; and a set of tools required to create executable programs and associated libraries for your target device. The end result is a toolchain.

Producing binaries for an embedded target presents an interesting bootstrapping dilemma. Because you do not have Linux running on your target hardware platform yet, you don’t have anywhere to run a native version of the most popular C compiler, GCC. Similarly, many device software platforms don’t have sufficient resources to store and run a native compiler. For these reasons, Linux platform development is typically done on a desktop machine using a special toolchain known as a cross-compiler, which runs on a host system, yet produces binaries that will run on a different target system.

Once the source code is ready, you must then build the components of the cross-compilation environment in the right order. Typically there are many cycles of compiling the kernel, evaluating any compilation problems, fixing that problem, and recompiling, and then eventually a kernel will compile cleanly and may run on your target hardware. The next step is to download the kernel to the board and test it. It is likely you'll need to iterate through multiple kernel build cycles to resolve problems for the kernel to initialise your hardware correctly and display kernel status messages. The process of creating a root file system is essentially the same regardless of whether you will be deploying an in-memory or physical file system.

Once the roll-your-own Linux system boots and correctly executes applications from the root file system, you can begin writing any custom device drivers for hardware specific to your embedded device. If using off-the-shelf hardware, the semiconductor or board manufacturer may provide these device drivers. Otherwise, you will have to write your own drivers and either integrate them directly into the kernel or cross-compile them as loadable kernel modules that the kernel will load from its root file system. Assuming the package cross-compiles cleanly, you can then install it in your working root file system, rebuild that file system if necessary, recompile the kernel if required, and then redeploy and test the updated root file system and/or kernel.

In preference to working on features or product differentiation, developers often spend valuable time supporting, maintaining, and updating a cross-compilation environment, Linux kernel, and root file system.  All of which, requires a significant investment of personnel and wide range of expertise.

The CVE landscape

Compounding the efforts to manage and maintain the development environment, there are high levels of incoming CVEs - the list of common identifiers for publicly known cyber security vulnerabilities. Monitoring the increasing number of exploits, identification, categorising and assessing the severity on your Linux platform is complex and time consuming.

Source: https://www.cvedetails.com/browse-by-date.php

To illustrate the impact, see the above chart.  After setting a record of 16,556 in 2018, the number of reported CVEs in 2019 was on pace to set yet another record.  The most recent information available reflects the first nine months of 2019, which includes 12,174 new CVEs discovered.

Of course, not every CVE that is registered will affect the version of Linux a developer might be using for their RYO, but how do you know that? You still have to do an assessement on a staggering number of exposures, just to find the ones that do affect your release. You might recall two exploits that made a dramatic splash in the public consciousness - Spectre/Meltdown and WannaCry - both illustrative of the type of challenges facing developers in 2019.

Wind River has discovered two very interesting things as vulnerability reports have skyrocketed. First, the frequency of vulnerabilities creates an endlessly moving target that many developers don't have the resources or simply cannot address, which exposes them to significant business risk. Second, the nature of CVEs is not black and white, but many shades of gray.

Conclusion

The Wind River team continuously scans the onslaught of known reports and new vulnerabilities and compares them to the specific components contained in their Linux-based releases and products. The team triages vulnerabilities, prioritises and tracks mitigation actions, and streamlines fixes. The Wind River dashboard provides highlights and their CVE monitoring and notification process enables customers to cut through the flood of CVE notifications and clearly understand which of them will need attention to mitigate and which can be safely ignored.

It's important to consider the real costs of 'free' Linux, which can be an alarming business prospect. Commercially supported Linux is proven to be significantly less expensive and faster to develop than roll-your-own. It also reduces risks related to security vulnerabilities and licensing issues.

Want to learn more? Please visit the Build vs. Buy website or for more information on Wind River Linux.

Courtesy of Wind River.