OS Hardening

Summary

In this article the term security is explored and applied to the security requirements of an operating system. A general overview is given on how OS security is defined. Then, the three main sectors on OS security are presented as a high-level overview.

This is based on the seminar paper "OS Hardening - Making systems more secure" (File:OS Hardening.pdf), which focuses on creating safe execution environments for exploitable applications with buffer overflows.

Operating System Security Evaluation

Creating a completely secure system is hard if not impossible to achieve. Striving to such a goal gets even harder when dealing with enormous complex systems.

There are three main evaluation criteria for testing and classify computer systems:

• TCSEC: Trusted Computer System Evaluation Criteria

developed under the United States Government Department of Defense

• ITSEC: Information Technology Security Evaluation Criteria

published under the Commission of the European Communities

• CC: Common Criteria

These two standards got unified under ISO/IEC 15408 standard.

These standards classify computer systems on a set of criteria ranging from minimal protection (EAL 1) to verified protection (EAL 7).

To bring these requirements into perspective here is a short list, for common OS classification:

• EAL 4+: Red Hat Enterprise 7.1 (2017), Windows XP (2005), Windows 2008 (2008)
• EAL 3: Apple Mac OS X 10.6 (2009)
• EAL 2: Ubuntu 16.04 (2018)

It has to be noted that examination takes a long time to conducted and only applies to one specific version. This is one of the reasons why such investigations are performed rarely. Additionally, they are very costly, leading to only a marginal gain for the product.

Common Threats

The main security purpose of an operating system is the separation of user data and applications. Other applications or users should not be able to access data they are not authorized to. The operating system has different techniques for achieving this goal. But before we look into the protection and hardening of an operating system, the most prominent attacks are presented. This list of threats is by no means complete and should only give a brief overview.

Application Exploit

Application exploits utilize programming errors to take advantage of the system with the goal in mind to have some kind of privilege escalation and accessing files that should normally not be accessible to a not authorized user.

Buffer Overflow

Buffer overflows are a subtype of an application exploit. As they are one of the more common attacks out there we examine them closer. Buffer overflow is the practice of exploiting unprotected size-unrestrained memory buffers. The goal of such an attack is the modification of the return pointer of a program, enabling the attacker to jump to an arbitrary location in memory. For more info, please visit Buffer Overflows

Race Condition

Race conditions can occur when concurrent processes can read or write on the same memory location. This attack is also known as time-to-check and time-of-use (TOC/TOU) as a shared value is checked and subsequently used to exploit some kind of software bug.

For example, an attacker can create a symbolic link to a file in the precise moment a privileged program wants to write to it. If this symbolic link points to \textit{/etc/passwd}, the privilege program unwillingly overwrites the password file, allowing the attacker root access to the machine

Other Attacks

There are many more techniques attacking the CIA requirements of an operating system. Many malicious actors attack the confidentiality as it usually has the most value to an adversary.

Other attacks on the CIA criteria include SQL injections, web exploits, password cracking, denial-of-service attacks, eavesdropping attacks and malware.

Background of attackers

In the 2019 Verizon data breach investigations report shows that the motives of threat actors were 67% financially motivated, 20% espionage driven, 8% for fun and self-esteem related.

OS Hardening

It is to be noted that achieving a completely secure system is almost impossible. What makes this especially challenging is the fact, that a general-purpose operating system has to account for a diverse set of applications running on it. The operating system has only a small influence changing the behavior of a program, as the source code is mostly controlled by a third party. It is the job of the operating system to manage all of the programs and create a safe environment for the user and the applications.

Safe Environments

Even though all applications operate in the same memory and on the same CPU, they should not be able to access files that they are not authorized to.

Kernel-Enforced Protection

The kernel does not know the internal functionality of the executing program, therefore it can only modify the layout of the memory and enforce access control rights.

Memory Access Control

By creating non-executable (NOEXEC) memory spaces. This prevents the execution of injected shellcode in the heap or the stack of an application.

Memory Address Randomization

Address Space Layout Randomization (ASLR) introduces randomness of the placement in virtual memory. For an attacker it is therefore hard to know where the location of variables, binary, libraries, heap and stack are in memory, as it is different for each execution.

Compiler-Enforced Protection

This method tackles the problem during the compilation of the program. The compiler leverages the knowledge it has on the structure of the program and could modify it in the way of securing against buffer overflows.

Stack Canaries

The compiler can insert special data, called canaries, into different parts of the program memory that get checked during the program execution. If an attacker tries to overflow a buffer, the possibility of overwriting a canary is very high, since they are placed in strategical positions.

For a more practical visualization please visit Buffer Overflows.

Access Control

Access control on an operating system is needed to preserve the confidentiality of a multi-user and multi-application system. The system has to control who is allowed to access, modify and execute certain data.

Access Control List

In Linux basic file access control is based on the traditional UNIX file model. Each file and directory has a total of 9 bits and some special markers to set the permission. The three bits resemble the privilege to read (r), write (w) and execute (x) a given file.

Dividing the access control into three groups can be quite limiting, especially if there are users that need special permissions and is called simple ACL. Maybe we want to grant permissions to additional users or groups. This is where the traditional system becomes quite constricting.

$ ls -l example.txt 
 -rw-r--r-- 1 root root 0 Jan 31 21:32 example.txt

Therefore, the extension for a extended ACL is provided in Linux per default and enables the definition of additional users (named user) and additional groups (named group). Accessing and viewing the extended ACL in Linux is possible with the command getfacl

// view extended ACL
$ getfacl example.txt

 # file: example.txt
 # owner: root
 # group: root
 user::rw-
 group::r--
 other::r--

// modify or add permissions
$ sudo setfacl -m "u:someuser:rwx" example.txt
$ sudo setfacl -m "g:somegroup:r-x" example.txt 
$ getfacl eammple.txt 

 # file: example.txt
 # owner: root
 # group: root
 user::rw-
 user:someuser:rwx
 group::r--
 group:somegroup:r-x
 mask::rwx
 other::r--

// view with ls command (note the # that indicates the extended acl)
ls -l example.txt 
 -rw-rwxr--+ 1 root root 0 Jan 31 21:32 example.txt

// remove permissions
$ sudo setfacl -x user:someuser example.txt
$ getfacl example.txt 
 # file: example.txt
 # owner: root
 # group: root
 user::rw-
 group::r--
 group:somegroup:r-x
 mask::r-x
 other::r--

// view with ls command
$ ls -l example.txt 
 -rw-r-xr--+ 1 root root 0 Jan 31 21:32 example.txt

Manditory Access Control

The aforementioned method of access control is called the Discretionary Access Control (DAC) method and is the most common mechanism to enforce confidentiality. The idea behind DAC is that the owner specifies who can access an object. This decision is based on the owner's discretion.

The so-called Mandatory Access Control (MAC) tries to tackle the problem with a changed fundamental concept. Instead of letting the user decide the permission of objects, each user is given a certain clearance and each object is given a security classification. Only when a user has a higher clearance level as the required security classification (secret, top-secret, confidential) the system allows access.

Well-known implementations in Linux that implement MAC is SE-Linux and AppArmor.

Reducing the Attack Surface

Reducing the attack surface is a vital part of securing the operating system. As seen before we can use access control to prevent flawed applications to breach confidentiality. We should not rely on this technique entirely and should reduce the possible attacks in the first place.

The most common techniques on reducing the attack surface include:

• Removing unnecessary services
• Secure the network (firewalls)
• Securing the hardware (physical)

References

Seminar paper

• File:OS Hardening.pdf

References

• Commission of the European Communities. Information Technology Security Evaluation Criteria (ITSEC): Provisional Harmonised Criteria. Office For Official Publications Of The European Communities, Luxembourg, 1991.
• Luis Franco, Tony Sahama, and Peter Croll. Security Enhanced Linux to Enforce Mandatory Access Control in Health Information Systems. 2008.
• James Graham. Cyber Security Essentials. Auerbach Publications, 2011.
• Andreas Grunbacher. POSIX Access Control Lists on Linux. USENIX Annual Technical Conference, FREENIX Track, 2003.
• Aneesh Kumar, Andreas Grünbacher, and Greg Banks. Implementing an advanced access control model on Linux. 2010.
• Vasudevan Nagendra and Yaohui Chen. Access Control Lists in Linux & Windows. Stony Brook University, 2014.
• P. K. Patra and P. L. Pradhan. Hardening of UNIX Operating System. Int J. of Computer Communication and Technology, 1, 2009.
• Abraham Silberschatz, Peter B Galvin, and Greg Gagne. Operating System Concepts. 2013.
• Peter Silberman and Richard Johnson. A Comparison of Buffer Overflow Prevention Implementations and Weaknesses.
• Verizon. 2019 Data Breach Investigations Report. Technical report, 2019.

Links

• https://www.redhat.com/en/about/press-releases/red-hat-adds-common-criteria-security-certification-red-hat-enterprise-linux
• https://www.schneier.com/blog/archives/2005/12/microsoft_windo.html
• https://www.niap-ccevs.org/Product/Archived.cfm?par303=Microsoft%20Corporation