In order to design a set of protections needed within the X server, the set of vulnerabilities that are present in the current X server were investigated. The threats can be broadly classified into the traditional security property categories of confidentiality, integrity, and availability. The protections presented in this paper are designed to prevent compromises of these properties by X clients. We also discuss other relevant properties of secure systems, such as covert channels and the assurance of the system.
The X server does not exist as the only secured element on an otherwise untrusted system. Rather, we assume the X server is running on a secured SELinux system. The kernel enforcement layers will prevent unauthorized clients from connecting to the server or otherwise affecting the kernel structures of the server. Our design only considers clients that successfully connect using the base X Protocol.
We did not investigate attacks directly on the X Server. For example, a buffer overflow may be discovered which allows a client application to control the X server. Such attacks are outside the scope of this work. This analysis is only concerned with syntactically valid series of X11 requests.
There are many ways applications could use a standard X server to bypass a confidentiality policy. As screenshot applications demonstrate, standard X servers offer no protection for an application's output. Clipboard managers can grab content from an application automatically, without any malicious intent. Malicious applications can currently probe the entire state of the X server. By manipulating the input to a client process, a malicious application could cause the victim process to violate a confidentiality policy. The protections presented in this paper will make it possible for a policy to protect the confidentiality of data presented by X clients.
The current X server offers little protection for the integrity of client program data. The X server does not interact directly with the data of the client. Clients can draw directly into another client's window and can send input, or any other type of event, directly to a client. A malicious client can cause another client to present false information to the user. Or an attacker could insert malicious commands into the input stream of a terminal emulator. These types of attacks can cause an application to violate the system's integrity policy, and the protections presented in this paper are designed to protect the integrity of X clients.
The standard X server offers little protection for the availability of the server. Clients can close the windows of other clients, manipulate the font lists, and manipulate the host access lists. The design presented here provides minor improvements for the availability of the X server. While a client will still be able to cause the X server to use excessive CPU via various denial-of-service attacks, the protections presented in this paper will allow a policy to protect the X server against other availability attacks.
The focus of this work is to develop a security framework for the X server which can support strong, flexible security policies. While system assurance is an important element for a secure system, assurance is beyond the scope of this paper. This is commensurate with the assurance effort associated with the underlying system, SELinux, that "has not been on system assurance or other security features such as security auditing, although these elements are also important for a secure system. " [SELinux] The primary goal is to develop a MAC framework for the X server. We expect that formal means of verifying the design should be used.
The X server is a large system component, with a large code base that is hard to evaluate for correctness. Other mechanisms for providing separation between security domains (e.g., various poly-instantiation techniques) would provide increased assurance, but would not be as adaptable to the large number of domains in an SELinux system.
The X server is extensible and still under development. Addressing the common extensions in use and those being developed is beyond the scope of this paper. However, in order to provide complete coverage for the X server, the common extensions will also need to be evaluated from an enforcement perspective.
The SELinux project has not, in general, addressed covert channels. "Even with covert channels, an operating system with basic mandatory controls improves security by increasing the required sophistication of the adversary. Once systems with basic mandatory controls become mainstream, covert channel exploitation will become more common and public awareness of the need to address covert channels in computing systems will increase." [LoscoccoNISS1998]
As with SELinux, the protections presented here do not consider covert channels. However, this design should be compatible with attempts to manage covert channels. A more comprehensive design which considers covert channels will need enhanced visibility controls, and may also need to limit the visibility of changes to the global server objects.