UPM Institutional Repository

Runtime integrity validation of executable C binaries using trusted platform module


Citation

Teh, Jia Yew (2017) Runtime integrity validation of executable C binaries using trusted platform module. Doctoral thesis, Universiti Putra Malaysia.

Abstract

Software developers working under pressure and tight deadlines frequently fail to implement secure programming practices during software development stages. Both constraints are one of the major contributing factors leading to the exploitation of software vulnerabilities for malicious intentions. Furthermore, commercially developed software, owing to intellectual property protection, do not provide the source code – finished products are only shipped in the form of executable binaries. Unfortunately, the exploitation of executable binaries with vulnerabilities, especially those coded with the C programming language leads to malicious, undesirable effects such as buffer overflows, privilege escalation (spawning an attackers’ shell) or premature termination of an executing binary (the C language possesses powerful memory manipulation features i.e. pointers). The following gaps were identified in these three related past efforts into the integrity validation of executables: CBones, Dytan and RTC. First, in CBones, these limitations were identified: a memory debugging tool called Valgrind is required for validation, does not mitigate the vulnerable executable and reports high overheads (Normalized Performance of 0.87). Second, in Dytan, these limitations were identified: requires the use of a dynamic binary instrumentation tool called PINTOOLS for validation, does not mitigate the vulnerable executable and reports high overheads (Normalized Performance of 0.85). Lastly, in RTC, these limitations were identified the executable source code is required for validation and heavy use of static analysis leading to high overheads (Normalized Performance of 1.10). In this thesis, we propose our framework for the runtime integrity validation of executable binaries: Runtime Integrity Validation of Executable Binaries or RIBS. Our framework merges both static (for offline profiling) and dynamic (for runtime validation) analysis techniques for the runtime validation of the integrity of executables compiled with the C programming language. The integrity validation metadata of a trusted, origin executable is stored in a Trusted Platform Module (TPM) hardware register to prevent tampering. Such ensures that the executable binary integrity validation metadata can be totally trusted. In the security evaluation of RIBS, we had subjected RIBS to mitigate 10 categories of buffer overflow attack patterns in the Wilander and Kamkar testsuite (encompassing the stack, heap and data/bss executable userspace memory areas), 5 real world shellcodes and 3 real world applications with buffer overflow vulnerability. RIBS is successful in the detection and termination of all 18 attacks patterns deployed. In terms of performance evaluations, overhead was measured in terms of CPU execution time [via GNU clock()]. We measured the CPU execution time of RIBS and compared the results with the CPU execution time of two major categories of attack mitigation mechanism deployed in the testbed Fedora Core 20 Linux OS: Address Space Layout Randomization (ALSR) and 5 other attack mitigation mechanisms implemented via the gcc compiler. Performance evaluations reveal that RIBS reported highest Normalized Performance (NP) of 0.68, which is the lowest as compared to CBones (0.87), Dytan (0.85) and RTC (1.10). Conclusively, RIBS performs marginally better as compared to all three efforts (CBones, Dytan and RTC) which requires the use of memory debugging tools for integrity validation of executable binaries. RIBS does not require the use of any tools. RIBS is able to detect integrity violations caused by these categories of violations: all forms of buffer overflow attacks mounted via the Wilander and Kamkar testsuite and real world privilege escalation attack shellcodes. Furthermore, as a last line of defence, RIBS is able mitigate integrity violations in executables via the runtime termination of the offending executable. This feature not available in CBones, Dytan and RTC.


Download File

[img]
Preview
Text
FK 2017 24 - IR.pdf

Download (2MB) | Preview

Additional Metadata

Item Type: Thesis (Doctoral)
Subject: Application software - Design
Subject: Computer software - Development
Call Number: FK 2017 24
Chairman Supervisor: Khairulmizam bin Samsudin, PhD
Divisions: Faculty of Engineering
Depositing User: Nurul Ainie Mokhtar
Date Deposited: 29 Aug 2019 08:15
Last Modified: 29 Aug 2019 08:15
URI: http://psasir.upm.edu.my/id/eprint/71109
Statistic Details: View Download Statistic

Actions (login required)

View Item View Item