Research Paper on Rootkit.


Published on

This white paper includes all the basic things about Rootkit, how they work, their types, detection methods, their uses, the concept of payload, and rootkit removal.

Published in: Self Improvement, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Research Paper on Rootkit.

  1. 1. What is a Rootkit? The term rootkit is the combination of "root" -the traditional name of the privileged account on UNIX operating systems and the word "kit" -which refers to the software components that implement the tool. Legitimate Rootkits Rootkits can also be used for what some vendors consider valid purposes. For example, if digital rights management (DRM) software is installed and kept hidden, it can control the use of licensed, copyrighted material and also prevent the user from removing the hidden enforcement program. However, such usage is
  2. 2. no more welcomed than a rootkit that does damage or allows spyware to thrive without detection. Why rootkits are harmful? Rootkit has negative implications through its association with malware as it is a type of Trojan or a type of software, usually malicious, that is activated each time the infected system boots up. It designed to hide the existence of certain processes or programs from normal methods of detection and enable continued privileged access to a computer. Root keeps itself, other files, registry keys and network connections hidden from detection. It enables an attacker to have administrator access to the computer, which means it runs at the lowest level of the machine. A rootkit often allows the installation of hidden files, processes, hidden user accounts, and more in the systems OS. Rootkits are able to intercept data from terminals, network connections, common API calls and the keyboard. For example, it can intercept requests to a file manager such as Explorer and cause it to keep certain files hidden from display, even reporting false file counts and sizes to the user. Rootkits came from the UNIX world and started out as a set of altered utilities such as the “-ls” command, which is used to list file names in the directory (folder).
  3. 3. How rootkit works? Rootkit can be installed by an attacker or their installation can be automated once the Administrator access is gained. A direct attack on a system (i.e. exploiting a known vulnerability, password either by cracking, privilege increment, or social engineering) results in obtaining this access on the system. Once installed, it becomes possible to protect the intrusion as well as to maintain privileged access. The key is the Administrator access. Full control over a system implies that existing software can be modified, including software that might otherwise be used to detect or destroy it. Why are rootkits difficult to detect? Detection of a rootkit is difficult because a rootkit may be able to destabilize the software that is intended to find it, particularly by a kernel-level rootkit as it cannot be trusted to find unauthorized modifications to the rootkit itself or its components. Another reason is that they are activated before the system's OS has completely booted up. Rootkit detectors that work while running on infected systems are only effective against rootkits that have some defect in their hiding mechanisms, or that run with lower user-mode privileges than the detection software in the kernel. As with computer viruses, the detection and elimination of rootkits is still an ongoing struggle.
  4. 4. Detection approaches For kernel-mode rootkits, detection is considerably more complex, requiring careful scrutiny of the System to look for hooked functions where the malware may be disrupting system behavior, as well as forensic scanning of memory for patterns that indicate hidden processes. Removal can be complicated or practically impossible, especially in cases where the rootkit resides in the kernel; reinstallation of the operating system may be the only available solution to the problem. When dealing with firmware rootkits, removal may require hardware replacement, or specialized equipment. Detection by examining storage while the suspect operating system is not operational can miss rootkits not recognized by the checking software, as the rootkit is not active and suspicious behavior is suppressed; conventional anti-malware software running with the rootkit operational may fail if the rootkit hides itself effectively. For Windows, detection tools include Microsoft Sysinternals RootkitRevealer, Avast! Antivirus and WindowsSCOPE etc. Any rootkit detectors that prove effective ultimately contribute to their own unsuccessfulness, as malware authors familiarize and test their code to bypass detection by well-used tools. Detection can take a number of different approaches, including signatures (e.g. antivirus software), integrity checking (e.g. digital signatures), difference-based detection (comparison of expected vs. actual results), and behavioral detection (e.g. monitoring CPU usage or network traffic) and memory dump analysis. They are individually described below.
  5. 5. Alternative trusted medium or operating system The best and most reliable method for operating-system-level rootkit detection is to shut down the computer suspected of infection, and then to check its storage by booting from an alternative trusted medium (e.g. a rescue USB flash drive).The technique is effective because a rootkit cannot actively hide its presence if it is not running. Behavioral-based methods The behavioral-based approach to detecting rootkits attempts to infer the presence of a rootkit by looking for rootkit-like behavior. For example, by describing a system, differences in the timing and frequency of API calls or in overall CPU utilization can be attributed to a rootkit. The method is complex and is hampered by a high occurrence of false positives. Defective rootkits can sometimes introduce very obvious changes to a system. Logs from a packet analyzer, firewall or intrusion prevention system may present evidence of rootkit behavior in a networked environment. Signature-based methods Antivirus software hardly catches all viruses in system scanning (which still depends on which antivirus software is used and to what extent), even though security software providers include rootkit detection into their products. When a rootkit attempts to
  6. 6. hide during an antivirus scan, a stealth detector should notice it; if the rootkit attempts to temporarily unload itself from the system, signature detection or "fingerprinting" can still find it. These collective approach forces attackers to implement counterattack mechanisms, or old fashioned procedures, that attempt to shut down antivirus programs. Signature-based detection methods can be effective against well-published rootkits, but less effective against specially created, custom-root rootkits. Difference-based methods Another method that can detect rootkits compares "trusted" raw data with "malicious" content returned by an API (Application Programming Interface). But, a rootkit may detect the presence of such difference-based scanner and adjust its behavior so that no differences can be detected. Integrity checking methods Code signing uses public-key infrastructure to check if a file has been altered after being digitally signed by its publisher. Alternatively, a system owner or administrator can use a cryptographic hash function to determine a "fingerprint" at installation time that can help to detect successive unauthorized changes to on-disk code libraries. However, unsophisticated schemes check only whether the code has been modified since installation time; the previous version prior to that time is not
  7. 7. detectable. The fingerprint must be re-established each time changes are made to the system: for example, after installing updates More-sophisticated rootkits are able to challenge the verification process by presenting an unmodified copy of the file for inspection, or by making code modifications only in memory, rather than on disk. The technique may therefore be effective only against unsophisticated rootkits - for example, those that replace UNIX binaries like "-ls" to cover the presence of a file. Similarly, detection in firmware can be achieved by computing a cryptographic hash of the firmware and comparing it to a whitelist of expected values. The code that performs hash, compare, or cover operations must also be protected -in this context, that the very to measure security properties of a system must itself be trusted to ensure that a rootkit does not compromise the system at its most crucial level. Memory dumps Making a complete dump of virtual memory will capture an active- on-state rootkit (or a kernel dump in the case of a kernel-mode rootkit), allowing offline analysis to be performed with a debugger against the resulting dump file, without the rootkit being able to take any measures to cover itself. This technique is highly particular, and may require access to non-public source code. Memory dumps initiated by the operating system cannot always be used to detect a hypervisor-based rootkit, which is able to intercept and destabilize the lowest-level attempts to read memory—a hardware device, such
  8. 8. as one that implements a non-maskable interrupt, may be required to dump memory in this scenario. Uses Modern rootkits do not raise access, but rather are used to make another software payload undetectable by adding stealth capabilities. Most rootkits are classified as malware, because the payloads they are bundled with are malicious. For example, a payload might secretly steal user passwords, credit card information, computing resources, or conduct other unauthorized activities. A small number of rootkits may be considered utility applications by their users: for example, a rootkit might cloak a CD-ROM-emulation driver, allowing video game users to defeat anti-piracy measures that require insertion of the original installation media into a physical optical drive to verify that the software was legitimately purchased. Rootkits and their payloads have many uses:  Provide an attacker with full access via a backdoor, permitting unauthorized access to, for example, steal or falsify documents. One of the ways to carry this out is to destabilize the login mechanism For example, GINA on Windows. The replacement appears to function normally, but also accepts a secret login combination that allows an attacker direct access to the system with administrative privileges, bypassing standard authentication and authorization mechanisms.  Hide other malware, notably password-stealing key loggers and computer viruses.
  9. 9.  Fitting the victim machine as a “zombie” computer for attacks on other computers. The attack originates from the compromised system or network, instead of the attacker's system. Zombie computers are typically members of large botnets that can launch denial-of-service attacks (DOS attack) and distribute e-mail spam.  In some instances, rootkits provide desired functionality, and may be installed intentionally on behalf of the computer user:
  10. 10.  Conceal cheating in online games from software.  Detect attacks, for example, in a honeypot.  Enhance emulation software and security software. Daemon Tools is a commercial example of non-hostile rootkits used to defeat copy-protection mechanisms such as SafeDisc and SecuROM. Kaspersky antivirus software also uses techniques resembling rootkits to protect itself from malicious actions. It loads its own drivers to intercept system activity, and then prevents other processes from doing harm to itself. Its processes are not hidden, but cannot be terminated by standard methods.  Anti-theft protection: Laptops may have BIOS-based rootkit software that will periodically report to a central authority, allowing the laptop to be monitored, disabled or wiped of information in the event that it is stolen.  Bypassing Microsoft Product Activation Payload The term 'payload' is used to distinguish between the 'interesting' information in a chunk of data or similar, and the overhead to support it. It is borrowed from transportation, where it refers to the part of the load that 'pays': For example, a tanker truck may carry 20 tons of oil, but the fully loaded vehicle weighs much more than that - there's the vehicle itself, the driver, fuel, the tank, etc. It costs money to move all these, but the customer only cares about (and pays for) the oil, hence, 'pay-load'.
  11. 11. In programming, the most common usage of the term is in the context of message protocols, to differentiate the protocol overhead from the actual data. Another notable use of the term is in malware. Malicious software usually has two objectives: spreading itself, and performing some kind of modification on the target system (delete files, compromise system security, call home, etc.). The spreading part is the overhead, while the code that does the actual evil-doing is the payload. Examples of payloads include data destruction, messages with insulting text or spurious e-mail messages sent to a large number of people. While not all viruses have a payload, some payloads will perform destructive actions. Types of Rootkits There are at least five types of rootkit, ranging from those at the lowest level in firmware (with the highest privileges Ring 0), through to the least privileged user-based level (Ring 3).
  12. 12. Computer Security Rings User mode User-mode rootkits run in Ring 3, along with other applications as user, rather than low-level system processes. They have a number of possible installation routes to intercept and modify the standard behavior of application programming interfaces (APIs). Some inject a dynamically linked library (such as a .DLL file on Windows, or a .dylib file on Mac OS X) into other processes, and are thereby able to execute inside any target process to spoof it; others with sufficient privileges simply overwrite the memory of a target application.
  13. 13. Injection mechanisms include:  Use of vendor-supplied application extensions.  Interception of messages  Debuggers.  Exploitation of security vulnerabilities. Function hooking or patching of commonly used APIs, for example, to mask a running process or file that resides on a filesystem. Since user mode applications all run in their own memory space, the rootkit needs to perform this patching in the memory space of every running application. In addition, the rootkit needs to monitor the system for any new applications that execute and patch those programs' memory space before they fully execute. Kernel mode Kernel-mode rootkits run with the highest operating system privileges (Ring 0) by adding code or replacing portions of the core operating system, including both the kernel and associated device drivers. Most operating systems support kernel-mode device drivers, which execute with the same privileges as the operating system itself. As such, many kernel-mode rootkits are developed as device drivers or loadable modules, such as loadable kernel modules in Linux or device drivers in Microsoft Windows. This class of rootkit has unrestricted security access, but is more difficult to write. The complexity makes bugs common, and any bugs in code
  14. 14. operating at the kernel level may seriously impact stability of the system, leading to discovery of the rootkit. Kernel rootkits can be especially difficult to detect and remove because they operate at the same security level as the operating system itself, and are thus able to intercept or disrupt the most trusted operating system operations. Any software, such as antivirus software, running on the affected system is equally vulnerable. In this situation, no part of the system can be trusted. Operating systems are evolving to counter the threat of kernel-mode rootkits. For example, 64-bit editions of Microsoft Windows now implement mandatory signing of all kernel-level drivers in order to make it more difficult for untrusted code to execute with the highest privileges in a system. Bootkits A kernel-mode rootkit variant called a bootkit is used predominantly to attack full disk encryption systems in which a bootkit replaces the legitimate boot loader with one controlled by an attacker; typically the malware loader persists through the transition to protected mode when the kernel has loaded. The only known prevention against bootkit attacks are the prevention of unauthorized physical access to the system. Hypervisor level This type of rootkit runs in Ring -1 and hosts the target operating system as a virtual machine, thereby enabling the rootkit to
  15. 15. intercept hardware calls made by the original operating system. Unlike normal hypervisors, they do not have to load before the operating system, but can load into an operating system before promoting it into a virtual machine. A hypervisor rootkit does not have to make any modifications to the kernel of the target to destabilize it; however, that does not mean that it cannot be detected by the guest operating system. For example, timing differences in CPU instructions can be noticed and the rootkit can be detected. Hardware/Firmware A firmware rootkit uses platform firmware to create a persistent malware image in hardware, such as a network card, hard drive, or the system BIOS. The rootkit hides in firmware, because firmware is not usually scanned for code truthfulness. Removal Manual removal of a rootkit is very difficult for a typical computer user, but a number of security-software sellers offer tools to automatically detect and remove some rootkits, typically as part of antivirus programs. Experts believe that the only unfailing way to remove them is to re- install the operating system from trusted media. This is because antivirus and malware removal tools running on an untrusted system may be ineffective against well-written kernel-mode rootkits. Booting an alternative operating system from trusted
  16. 16. media can allow an infected system volume to be mounted and potentially safely cleaned and dangerous data to be copied off - or, alternatively, a forensic examination is performed. Even if the type and nature of a rootkit is known, manual repair may be impractical, while re-installing the operating system and applications is safer, simpler and quicker. Author: Anuj Khandelwal Facebook: Twitter: Blog: Courtesy: