This document discusses various OS-based attacks and password attacks that can be used by attackers once they have exploited vulnerabilities in operating systems or application software. It describes buffer overflows, stack smashing attacks, dangling and wild pointers, and different types of password attacks like guessing, dictionary attacks, and brute-force attacks. The document is divided into multiple pages and provides details on how these attacks work, including diagrams of call stacks and processes in memory.

1. Computer Security Management
(ISYS20261)
Lecture 8 - Network-based Attacks (3)
Module Leader: Dr Xiaoqi Ma
School of Science and Technology

2. Last week …
• IP address spoofing
• Man-in-the-middle attack
• Denial-of-service attack (DoS)
– SYN flooding
– Smurf attack
– Distributed Denial of Service attack (DDoS)
Computer Security Management
Page 2

3. Today ...
• OS-based attacks
• Buffer overflows
• Stack smashing
• Dangling and wild pointers
• Password attacks
Computer Security Management
Page 3

4. OS-based attacks
• Attackers often look for
– Unpatched operating systems
– Badly designed application software
• Why?
– known vulnerabilities can easily be exploited
• Attacker can then steal, copy, or manipulate data
• Once the OS and services running on the system have been
identified the attacker can mount a number of attacks:
– Stack smashing
– Buffer overflows
– Password attacks
– Etc.
Computer Security Management
Page 4

5. Buffer overflows
• Program tries to write data beyond the bounds of allocated memory
• If not detected and managed by the program data is written in an
unexpected location, causing unexpected results
• Problems:
– Often the program will abort
– The overflow can cause data to be written to a memory-mapped file
– Overflow can cause security problems through stack-smashing attacks
• Example: // ...
int *ptr;
int idx=500;
ptr = new int[500];
ptr[idx]=255;
// ...
Computer Security Management
Page 5

6. Processes in memory
Computer Security Management
Page 6

7. Heap attacks
• Buffer overflow occurs in the dynamically allocated data in the heap
at runtime
• Memory on the heap is dynamically allocated by the application at
run-time and typically contains program data
• Exploitation is performed by corrupting this data in specific ways to
cause the application to overwrite internal structures
• Can be used for example to mount a denial-of-service attack
Computer Security Management
Page 7

8. Stacks
• Stack: data structure that works on the last-in-first-out (LIFO)
principle
push
pop
17 17
17 Storage
for n
255
data
166 items
45
0
99
Computer Security Management
Page 8

9. Stack overflow
• Trying to push a data item onto a stack that is full:
push
17
128
0
17 Storage
for n
255
data
166 items
45
0
99
Computer Security Management
Page 9

10. Stack underflow
• Trying to pop a data item from an empty stack:
push
Storage
for n
data
items
Computer Security Management
Page 10

11. Call stack (1)
• Stores information about the active subroutines (functions) of a
computer program
• Keeps track of the point to which each active subroutine should
return control when it finishes executing
• Stores also local variables and parameters (arguments)
• Implementation is machine dependent
• Stores special data structures called stack frames or activation
records
Computer Security Management
Page 11

12. Call stack (2)
stack pointer
local variables
frame pointer return address stack frame for function n+1
parameters
local variables
return address stack frame for function n
parameters
Computer Security Management
Page 12

13. Stack smashing attack
• Tries to insert arbitrary code into the program to be executed
• Attacker purposely overflows a stack to get access to forbidden
regions of computer memory
• Often used to redirect thread of control to shell, which can then be
used to execute commands on the target system
Computer Security Management
Page 13

14. Dangling and wild pointers
• Pointers that do not point to a valid object of the appropriate type
• Dangling pointers arise when an object is deleted or deallocated,
without modifying the value of the pointer, so that the pointer still
points to the memory location of the deallocated memory
• If system reallocate the previously freed memory to another process
and the original program dereferences the dangling pointer,
unpredictable behaviour may result, as the memory may now
contain completely different data
• Wild pointers arise when a pointer is used prior to initialisation to
some known state
• They show the same erratic behaviour as dangling pointers, though
they are less likely to stay undetected
Computer Security Management
Page 14

15. Password attacks
• Passwords are most common form of authentication of users to an
OS
• Password attacks are most common mode of attack against an OS
• Often default passwords are unchanged: if known it is easy to break
into system
• Other methods
– Guessing
– Dictionary attack
– Brute-force attack
Computer Security Management
Page 15

16. Password guessing
• Passwords are sequences of symbols associated with a user name
• Provide a mechanism for identification and authentication of a
particular user
• Unique and grant privileges only to the account's owner
• If users can choose their own password sequences they tent to use
sequences they can remember easily, e.g. pet names, birth places,
etc.
• Attacker can easily guess passwords!
• Password policy: set of rules designed to enhance computer security
by encouraging users to employ strong passwords and use them
properly
Computer Security Management
Page 16

17. Dictionary attack
• Steal password file from the target machine
• Parsing a word file (dictionary)
• Encrypting or hashing that word (depending on the target system)
• Comparing the result to the encrypted or hashed password from the
victim machine
• If the comparison matches: password found
• Difficult if the correct algorithm is not known or if attacker has not
access to the encrypted password file
Computer Security Management
Page 17

18. Brute-force attack
• Similar to dictionary attack but uses all possible combinations of
letters, numbers, and special characters
• Computationally expensive
• Unlikely to succeed unless password is very small
Computer Security Management
Page 18

19. Next week …
… we will continue to look at web application attacks
Computer Security Management
Page 19