The document discusses program security and secure programming. It covers various types of programming errors that can lead to security issues like buffer overflows and incomplete access controls. It also discusses malicious code like viruses, worms, and Trojan horses. The document outlines controls needed against vulnerabilities in programs and flaws during execution. It defines different types of programming flaws like intentional, inadvertent, validation errors, and boundary violations that can be exploited. Specific issues like buffer overflows, incomplete mediation, and time-of-check to time-of-use errors are explained in detail along with their security implications. Finally, it covers different types of malicious code and how viruses specifically spread and infect systems.

1. Program Security
VAMSEE KRISHNA KIRAN
ASST.PROF, CSE, AMRITA UNIVERSITY, COIMBATORE

2. Objectives
 To learn the concept of secure programming
 Programming errors with security implications: buffer overflows,
incomplete access control
 Malicious code: viruses, worms, Trojan horses
 Controls against malicious code and vulnerabilities
 Controls against program flaws in execution
2

3. Lets start with
 Why we need security at the program level?
 Because programs constitute most to a computing system and
Protecting programs is the heart of computer security.
 All kinds of programs, from apps via OS, DBMS, networks
 How can we achieve it?
Issues:
1. How do we keep programs free from flaws?
2. How do we protect computing resources against programs that
contain flaws?
3

4. Secure programs
 Security implies some degree of trust that the program enforces
expected confidentiality, integrity, and availability.
 What is “Program security?”
Depends on who you ask
 user - fit for his task
 programmer - passes all “his/her” tests
 manager - conformance to all specs
4

5.  Fault tolerance terminology:
 Bug – mistake in interpreting a requirement, syntax error
 Error – human made mistake , may lead to a fault
 Fault – misinterpreted requirements may lead to several
faults in the coding and testing phases
 Failure - system malfunction caused by fault, can be
discovered before or after system delivery
 Note:
 Faults - seen by “insiders” (e.g., programmers)
 Failures - seen by “outsiders” (e.g., independent testers,
users)
 Error/fault/failure example:
 Programmer’s indexing error, leads to buffer overflow fault
 Buffer overflow fault causes system crash (a failure)
5

6. Fixing faults
 Software that has many faults early on is likely to have many others
still waiting to be found.
 Earlier paradigm to judge s/w security: penetrate and patch
 Red Team /Tiger Team tries to crack s/w
 If software withstands the attack => security is good
 Is this true? - Rarely.
6

7. Too often developers try to quick-fix problems
discovered by Tiger Team
 Quick patches often introduce new faults due to:
 Pressure – causing narrow focus on fault, not context
 Non-obvious side effects
 Fixing one problem often caused a failure somewhere else
 system performance requirements not allowing
for security overhead
7Fixing faults

8. Unexpected Behavior
 Compare program requirements with behavior to identify program
security flaws
 Flaw is either a fault or failure
 Vulnerability is a class of flaws (e.g. buffer overflows)
 Program security flaws can derive from any kind of software fault.
 Therefore we categorize the faults into inadvertent human errors
and intentionally induced faults.
8

9. Unexpected Behavior
 We don’t have techniques to eliminate or address all program
security flaws.
 There are 2 reasons for this distressing situation:
 Program controls apply at the level of the individual program and
programmer. Programmer concentrates on “Should do” checklist and
least bother about “shouldn’t do” checklist.
 Programming and software engineering techniques evolve more rapidly
than computer security techniques.
9

10. Types of Flaws
 Intentional
 Malicious
 Non malicious
 Inadvertent
 Validation error (incomplete / inconsistent) : permission checks
 Domain error : controlled access to data
 Serialization and aliasing: program flow order
 Inadequate identification and authentication : basis for authorization
 Boundary condition violation : failure on first and last case
 Other exploitable logic errors
10

11. Non malicious program errors
 most of the mistakes made by the programmers are unintentional
and non malicious.
 Many such errors will not lead to more serious vulnerabilities but few
will put many security professionals in trouble.
 We look at three such classic error types and explain why they are
relevant to security and how can they be prevented.
11

12. Buffer overflows
 Its like pouring 2 liters of water into a 1 liter jug.
 Definition
 A buffer is a space in memory in which data is held.
 As memory in finite => buffer capacity is finite
 Therefore, in programming languages the programmer must declare
the buffers maximum size.
12

13. Buffer overflow example - C
char sample[10];
// compiler sets 10 bytes to store this buffer.
sample[10]=‘B’;
// out of bounds error, compiler detects this during compilation.
Now, what if we do
sample[i]=‘B’;
In some programming languages, buffer sizes need not be predefined.
 C does not perform array bounds checking.
 Similar problem caused by pointers
 No reasonable way to define limits for pointers
13

14. Buffer overflows
 Where does ‘B’ go?
 Depends on what is adjacent to ‘sample[10]’
 Affects user’s data - overwrites user’s data
 Affects users code - changes user’s instruction
 Affects OS data - overwrites OS data
 Affects OS code - changes OS instruction
14

15. Buffer overflows 15

16. Buffer overflows
 Implications of buffer overflow:
 Attacker can insert malicious data values/instruction codes into
“overflow space”
 Buffer overflow affects OS code area
 Attacker code executed as if it were OS code
 Attacker might need to experiment to see what happens when he inserts B
into OS code area
 Can raise attacker’s privileges (to OS privilege level)
 When B is an appropriate instruction
 Attacker can gain full control of OS
16

17. Buffer overflows
 Buffer overflow affects a call stack area
A scenario:
 Stack: [data][data][...]
 Pgm executes a subroutine
=> return address pushed onto stack
(so sub-routine knows where to return control to when finished)
Stack: [ret_addr][data][data][...]
 Subroutine allocates dynamic buffer char sample[10]
=> buffer (10 empty spaces) pushed onto stack
Stack: [..........][ret_addr][data][data][...]
 Subroutine executes: sample[i] = ‘A’ for i = 10
Stack: [..........][A][data][data][...]
Note: ret_address overwritten by B!
(Assume: size of ret_address is 1 char)
17

18. Buffer overflows
 Buffer overflow affects a call stack area
Stack: [..........][A][data][data][...]
 Subroutine finishes
 Buffer for char sample[10] is de-allocated
Stack: [A][data][data][...]
 RET operation pops B from stack (considers it ret. addr)
Stack: [data][data][...]
 Pgm (which called the subroutine) jumps to B
=> shifts program control to where attacker wanted
18

19. Buffer overflows
 C programming language specifications do not
specify how data is to be laid out in memory (incl. stack
layout)
 Some implementations of C may leave space
between arrays and variables on the stack, for
instance, to minimize possible aliasing effects.
(Source: Wikipedia)
19

20. Buffer overflows- security implication
 Even if the flaw came from a honest mistake, the flaw can still cause
great harm. A malicious attacker can exploit these flaws.
20

21. Buffer overflows- security implication
 Web server attack similar to buffer overflow attack:
pass very long string to web server
 Buffer overflows still common
 Used by attackers
 to crash systems
 to exploit systems by taking over control
 Large number of vulnerabilities due to buffer overflows
still persists in many software’s and systems
21

22. Web server attack example
 Parameter passing in the URL:
Consider:
http://www.somesite.com/subpage/userinput.asp?param1=(808)555-
1212&param2=2009Jan17
What can be the possible attack on this URL?
Passing a very long string is a slight variation on the classic buffer
overflow, but no less effective.
22

23. Incomplete mediation
 Consider the same previous example
http://www.somesite.com/subpage/userinput.asp?param1=(808)555-
1212&param2=2009Jan17
What happens if we pass values like 1800Jan01 or 1800Feb30 or
2048Min32 or 1Aardvark2Many?
1. Data type error
2. Continue to execute but ends up with a wrong result
What if we do all the validations properly on the client browser?
23

24. Security implication
 Unchecked data values represent a serious potential vulnerability.
 Example: A firm named “Things” started a e-commerce site to sell
their products.
 Once a person places his order the return URL is as follows:
http://www.things.com/order.asp?custID=101&part=555A&qy=20&pric
e=10&ship=boat&shipcost=5&total=205
If you’re a malicious attacker, what will you do?
Serious concern about this flaw was the length of time it could have
run undetected.
24

25. Time of check to Time of use errors
 Unintentional but with serious security consequences.
 Modern processors and OS usually change the order in which the
instructions and procedures are executed.
 Adjacent instructions may not even execute in the same order.
 Time-of-check to time-of-use (TOCTTOU) flaw is performed by “bait
and switch” strategy.
 Also called as synchronization or serialization flaw.
 Time-of-check to time-of-use flaw exploits the time lag between the
time we check and the time we use.
25

26. TOCTTOU
Example problem: DBMS/OS
pgm1 reads value of X = 10
pgm1 adds X = X+ 5
 pgm2 reads X = 10, adds 3 to X, writes X = 13
pgm1 writes X = 15
X ends up with value 15 , where X should be = 18
Prevention:
 Be aware of time lags
 Use digital signatures and certificates to “lock” data values
after checking them
 So nobody can modify them after check & before use
26

27. TOCTTOU prevention in DBMS
E.g., DBMS: locking to enforce proper serialization
(locks need not use signatures—fully controlled by DBMS)
In the previous example:
will force writing X = 15 by pgm 1, before pgm2
reads X (so pgm 2 adds 3 to 15)
OR:
will force writing X = 13 by pgm 2, before pgm1
reads X (so pgm 1 adds 5 to 13)
 An intelligent attacker uses each of the previously mentioned three
flaws(buffer overflow, incomplete mediation, TOCTTOU) as one step
in a multistep attack.
27

28. Viruses and other Malicious code
 Work done by a program is invisible to users and they will not be
aware of any malicious activity.
 Example:
1. When is the last time you saw a bit?
2. Do you know in what format a document file is stored?
3. If a document is stored on a disk, can you tell the exact location where
is it residing?
4. Which programs execute when we start our computer and how they
are executed?
 We cannot answer these question properly, since we don’t see
computer data directly.
28

29. Malicious code
 Malicious code executes just like any other program on the system.
But, it is written to exploit the vulnerabilities of a system/software.
 Malicious code can change: data and other programs.
 Malicious can do anything like writing a message to the screen,
stopping a running program, erasing a stored record etc. or
sometimes malicious code will not do anything at all and stay
dormant in the system.
 Dormant malicious code just needs a trigger to become active.
 Malicious codes are not new to computers, they have been in
existence for the past few decades.
29

30. Kinds of malicious code
 Malicious code or Rouge code is the general name for
unanticipated and undesired effects in programs.
 Agent is the writer of the program or the person who causes its
distribution.
 Virus is a program that can replicate itself and pass onto other non
malicious programs.
 Virus can be: transient or resident
 Transient virus has a life that depends on the life of its host.
 Resident virus located itself in the memory and will be active in the
system even after the attached program ends.
 Trojan horse is an unauthorized program that performs functions
unknown to the user.
30

31. Cont.
 Trojan horse gets installed along with an infected legitimate
program.
 Effects of a Trojan horse:
 Deleting, editing files.
 Transmitting files to intruders.
 Installing malicious code that can gain network access.
 Privilege elevation attacks etc.
 Logic bomb is a special class of malicious code that “detonates” or
goes off when a certain condition is met. Time bomb is a logic
bomb whose trigger is time or date.
 Trapdoor or backdoor is a feature in program, which provides an
alternate entry or access to the program avoiding the direct calls
and perhaps with special privileges.
31

32. Cont.
 Worm is a program that replicates itself and spreads across a
network of systems. Primary difference between a worm and a virus
is that, a worm operates through networks whereas a virus spread
through any medium.
 Rabbit is a virus or a worm that replicates itself without any bound to
exhaust the computing resources of a system.
 Often the term “Virus” is used to refer to any malicious code.
32

33. Summary of malicious code
Code type Characteristics
Virus Attaches itself to program and propagates copies of itself to
other programs.
Trojan horse Contains unexpected additional functionality
Logic bomb Triggers action when condition occurs
Time bomb Triggers action when specified time or date occurs
Trapdoor Allows unauthorized access to functionality
Worm Propagates copies of itself through network
Rabbit Replicates itself without limit to exhaust system resources
33

34. How viruses work?
 Program containing virus must be executed to spread virus or
infect other pgms
 Even one pgm execution suffices to spread virus widely
 Virus actions: spread / infect
 Spreading – Example 1: Virus in a pgm on installation CD
 User activates pgm contaning virus when she runs INSTALL
or SETUP
 Virus installs itself in any/all executing pgms present in
memory
 Virus installs itself in pgms on hard disk
 From now on virus spreads whenever any of the infected
pgms (from memory or hard disk) executes
34

35. Cont.
 Spreading – Example 2: Virus in attachment to e-mail
msg
 User activates pgm contaning virus (e.g. macro in MS
Word) by just opening the attachment
=> Disable automatic opening of attachments!!!
 Virus installs itself and spreads
 Spreading – Example 3: Virus in downloaded file
 File with pgm or document (.doc, .xls, .ppt, etc.)
 You know the rest by now...
 Document virus
 Spreads via picture, document, spreadsheet, slide
presentation, database, ...
 E.g., via .jpg, via MS Office documents .doc, .xls, .ppt etc.
35

36. Kinds of viruses- based on their way of
attaching
1. Appended Viruses
 Appends to program. Often virus code precedes the program code
execution by running its code before the 1st program instruction in exec file.
 Executes whenever program gets executed.
36
Original
program
Virus
Code
Virus
Code
Original
program

37. 2. Surrounding viruses
 Surrounds program
 Executes before and after infected program
 Intercepts its input/output
 Erases its tracks
 The “after” part might be used to mask virus existence.
37
Original
program
Virus
Code(part a)
Virus
Code(part b)

38. 3. Integrating and replacing viruses
 Integrates into pgm code
 Spread within infected pgms
 (Replacing) virus V gains control over target pgm T by:
 Overwriting T on hard disk
OR
 Changing pointer to T with pointer to V
 OS has File Directory
 File Directory has an entry that points to file with code for T
 Virus replaces pointer to T’s file with pointer to V’s file
In both cases actions of V replace actions of T when user executes what
she thinks is “T”
38
Original
program
Virus
Code
Modified
Code

39. Document virus- one form of integrated virus
 Virus implemented in a formatted document.
 Document consists of data and some commands like macros,
formatting controls, links etc.
 Commands are part of rich programming language.
 Attacker uses these command portions to integrate his virus code
with the document.
 Ordinary user just sees the plain document but not the virus code
embedded in commands portion.
39

40. Characteristics of a “Virus”
 Hard to detect
 Not easily destroyed or deactivated
 Spreads infection widely
 Can re-infect programs
 Easy to create
 Machine and OS independent
40

41. Homes for viruses
 Most viruses are passed through e-mails or drive-by-downloads.
 Attackers lure the victims to open the emails / click the malicious
links that enable drive-by-download.
 Ways for virus to take control over program:
 Overwriting the complete program
 Changing the pointer to point to a virus code instead of program on the
disk.
 One-time execution: majority of the viruses today execute only
once, spreading their effect in that once execution.
41

42. Boot sector viruses
 When OS is started, firmware detects the hardware components
present, tests them and then transfers the control to the OS.
 OS is invoked dynamically and not coded in the firmware.
 OS resides on the disk. It is fetched into memory by a program
called Bootstrap.
 Firmware reads fixed number of bytes from a fixed location (boot
sector) on the disk to a fixed location in the memory and jumps to
that address for execution.
 Often the boot sector size will be less than 512 bytes whereas the
bootstrap loader will be of larger size.
 To support this situation most of the hardware designers support
“chaining”.
42

43. Cont.
 This chaining has both pros and cons.
 Virus writer will simply break the chain at any point, inserts a pointer
to the virus code, and reconnects the chain later.
43

44. Memory resident viruses
 Most of the user programs will execute, terminate and disappear
making space for other programs.
 Few specialized programs are called very often and loading them
each time takes a long time. So, OS keeps such programs and
resident programs in the memory.
 Ex: resident code that interprets the keys pressed on keyboard.
 Resident routines are also called as “terminate and stay resident”
TSR.
 Viruses attach with this programs in memory so that virus gets control
whenever this program is invoked.
 This viruses are also capable of modifying Windows tables (registries).
44

45. Other homes for viruses
 Other home is application programs, like spreadsheets, word
processors having “macro” feature, by which user can record series
of commands and can repeat same by single invocation.
 Libraries are also excellent places for virus to reside. Often libraries
are called from legitimate code and also libraries are shared
between users.
 Compilers, loaders, linkers, runtime debuggers and even virus control
programs are good candidates for hosting viruses as they are mostly
shared.
45

46. Virus signatures
 Viruses executes in a particular way, using certain methods leaving
some patterns.
 These patterns of virus can be used to design programs like “virus
scanners”.
 Patterns can be:
1. Storage patterns
2. Execution patterns
3. Transmission patterns
 Symantec reports on viruses gives statistical information on viruses.
46

47. Storage pattern
 Often attached virus piece is invariant, so the start of the virus codes
becomes detectable.
 Virus attaches itself to a file, increasing the size of the file.
 Else, virus can obliterate the actual code, which will not increase the
size of the code but impacts the program functioning.
 Virus scanner can use a code or checksum to detect changes to a
file. It can also look for suspicious statements like JUMP at the starting
instruction of the code.
47

48. Execution pattern
 Most of the operations that a virus does are the common operations
like removing directory, modifying files etc. which are common in
OS.
 Damage is bounded only by the creativity of the virus’s writer.
48

49. 49

50. Transmission pattern
 Virus travel is not confined to any single medium or execution
pattern.
 A virus may come through a network, reside in disk, may get
attached to a program in execution, while executing may transfer a
copy of itself to memory staying there as a resident and etc.
 These transmissions have to be observed in order to detect virus
patterns in the system.
50

51. Polymorphic viruses
 Virus signatures or patterns are useful for a virus scanner to detect their
existence in the systems.
 Virus scanners look for such pre-defined patterns in the application
code.
 Intelligent virus writers can change these patterns just by sprinkling some
no-ops(jumps, adding 0 to a num, comparing with itself) to distort the
pattern.
 A virus that can change its pattern/appearance is called as a
polymorphic virus.
 Ex: if a virus writer has 100 bytes of code and 50 bytes of data; there
can be ‘n’ arrangements of this code using several jump statements.
51

52. Prevention of virus infection
 Do not receive executable code from an unknown source.
 But today, non executable file can have executable code, like
macro’s in docs.
 Hidden extension types are another problem, which deceives the
user with a fake format.
 Hiding and making the files as read-only will not prevent the attacks
of virus.
 Some prevention steps possible are:
52

53. 1. Use only commercial software acquired from reliable and well
established sources/vendors.
2. Use all new software on an isolated computer.
3. Open attachments only when you know them to be safe.
4. Make a recoverable system image and store it safely
5. Make and retain backup copies of executable system files.
6. Use virus detectors/scanners regularly and update them frequently
with latest virus definitions.
53Prevention of virus infection

54. Truths and misconceptions about viruses
1. Viruses can infect only Microsoft Windows systems.
2. Viruses can modify “hidden” and “read-only” files.
3. Viruses can appear only in data files, or only in word documents, or
only in programs.
4. Viruses spread only on disk or only through emails.
5. Viruses cannot remain in memory after a complete shutdown or on
reboot.
6. Viruses cannot infect hardware.
7. Viruses can be malevolent, benign or benevolent.
54

55. First example of malicious code:
Brain Virus
 It changes the label of any disk it attacked to the word “BRAIN”.
 What is does?
 First locates itself in the upper memory
 Executes system call to reset the upper memory bound below itself. (do
not disturb mode)
 Traps interrupt number 19 (disk read) by resetting the interrupt address
table to point to it and then sets the address for interrupt number 6 (un-
used) to the former address of the interrupt 19.
 Virus screens the disk read calls, that would read the boot sector.
 It will allow all the other disk calls through the interrupt 6.
55

56. Brain virus
 How it spreads?
 Brain virus settles in the boot sector along with other 6 sectors.
 One of the 6 sectors contain he actual boot code.
 While 2 others contain the parts of the virus code.
 Rest 3 sectors contain the duplicate of the others.
 Virus marks these 6 sectors as “faulty”, so that OS will not use them.
 Sitting in the memory, this virus will intercept all the disk reads to boot
sector, and verifies the 5th and 6th bytes for its signature.
 If signature found: already infected, if not found: infect them.
56

57. What did we learn from Brain virus?
 Uses standard tricks like hiding the virus in the boot sector,
intercepting and screening the interrupts.
 This virus just infects every device that tries performing a disk read. It
doesn’t have any other effect than passing its infection.
 This has served as a prototype for the viruses later.
 Many extensions to this has come ex: Lehigh virus that swept across
all the systems in Lehigh University.
57

58. Internet Worm
 Morris, a jr college student from Cornell university programmed the
internet worm to accomplish three objectives:
1. Determine where it could spread to
2. Spread its infection
3. Remain undiscovered and undiscoverable
 What effect it had?
 Primary goal is resource exhaustion, it checks whether a system is
already infected or not, if so it negotiates with the existing infection or
the new infector will terminate.
 Unfortunately many copies did not terminate causing great loss to
servers and system in universities.
 Many system and machines were disconnected to stop the transfer to
other systems, coz of which work and research is halted for a long time.
58

59.  How it worked?
 Exploited several known flaws and config failures in UNIX Berkeley
version 4.
 It accomplished three objectives as mentioned previously.
 Determine where to spread: used three techniques for locating
potential victims
1. Guessing passwords attack on machines.
2. Buffer overflow exploit in the program “finger”, that runs continuously to
respond to other computers.
3. Trapdoor in the “sendmail” program.
59

60.  Spread infection
 Once a target machine is acquired, worm would send a bootstrap
loader to the target.
 This loader is a 99 lines C code that is to be compiled and executed on
the target machine.
 This will fetch the rest of the worm code from the sending machine.
 Worm supplies a OTP to the host, so as to differentiate a rogue program
written by administrator to obtain a copy of worm for analysis.
 Remain undiscovered and undiscoverable
 If a transmission error occurs during the worm transfer, the loader zeroed
and deletes all the code on the host.
 As soon as the worm received its full code, it will bring the code to
memory, encrypt it and delete the copies in disk.
 Also, this worm will change its and name and process id frequently to
main undiscoverable.
60

61. Code Red
 Devastating effect, propagates itself onto web servers running
Microsoft IIS web server.
 6 million web servers are infected across the globe.
 Takes two steps: infection and propagation
 Exploited buffer overflow vulnerability in IIS- DLL idq.dll to reside in
servers memory.
 To propagate, code red uses IP addresses on port 80 of the PC to
see if that web server is vulnerable.
 After propagation, code red started DOS attack on all the servers
flooding messages.
61

62. Web bugs
 Not malicious, but do track the personal information.
 Web bugs are invisible. It is an image hidden in any type of
document that can display HTML tags.
 If you visit www.bluenile.com web bug code is automatically
downloaded as a one-by-one pixel image from Avenue A
marketing agency.
 Web bugs are mainly used to track the user activities on a page, his
interests, buying habits etc. so that advertising agencies can use this
data to give user suggestions.
 This web bug places a cookie on the system to capture the data.
62