Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

OS course – lecture 1


Published on

  • Be the first to comment

OS course – lecture 1

  1. 1. OS course – lecture 1 Amir Averbuch and Nezer J. Zaidenberg
  2. 2. Announcements <ul><li>Lecturer : Prof. Amir Averbuch </li></ul><ul><li>Tutoring assistant :Mr. Nezer Zaidenberg </li></ul><ul><li>Ex. Grader : Mr. Tomer Margalit </li></ul><ul><li>The course form has changed since previous years. It is recommended that you will attend lectures even if you taken this course previously </li></ul>
  3. 3. Scribe <ul><li>Following success last term… </li></ul><ul><li>1-2 Volunteers are needed as scribe </li></ul><ul><li>Scribe should attend every lecture and take notes. </li></ul><ul><li>Once a month the scribe will meet Nezer to check and review the class notes. </li></ul><ul><li>Notes will be published to the class in order to prepare for the test and homework </li></ul><ul><li>Scribe duty well done will be considered as one homework exercise submitted at 100 grade. (so scribe do gets something for effort) </li></ul><ul><li>A second benefit is off course reviewing the material! </li></ul><ul><li>People volunteering for scribe duty should contact Nezer via email </li></ul>
  4. 4. Grades <ul><li>55% exercises </li></ul><ul><li>55% exam (3 best are 15% worst is 10%) </li></ul><ul><li>The exam will include multiple choice questions. </li></ul><ul><li>You must pass both the exam and the homework to pass this course! </li></ul>
  5. 5. References <ul><li>The main books for this course is </li></ul><ul><ul><li>Operating Systems Concepts (6-8 Edition), Silberschatz, Galvin, Gagne </li></ul></ul><ul><ul><li>Understand the Linux kernel 3 rd edition (For kernel space) </li></ul></ul><ul><ul><li>Advanced programming for the UNIX environment 3 rd edition by Richard W. Stevens (For user space) </li></ul></ul><ul><li>Additional books (that will be noted on specific lectures) </li></ul><ul><ul><li>UNIX filesystems by S. Pate </li></ul></ul><ul><ul><li>Beej Manual for network programming (web guide) </li></ul></ul><ul><ul><li>Solaris internals by Jim Mauro </li></ul></ul>
  6. 6. Course outline <ul><li>The goal of this course is to gain theoretical and practical understanding of OS </li></ul><ul><li>The course is very practical with several homework assignments and exam based on class and homework </li></ul><ul><li>You will have to defend your work </li></ul><ul><li>The course will focus on UNIX OS (in particular Linux), Windows and general principles of OS. </li></ul><ul><li>The exercises will be mainly coding work or reviewing kernel and low-level user code. </li></ul>
  7. 7. What this course is not about <ul><li>Coding (to the extent of coding standards, best coding practices or efficiency) </li></ul><ul><li>MPP (beyond simple sync algorithms) </li></ul><ul><li>Network algorithms </li></ul><ul><li>Encryption </li></ul><ul><li>Etc. </li></ul><ul><li>This code is on OS and OS related programming only </li></ul>
  8. 8. What is OS <ul><li>Once upon a time there was no operating systems </li></ul><ul><li>Developers used to develop hardware and software, together with everything needed to make the application run </li></ul><ul><li>As computers become more standard and tasks become common several tasks stood out as needed by all (or practically all) software. </li></ul><ul><li>Those common tasks became “OS” and were supplied by 3 rd party vendors </li></ul><ul><li>Commercial software used to require specific OS. This became one of the first practices of “code reusability” </li></ul>
  9. 9. Common OS tasks <ul><li>Memory management </li></ul><ul><li>Hardware (driver) support </li></ul><ul><li>Task scheduling </li></ul><ul><li>File management </li></ul><ul><li>User management (on multi-user systems) </li></ul><ul><li>Software providers developed a single software for all those components (and more) – the OS </li></ul><ul><li>This software became the base for all further software development </li></ul>
  10. 10. What the OS includes <ul><li>Core OS function </li></ul><ul><ul><li>Process management </li></ul></ul><ul><ul><li>User management </li></ul></ul><ul><ul><li>File management </li></ul></ul><ul><ul><li>Memory management </li></ul></ul><ul><ul><li>Hardware management </li></ul></ul><ul><ul><li>Network management </li></ul></ul><ul><li>Additional functions are added and removed according to political and marketing scenarios </li></ul><ul><ul><li>Initial UNIX implementation had built in C compiler (sold separately with almost all modern OS) </li></ul></ul><ul><ul><li>Most UNIX OS comes with wide range of servers such as ftp server, telnet server, daytime server, printing server and even HTTP server, J2EE platform, etc. </li></ul></ul><ul><ul><li>Many OS also come with GUI (windows, OSX aqua, X Windows) and other user tools (calculator, Text processor etc.) </li></ul></ul><ul><ul><li>Microsoft wished to add Internet Explorer, Windows media player etc. as part of the OS. However, the European union prevented the inclusion of media player in all windows sold in Europe… (anti trust reasons) </li></ul></ul><ul><li>The course will deal with “CORE OS” functions only. These services are usually managed by the OS “Kernel” </li></ul><ul><li>We will focus on UNIX environment, Linux, Xubunto distribution </li></ul>
  11. 11. What we will cover <ul><li>UNIX software development </li></ul><ul><li>Process </li></ul><ul><ul><li>What is a process </li></ul></ul><ul><ul><li>Memory management </li></ul></ul><ul><ul><li>Opening files </li></ul></ul><ul><li>Multi tasking </li></ul><ul><ul><li>Multi process </li></ul></ul><ul><ul><li>Multi threading </li></ul></ul><ul><ul><li>Sync </li></ul></ul><ul><li>Memory management </li></ul><ul><li>Network </li></ul><ul><li>Kernel development </li></ul><ul><ul><li>IO devices ( char device and block device ) </li></ul></ul><ul><ul><li>File systems </li></ul></ul>
  12. 12. What is UNIX <ul><li>UNIX is a general term for lots of things </li></ul><ul><li>UNIX is the name of historical OS developed in 1970’s in Bell labs </li></ul><ul><li>It’s also a trademark </li></ul><ul><li>UNIX may refer to set of standards (POSIX) that defines OS interface that many OS follow </li></ul><ul><li>UNIX is also general name for a family of many proprietary OS that were tested for standard completion. None of these OS is called UNIX but usually their name has an X that gives them away (AIX, OSX, HP-UX, IRIX) with Solaris as the exception </li></ul><ul><li>Commercial UNIX OS pay royalties to use the UNIX trademark </li></ul><ul><li>Since UNICES (plural for UNIX) share the same API and design principle it is relatively easy (but not effortless) to port software from one UNIX to the other. </li></ul>
  13. 13. UNIX-Like OS’s <ul><li>Open source OS (such as Linux, FreeBSD, OpenBSD) that were not tested by standard committee (but tend to have the same API) are often referred to as UNIX-like. </li></ul><ul><li>Open Source OS’s also don’t pay royalties and cannot be officially named UNIX (due to trademark rules) </li></ul><ul><li>Open source UNIX-Like OS have become more and more popular (especially Linux) in many fields more common then real “UNIX” boxes. </li></ul><ul><li>Today, for most people, UNIX means an OS with specific set of API and applications regardless of standards testing. (so many people will say Linux is as much of a UNIX as AIX, Solaris and OSX are.) </li></ul><ul><li>So it is common jargon for both UNIX and UNIX-like OS’s to be considered UNIX. (at least for this course) </li></ul>
  14. 14. UNIX OS internals (IMPORTANT) <ul><li>UNIX OS and the UNIX API is built on top of two basic principals </li></ul><ul><ul><li>Process </li></ul></ul><ul><ul><li>File </li></ul></ul><ul><li>Almost all OS services are implemented as either process or file. (for example the system logger is a “process” while a socket is a file.) </li></ul><ul><li>This is one of the first implementations of the interface design pattern </li></ul><ul><li>Throughout the course we will see how complex services are handled using these two basic concepts. </li></ul>
  15. 15. UNIX history <ul><li>UNIX (acr. for uniplexed information) was initially pun on multics (OS that is virtually extinct that allowed for multi-user to run multiple tasks on multi process environment) and was the name of OS that was developed at bell labs for their PDP11 computers. </li></ul><ul><li>Original UNIX developers wanted to build a better multics. </li></ul><ul><li>The OS was initially distributed in source form so that anybody can modify and customize it. </li></ul><ul><li>The source was later brunched into two main trunks SYSV and BSD code (with both brunches often borrowing code from each other) </li></ul><ul><li>Most modern Unices are usually based on either of those brunches. </li></ul><ul><li>Commercial UNIX today mainly in high end server, workstation and desktop market. </li></ul><ul><li>Open source UNIX tends to rule several market segments (Linux and apache rule the http serving and Linux practically runs on all Wireless routers.) and is heavy favorite on several others (running database servers) </li></ul>
  16. 16. UNIX OS in the wild (partial list) <ul><li>SVR4 Unices </li></ul><ul><li>Solaris and Open Solaris – Unix by SUN Microsystems </li></ul><ul><li>HP-UX – Unix by HP </li></ul><ul><li>Linux – The most popular open source OS (sponsored by Google, RedHat, IBM, Silicon graphics, Novell and more) </li></ul><ul><li>AIX – Unix by IBM </li></ul><ul><li>BSD Unices </li></ul><ul><li>OS X and Darwin iPhoneOS – Apple OS (OS X is the proprietary version and darwin is the open source version) </li></ul><ul><li>FreeBSD, NetBSD, OpenBSD – Open source BSD distributions </li></ul><ul><li>NOT UNIX systems (and how much they support POSIX) </li></ul><ul><li>Windows – not UNIX. But cygwin (open source) and microsoft POSIX subsystems each provide compatibility layer </li></ul><ul><li>Z/OS or MVS – IBM OS for mainframe – doesn’t follow UNIX at all but have a compatibility layer </li></ul>
  17. 17. UNIX API summary <ul><li>All UNIX and UNIX like OS’s follow the same API </li></ul><ul><li>All OSs that follow the UNIX API share similar design principles and concepts </li></ul><ul><li>As long as we program using the UNIX API it is relatively easy to port from one OS to the other (and for a developer or administrator from one environment to move to another) </li></ul><ul><li>Please note – some UNIX OS’s also have proprietary “non POSIX” APIs to support more features (for example OSX COCOA) naturally such features cannot be ported easily. </li></ul>
  18. 18. UNIX acceptance <ul><li>Super computers – 88.6% of top 500 world super computers run Linux. 99% run UNIX </li></ul><ul><li>Midrange servers (UNIX traditional role) – AIX is the most popular midrange servers in the financial industry while Solaris and HP-UX contend for the crown in Telco enterprises </li></ul><ul><li>Low end servers - About 70-75% of web servers (and most J2EE platforms) run on Linux. Many network products (firewalls, switches, routers) are based on Linux </li></ul><ul><li>High end Desktops and workstations – Over 65% of workstations and desktops costing above 1000 USD run OSX </li></ul><ul><li>Smart phones – both iPhoneOS (OSX) and Google Android (Linux) are unices </li></ul><ul><li>Databases – Sun was recently acquired by Oracle and Solaris will be the OS used by Oracle’s Exadata. UNIX is also Tier-1 for most database vendors </li></ul><ul><li>Embedded systems – VxWorks (arguably the most popular embedded OS) is POSIX complaint. Linux dominates STB and IGD markets </li></ul><ul><li>Storage – most storage products run on top of UNIX (such as Linux or AIX) </li></ul><ul><li>SAP environments – are typically managed by UNIX hosts </li></ul>
  19. 19. Almost regardless of the industry or company you will end up working for… UNIX skills thought in this course will be a vital tool in your toolbox
  20. 20. GNU source and the FSF <ul><li>GNU – acr. For “GNU’s not UNIX” </li></ul><ul><li>GNU – a public license granting permission to use, modify and redistribute code provided it remains GNU </li></ul><ul><li>FSF – acr. For “free software foundation” </li></ul><ul><li>FSF – distribute the source for most of the free UNIX applications (The unix look and feel). Promote the GNU license </li></ul><ul><li>Linux kernel – the “main functionality” of the OS developed by volunteers (not the FSF, but the groups are connected) under GNU license </li></ul>
  21. 21. The many types of free <ul><li>MIT license – allow you to do what ever you want with the code </li></ul><ul><li>BSD license – allow modification and redistribution (modification may not be free). You must keep credits to original author </li></ul><ul><li>GNU GPL – modifications (derived work) must be free </li></ul><ul><li>GNU LGPL – modification must be free, but linking with GNU source is not a modification (license used by GNU libraries) </li></ul><ul><li>Other free licenses - check on FSF website </li></ul><ul><li>Commercial Open source – you may view the source but not use it commercially or you may edit the source, but your modification belong to specific company or modifications may not be redistributed. Are not considered “free” (such as Apple public license, Sun Public license, Netscape public license, MySQL) </li></ul><ul><li>Dual license – software available as GPL (if you modify you must open the source of the modification) same software is also available with closed source (you can modify and not open the source) for paying customers. This is the business model of many open source companies </li></ul>
  22. 22. Introducing Linux….
  23. 23. History of Linux <ul><li>In the early 1990’s there were commercial UNIX (SYS V) and free UNIX (BSD) </li></ul><ul><li>A T&T (copyright holders of SYS V) sued BSDi (BSD) for copyright infringement. BSD was crippled and many stopped using it </li></ul><ul><li>During the legal vaccum a young Finnish student – Linus Turvalds started implementing UNIX for intel 80386 from scratch. The OS he developed was called Linux </li></ul><ul><li>Linus developed Linux “Just for fun” </li></ul><ul><li>With BSD in legal struggle and Linux proving itself as efficient and scalable alternative Linux started growing in popularity </li></ul><ul><li>Linux was adopted by FSF for it’s GNU/Linux platform </li></ul>
  24. 24. Linux today <ul><li>Linux today is the most widely used UNIX OS (arguably the most widely used OS) found in cell phones, netbooks, embedded systems, notebooks, desktops, workstations, servers and supercomputers </li></ul><ul><li>Linux runs on most architectures today (x86, Itanium, ARM, Power, MIPS, s390 and a lot more) </li></ul><ul><li>Is distributed by commercial companies in what is known as a Linux distribution. </li></ul>
  25. 25. What is Linux distribution <ul><li>Linux distribution is a gathering of software from several sources, compiling them and branding them by single company </li></ul><ul><li>All Linux distribution include the Linux kernel and most GNU sources (thereby Linux distribution….) </li></ul><ul><li>Linux distributions by Red hat SuSe, Debian, Ubunto, slackware have been popularized. </li></ul><ul><li>In this course we will focus on Xubunto 9.04 </li></ul>
  26. 26. Differences between distributions <ul><li>Why and what is Xubunto </li></ul><ul><ul><li>The popular Ubunto with X (i.e. not KDE or Gnome) desktop </li></ul></ul><ul><ul><li>The lowest foot print and hardware requirements of all modern Ubunto flavours </li></ul></ul>
  27. 27. The “founding fathers” of Linux (partial list) <ul><li>Linus Torvalds – Finnish – creator and maintainer. About 2% of the current Linux code (which is A LOT) is Linus’s </li></ul><ul><li>Alan Cox – Walsh </li></ul><ul><li>Greg Kroah-Hartman - American </li></ul><ul><li>Ingo Molnar – Hungarian </li></ul><ul><li>Robert Love – American </li></ul>
  28. 28. Shaped UNIX (partial list) <ul><li>Bill Joy </li></ul><ul><li>Eric S. Raymond </li></ul><ul><li>Richard M. Stallman </li></ul><ul><li>Ken Thompson </li></ul><ul><li>Dennis Ritchie </li></ul><ul><li>Marshall Kirk McKusick </li></ul>
  29. 29. But maybe we are moving too fast here…
  30. 30. Why do we need OS in the first Place <ul><li>Well we don’t NEED OS. But we want to. </li></ul><ul><li>Almost every piece of software require some set of services such as managing files, allocating memory and usually opening threads, syncing etc. </li></ul><ul><li>Initially people implemented it from scratch with any new software. </li></ul><ul><li>OS is the basic set of services which allows us to write our software. </li></ul><ul><li>The strategy of rewriting the OS with a new software is almost extinct since long ago. </li></ul>
  31. 31. OS code and User code <ul><li>OS code – usually invoked in “the kernel” is the code that “makes the computer work”. Including drivers, memory management etc. </li></ul><ul><li>User code – most code we write </li></ul><ul><li>In this code we will use the terms Userland and kernelspace to distinguish between them. </li></ul><ul><li>We will write both kinds of code in the exercises </li></ul>
  32. 32. Where does the OS “ends” <ul><li>The OS API is often called “system calls” </li></ul><ul><li>Often when we write code in C we call library functions (such as printf, fopen, malloc) that provide us with “OS” services (such as producing output, opening a file, allocating memory) </li></ul><ul><li>Those are not system calls but C wrappers to system calls. (the C wrappers remain the same even on non-POSIX systems such as windows) </li></ul>
  33. 33. Help request <ul><li>In UNIX when we want brief help on something we can type man (something) and get the help page. (try it. For example man man) </li></ul><ul><li>When the man refers to command line executable we will find the manual in section 1 of the man </li></ul><ul><li>When the man refers to a system call we will find the manual in section 2 of the man </li></ul><ul><li>When the man refers to a library function we will find the manual in manual 3 of the man </li></ul><ul><li>When a concept is found in two or more sections we can request the right manual by typing “man 1 write” or “man 2 write” (try it!) </li></ul>
  34. 34. Common convention <ul><li>When discussing executable (such as ls) we will use (1) after the executable name (for example ls(1)) </li></ul><ul><li>Similarly when discussing system calls and library functions we will use 2 and 3 respectfully. (open(2), printf(3)) </li></ul><ul><li>There may be some inconsistencies on sections between UNIX flavours (functions moving between section 2 and 3) but it’s rare. </li></ul>
  35. 35. Library code relation with OS code <ul><li>Library functions invoke OS code to complete their work </li></ul><ul><li>For example printf(3) (short for print formatted) parses the input and format the text. Then it calls for write(2). </li></ul><ul><li>Printf(3) is library function (and not OS function) while write(2) is a system call. </li></ul>
  36. 36. Some unix file related system calls you should already know <ul><li>Open(2) </li></ul><ul><li>Close(2) </li></ul><ul><li>Read(2) </li></ul><ul><li>Write(2) </li></ul><ul><li>To refresh your memory about these functions type “man 2 open” to get the function C info. </li></ul>
  37. 37. Example – man open <ul><li>OPEN(2) BSD System Calls Manual OPEN(2) </li></ul><ul><li>NAME </li></ul><ul><li>open -- open or create a file for reading or writing </li></ul><ul><li>SYNOPSIS </li></ul><ul><li>#include <fcntl.h> </li></ul><ul><li>int open(const char *path, int oflag, ...); </li></ul><ul><li>DESCRIPTION </li></ul><ul><li>The file name specified by path is opened for reading and/or writing … </li></ul>Note section 2 for system call This is a man page from OSX which is BSD system Open is the system call name What the system call does Required header to use this function C style function declaration (prototype) Brief description
  38. 38. More from man open(2) <ul><li>… (description snipped)… </li></ul><ul><li>RETURN VALUES </li></ul><ul><li>If successful… (snipped) </li></ul><ul><li>ERRORS </li></ul><ul><li>The named file is opened… (snipped) </li></ul><ul><li>SEE ALSO </li></ul><ul><li>chmod(2), close(2), dup(2), getdtablesize(2), lseek(2), read(2), umask(2), write(2) </li></ul><ul><li>HISTORY </li></ul><ul><li>An open() function call appeared in Version 6 AT&T UNIX </li></ul>What the function returns When something interesting/bad happens Related man pages Standards etc.
  39. 39. The difference between system and library function Library function OS function Usually calls OS function Lowest level interface Section 3 of man Section 2 of man Programming language dependent (but not system dependent) System dependent and may not exist on different systems (or used with different name or parameters) Almost always bugfree Almost always bug free Almost always efficient Almost always efficient Usually - user space code Usually – kernel space code
  40. 40. fopen(3) implementation <ul><li>  FILE *fopen(char *name, char *mode) </li></ul><ul><li>{        int fd;        FILE *fp; </li></ul><ul><li>  if (*mode != ‘r’ && *mode != ‘w’ && *mode != ‘a’)            return NULL;        for (fp = _iob; fp < _iob + OPEN_MAX; fp++)            if ((fp->flag & (_READ | _WRITE)) == 0)                break;        /* found free slot */        if (fp >= _iob + OPEN_MAX)   /* no free slots */            return NULL; </li></ul><ul><li>       if (*mode == ‘w’)            fd = creat(name, PERMS);        else if (*mode == ‘a’) {            if ((fd = open(name, O_WRONLY, 0)) == -1)                fd = creat(name, PERMS);            lseek(fd, 0L, 2);        } else            fd = open(name, O_RDONLY, 0);        if (fd == -1)         /* couldn’t access name */            return NULL;        fp->fd = fd;        fp->cnt = 0;        fp->base = NULL;        fp->flag = (*mode == ‘r’) ? _READ : _WRITE;        return fp; </li></ul><ul><li>} </li></ul>System calls!
  41. 41. Conclusion <ul><li>So OS code is the underlying layer of the library function </li></ul><ul><li>Homework – (not for submission) </li></ul><ul><ul><li>Examine printf(3) code – find the OS function call references. </li></ul></ul><ul><ul><li>Follow all functions printf calls! </li></ul></ul>