Lecture1 Introduction

 Open Source Community. Why should we care about OS Programming? What is ArabBSD? OS Categories. FreeBSD Vs. Linux. FreeBSD Kernel Design Overview. FreeBSD File Hierarchy. FreeBSD Booting Process. FreeBSD Kernel Basics.

KernelMonolothic Microkernel Exokernel Hybrid

 Composite Component-Based Operating System. Web-Based OS (Chrome OS).

 Kernel has no scheduler. Scheduling policies and synchronization protocols are defined by user-level schedulers. Mutable Protection Domains (MPD) instead of IPC.

 Kernel development. Different Systems Emulation Support (Binary Emulation). Levels of Security . Maintainers. License.

BSD FreeBSD OpenBSD NetBSDPC-BSD DragonFlyBSD

 FreeBSD Installation. X Window System Installation. UNIX Basics.

UserApplications Kernel Hardware

 Paged Virtual Address Space. Software Interrupts. Timers and Counters. Identifiers for accounting, protection, scheduling, ..etc. DescriptorsHigher layers provide “system” facilities such as filesystem, terminals and sockets.

 CPU Time. Asynchronous Events. Memory. IO Descriptors.Processes are the fundamental service provided by UNIX.

User process Blocking Wall Top half of KernelBottom Half of Kernel

Multiple User Processes Blocking Wall Top half of KernelBottom Half of Kernel

bin sbin boot cdrom dev etc lib mount media / proc (Root sys kernDirectory) tmp home src ports usr games bin bin Local sbin share crash run var tmp spool

Directory Description bin Essential command binaries boot Static files of the boot loader dev Device files etc Host-specific system configuration lib Essential shared libraries and kernel modules media Mount point for removable media mnt Mount point for mounting a filesystem temporarily sbin Essential system binaries tmp Temporary files usr Secondary hierarchy var Variable data

 home User home directories lib Alternate format essential shared libraries /sys/kern Kernel Directory.

 /bin contains tools required system to work (like ls, rm, mkdir etc.) which would be available at any time (for example if system crashes, they were available in single user mode). Used by both root and users. /usr/bin contains tools which are not needed in single user mode. Used by both root and users. /usr/local/bin is for local use - you install your binaries there. Used by both root and users (Third party binaries).

 /sbin : System binariesPurpose Utilities used for system administration are stored in /sbin, /usr/sbin, and /usr/local/sbin. /sbin contains binaries essential for booting, restoring, recovering, and/or repairing the system in addition to the binaries in /bin. Programs executed after /usr is known to be mounted (when there are no problems) are generally placed into /usr/sbin. Locally-installed system administration programs should be placed into /usr/local/sbin.

 /boot : Static files of the boot loaderPurpose This directory contains everything required for the boot process except configuration files not needed at boot time. Thus /boot stores data that is used before the kernel begins executing user-mode programs. This may include saved master boot sectors and sector map files. Also, the compiled kernel should be located in /boot/kernel/.

/dev : Device filesPurpose The /dev directory is the location of special or device files.

/etc : Host-specific system configurationPurpose The /etc hierarchy contains configuration files. A "configuration file" is a local file used to control the operation of a program; it must be static and cannot be an executable binary.

 File Description fstab Static information about filesystems ftpusers FTP daemon user access control list gateways File which lists gateways for routed host.conf Resolver configuration file inetd.conf Configuration file for inetd

/etc/X11 : Configuration for the X Window SystemPurpose /etc/X11 is the location for all X11 host- specific configuration.

/home : User home directories (optional)Purpose /home is a fairly standard concept, but it is clearly a site-specific filesystem.

/media : Mount point for removable mediaPurpose This directory contains subdirectories which are used as mount points for removable media such as floppy disks, cdroms and zip disks.

 /mnt : Mount point for a temporarily mounted filesystem.Purpose This directory is provided for temporarily mounted filesystem. For example, partitions from harddrives or network shares.

 /tmp : Temporary filesPurpose The /tmp directory must be made available for programs that require temporary files.

 The /usr HierarchyPurpose This directory includes all the user related stuff.

 /usr/include : Directory for standard include files.Purpose This is where all of the systems general-use include files for the C programming language should be placed.

 /usr/lib : Libraries for programming and packagesPurpose /usr/lib includes object files, libraries, and internal binaries that are not intended to be executed directly by users or shell scripts.

 /usr/local : Local hierarchyPurpose The /usr/local hierarchy is for use by the system administrator when installing software locally. It needs to be safe from being overwritten when the system software is updated. It may be used for programs and data that are shareable amongst a group of hosts, but not found in /usr.

 /usr/local/sharePurpose The requirements for the contents of this directory are the same as /usr/share. The only additional constraint is that /usr/local/share/man and /usr/local/m an directories must be synonomous (usually this means that one of them must be a symbolic link).

 /usr/sbin : Non-essential standard system binariesPurpose This directory contains any non-essential binaries used exclusively by the system administrator.

 /usr/share : Architecture-independent dataPurpose This hierarchy is intended to be shareable among all architecture platforms of a given OS. Any program or package which contains or requires data that doesnt need to be modified should store that data in /usr/share (or /usr/local/share, if installed locally).

 /usr/share/man : Manual pagesPurpose This section details the organization for manual pages throughout the system, including /usr/share/man.

 The /var HierarchyPurpose /var contains variable data files. This includes spool directories and files, administrative and logging data, and transient and temporary files. Some portions of /var are not shareable between different systems. For instance, /var/log and /var/lock. Other portions may be shared, notably /var/mail, /var/cache/man, /var/cache/fo nts, and /var/spool/news.

 /var/cache : Application cache dataPurpose /var/cache is intended for cached data from applications. Such data is locally generated as a result of time-consuming I/O or calculation.

 /var/crash : System crash dumps (optional)Purpose This directory holds system crash dumps.

 /var/mail : User mailbox files (optional)Purpose The mail spool must be accessible through /var/mail. User mailbox files in this location must be stored in the standard UNIX mailbox format.

 /var/run : Runtime variable dataPurpose This directory contains system information data describing the system since it was booted. Files under this directory must be cleared (removed or truncated as appropriate) at the beginning of the boot process.

 /proc : Kernel and process information virtual filesystemPurpose The proc filesystem is the standard method for handling process and system information.

 Take a tour in FreeBSD File Hierarchy.

 When a computer is powered on, there is nothing running on the CPU. For a program to be set running, the binary image of the program must first be loaded into memory from a storage device. Bootstrapping: cold start to the point at which user-mode programs can be run. In FreeBSD, the binary image is in /vmunix or /boot/kernel/kernel

 Initial bootstrapping is machine dependant. Usually 512 bytes. Used to load boot program. The boot programs task is to load and initialize the executable image of a program and to start that program running. Not required to be on the same storage device.

[BITS 16]ORG 0int 0x18TIMES 510-($-$$) DB 0DW 0xAA55

 Binary file reading in terms of Assembly language. FreeBSD Assembly Tools (as, nasm).

 The boot program reads the binary image of a program to be bootstrapped into main memory, and then initializes the CPU so that the loaded program can be started. Boot -> several stages of hardware and software initialization in preparation for normal system operation

 The first stage is responsible for initializing the state of the CPU, including the run-time stack and virtual-memory mapping. Once virtual-memory mapping is enabled, the system does machine-dependent initializations, and then machine- independent initializations.

 The machine-dependent operations include setting up virtual-memory page tables and configuring I/O devices; the machine- independent actions include mounting the root filesystem and initializing the myriad system data structures. This order is necessary because many of the machine- independent initializations depend on the I/O devices being initialized properly. Following the setup of the machine- independent portions

Turn on PC boot0Read the value 0xfffffff0 from instruction pointer register boot1Hardware translates the value to read BIOS Memory Block boot2Jump to BIOS’s POST routines loaderPOST performs different checks initOne of the POST instructions is INT 0x19 instruction whichreads 512 bytes from the first sector of boot device intothe memory at address 0x7c00

File •sys/boot/i386/boot0/boot0.SLocation •INT 0x19 instruction loads an MBR, i.e. the boot0 content, into theMemory memory at address 0x7c00Location •Starting from 0x1be, called the partition table, It has 4 records of 16 bytes each, MBR called partition recordStructure •the 1-byte filesystem type •the 1-byte bootable flagPartition Record •the 6 byte descriptor in CHS formatStructure •the 8 byte descriptor in LBA format

LBA (Logical Block Addressing)• has the starting sector for the partition and the partitions length.CHS (Cylinder Head Sector)• has coordinates for the first and last sectors of the partition.

The boot manager scans the partition table and prints the menu on thescreen so the user can select what disk and what slice to boot. Bypressing an appropriate key, boot0 performs the following actions: modifies the bootable flag for the selected partition to make it bootable, and clears the previous saves itself to disk to remember what partition (slice) has been selected so to use it as the default on the next boot loads the first sector of the selected partition (slice) into memory and jumps there

 Actually, there is a 512-byte file called boot1 in the directory /boot as well. It is used for booting from a floppy. When booting from a floppy, boot1 plays the same role as boot0 for a harddisk which is locating boot2 and runs it.

 You may have realized that a file /boot/mbr exists as well. It is a simplified version of boot0. The code in mbr does not provide a menu for the user, it just blindly boots the partition marked active (i.e. No Boot Manager Used).

 Configure boot file using boot0cfg. Note: Both boot0 and boot2 are invisible to listing commands because they are not under filesystem’s control.

 It’s the first specific file for FreeBSD since it’s used for locating FreeBSD slice.

File • sys/boot/i386/boot2/boot2.c Location • It must scan the harddisk. • knowing about the filesystem structure • finds the file /boot/loader • reads it into memory using a BIOS serviceProcedure • passes the execution to the loaders entry point. • boot2 prompts for user input so the loader can be booted from different disk, unit, slice and partition • passes the execution to BTX

• switches the processor to protected mode • prepares a simple environment before calling the client • boot0 passes the execution to BTXs entry point. BTX then switches the processor to protected mode, and prepares a simple environment before calling the client. This includes: • virtual v86 mode. That means, the BTX is a v86 monitor. Real mode BTX instructions like pushf, popf, cli, sti, if called by the client, will work.Procedure • Interrupt Descriptor Table (IDT) is set up so all hardware interrupts are routed to the default BIOSs handlers, and interrupt 0x30 is set up to be the syscall gate. • Two system calls: exec and exit. • Finally, BTX creates a Global Descriptor Table (GDT) • boot2 defines an important structure, struct bootinfo. This structure is initialized by boot2 and passed to the loader, and then further to the kernel. Some nodes of this structures are set by boot2, the rest by the loader. This Back to structure, among other information, contains the kernel filename, BIOS Boot2 harddisk geometry, BIOS drive number for boot device, physical memory available, envp pointer etc.

 Open Boot2 file. Take a look in Boot2 make file. How do the two system calls exec and exit are created within boot2? . Take a look in Global Descriptor Table. Bootinfo header file. Choosing the most appropriate choice in 10 seconds!!

File • sys/boot/i386/boot/loader Location • kernel bootstrapping final stage • When the kernel is loaded into memory, it is being called byProcedure the loader • The main task for the loader is to boot the kernel. • It provides a scripting language that can be used to automate tasks, do Functionality pre-configuration or assist in recovery procedures.

 Have a look in /boot/loader.conf

• sys/i386/i386/machdep.c File Location •Initialize the kernel tunable parameters, passed from the bootstrapping program. •Prepare the GDT. •Prepare the IDT. •Initialize the system console. •Initialize the DDB, if it is compiled into kernel.Functionality •Initialize the TSS. •Prepare the LDT. •Set up proc0s pcb

• sys/kern/init_main.c File Location • This function performs a bubble sort of all the system initialization objects and then calls the entry of each object one by one.Functionality • Call System Scheduler which represents a process with PID 0.

 boot0 : The first stage is written entirely in assembly language and does the work necessary for non-assembly-language code to operate. boot1: Used by floppy disks.. boot2: The third stage does machine- independent operations, completing its work by starting up the system-resident processes (Filesystems and kernel loading).

 Init : kernel starts Init as the first process. Init starts shell and runs Resource Configuration (rc) files. rc config files (for start and set options of services) are /etc/rc , /etc/defaults/rc.conf , /etc/rc.conf

 Machine Dependent. Assembly-Language Startup. Machine-Dependent Initialization. Message Buffer. System Data Structures. Autoconfiguration. Device Probing. Device Attachment

 Machine Independent. Proc0 Creation. User-Level Initialization

 System-Startup Topics. Kernel Configuration. Passage of Information To and From the Kernel.

KernelStatic Modules KLDsCompiled in the kernel Loaded during run time

 Some modules should be compiled in the kernel and can’t be loaded during run time.  Those modules are ones that used before control turned to the user. Kernel Loadable Modules result large flexibility in the kernel size since only needed KLD’s are loaded.  We can use Binary file for total KLD’s indexing and make it easier for the user to determine what KLD’s can be loaded at boot time without the need for loading each one separately.

Application KernelHardware

• <sys/sysctl.h> • <sys/socket.h> • <vm/vm_param.h> File • <netinet/in.h> Location • <netinet/icmp_var.h> • <netinet/udp_var.h> • get or set kernel state • The sysctl utility retrieves kernel state and allows processes with appropriate privilege to set kernel state. The state to be retrieved or set is described using a “Management Information BaseFunctionality (“MIB) style name, described as a dotted set of components. • To retrieve the maximum number of processes allowed in the system, one would use the following request sysctl kern.maxproc • Information about the system clock rate may be obtained with sysctl Example kern.clockrate

 Check Sysctl files and test them

 Q. How do we can realize object oriented approach in FreeBSD Kernel? A. Kernel Objects (Kobj)

Terminology Description A set of data - data structure - data Object allocation. Method An operation – function Class One or more methods. Interface A standard set of one or more methods

• Kobj works by generating descriptions of methods. Each description holds a unique id as well as a default function. The descriptions address is used Kobj to uniquely identify the method within a class method table. • A class is built by creating a method table associating one or more functions with method descriptions. Before use the class is compiled. The compilation allocates a cache and associates it with the class. A unique id is assigned to each method description within the method table of the class if not already Class done so by another referencing class compilation • To retrieve the maximum number of processes allowed in the system, one would use the following request sysctl kern.maxproc • Information about the system clock rate may be obtained with sysctlExample kern.clockrate

New Method Use Method New Method Use script to generate function associated function to qualify The default within the class is argumentsfunction associated used with the methoddescription is used Found Not Found automatically reference the generated function proceeds to use the class method table to find the method description for a lookup looks up the method by using the Not Cached unique id associated with the method description as a hash into the cache associated with the objects class

 Kobj Naming Conventions.

 FreeBSD Handbook Web Pagehttp://www.freebsd.org/doc/en_US.ISO8859- 1/books/handbook/ McKusick, M., et al. The design and implementation of the 4.4BSD operating system, Addison-Wesley, 1996. ArabBSD Web Pagehttps://sites.google.com/site/arabbsd/ This presentation is available on ArabBSD Website.

https://jujo00obo2o234ungd3t8qjfcjrs3o6k-a-sites-opensocial.googleusercontent.com	1358
http://jujo00obo2o234ungd3t8qjfcjrs3o6k-a-sites-opensocial.googleusercontent.com	95

Lecture1 Introduction

by Mohammed Farrag

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 1,453

Statistics

Lecture1 Introduction Presentation Transcript