A character device typically transfers data to and from a user application — they behave like
pipes or serial ports, instantly reading or writing the byte data in a character-by-character stream.
They provide the framework for many typical drivers, such as those that are required for
interfacing to serial communications, video capture, and audio devices. The main alternative to a
character device is a block device. Block devices behave in a similar fashion to regular files,
allowing a buffered array of cached data to be viewed or manipulated with operations such as
reads, writes, and seeks. Both device types can be accessed through device files that are attached
to the file system tree. For example, the program code that is presented in this article builds to
become a device /dev/ebbchar, which appears on your Linux system as follows:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ lsmod
Module Size Used by
ebbchar 2754 0
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ls -l /dev/ebb*
crw-rw-rwT 1 root root 240, 0 Apr 11 15:34 /dev/ebbchar
A straightforward character driver that can be used to pass information between a Linux user-
space program and a loadable kernel module (LKM), which is running in Linux kernel space. In
this example, a C user-space application sends a string to the LKM. The LKM then responds
with the message that was sent along with the number of letters that the sent message contains.
Later in the article I describe why we need to solve synchronization problems that arise with this
approach, and I provide a version of the program that uses mutex locks to provide a solution.
Before describing the source code for the driver in this article, there are a few concepts that need
to be discussed, such as device driver major and minor numbers, and the File Operations data
structure.
Major and Minor Numbers
Device drivers have an associated major and minor number. For example, /dev/ram0 and
/dev/null are associated with a driver with major number 1, and /dev/tty0 and /dev/ttyS0 are
associated with a driver with major number 4. The major number is used by the kernel to identify
the correct device driver when the device is accessed. The role of the minor number is device
dependent, and is handled internally within the driver. You can see the major/minor number pair
for each device if you perform a listing in the /dev directory. For example:
molloyd@beaglebone:/dev$ ls -l
crw-rw---T 1 root i2c 89, 0 Jan 1 2000 i2c-0
brw-rw---T 1 root disk 1, 0 Mar 1 20:46 ram0
brw-rw---T 1 root floppy 179, 0 Mar 1 20:46 mmcblk0
crw-rw-rw- 1 root root 1, 3 Mar 1 20:46 null
crw------- 1 root root 4, 0 Mar 1 20:46 tty0
crw-rw---T 1 root dialout 4, 64 Mar 1 20:46 ttyS0
…
Character devices are identified by a ‘c‘ in the first column of a listing, and block devices are
identified by a ‘b‘. The access permissions, owner, and group of the device is provided for each
device. Regular user accounts on the BeagleBone are members of some of these .
A character device typically transfers data to and from a user appli.pdf
1. A character device typically transfers data to and from a user application — they behave like
pipes or serial ports, instantly reading or writing the byte data in a character-by-character stream.
They provide the framework for many typical drivers, such as those that are required for
interfacing to serial communications, video capture, and audio devices. The main alternative to a
character device is a block device. Block devices behave in a similar fashion to regular files,
allowing a buffered array of cached data to be viewed or manipulated with operations such as
reads, writes, and seeks. Both device types can be accessed through device files that are attached
to the file system tree. For example, the program code that is presented in this article builds to
become a device /dev/ebbchar, which appears on your Linux system as follows:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ lsmod
Module Size Used by
ebbchar 2754 0
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ls -l /dev/ebb*
crw-rw-rwT 1 root root 240, 0 Apr 11 15:34 /dev/ebbchar
A straightforward character driver that can be used to pass information between a Linux user-
space program and a loadable kernel module (LKM), which is running in Linux kernel space. In
this example, a C user-space application sends a string to the LKM. The LKM then responds
with the message that was sent along with the number of letters that the sent message contains.
Later in the article I describe why we need to solve synchronization problems that arise with this
approach, and I provide a version of the program that uses mutex locks to provide a solution.
Before describing the source code for the driver in this article, there are a few concepts that need
to be discussed, such as device driver major and minor numbers, and the File Operations data
structure.
Major and Minor Numbers
Device drivers have an associated major and minor number. For example, /dev/ram0 and
/dev/null are associated with a driver with major number 1, and /dev/tty0 and /dev/ttyS0 are
associated with a driver with major number 4. The major number is used by the kernel to identify
the correct device driver when the device is accessed. The role of the minor number is device
dependent, and is handled internally within the driver. You can see the major/minor number pair
for each device if you perform a listing in the /dev directory. For example:
molloyd@beaglebone:/dev$ ls -l
crw-rw---T 1 root i2c 89, 0 Jan 1 2000 i2c-0
brw-rw---T 1 root disk 1, 0 Mar 1 20:46 ram0
brw-rw---T 1 root floppy 179, 0 Mar 1 20:46 mmcblk0
crw-rw-rw- 1 root root 1, 3 Mar 1 20:46 null
2. crw------- 1 root root 4, 0 Mar 1 20:46 tty0
crw-rw---T 1 root dialout 4, 64 Mar 1 20:46 ttyS0
…
Character devices are identified by a ‘c‘ in the first column of a listing, and block devices are
identified by a ‘b‘. The access permissions, owner, and group of the device is provided for each
device. Regular user accounts on the BeagleBone are members of some of these groups and
therefore have permissions to use the i2c-0 and ttyS0 devices etc. See:
molloyd@beaglebone:/dev$ groups
molloyd dialout cdrom floppy audio video plugdev users i2c spi
The device that is developed in this article appears as a device (/dev/ebbchar) in the /dev
directory.
It is possible to manually create a block or character device file entry and later associate it with
your device (e.g., sudo mknod /dev/test c 92 1), but this approach is prone to problems. One such
problem is that you have to ensure that the number you choose (e.g., 92 in this case) is not
already in use. On the BeagleBone, you could examine the file/usr/src/linux-headers-3.8.13-
bone70/include/uapi/linux/major.h for a list of all system device major numbers. However, a
device that idenfies a “unique” major number using this approach would not be very portable, as
the major number of the device could clash with that of another device on another Linux SBC or
Linux distribution. The code that is provided in this article automatically identifies an
appropriate major number to use.
The File Operations Data Structure
The file_operations data structure that is defined in /linux/fs.h holds pointers to functions
(function pointers) within a driver that allows you to define the behavior of certain file
operations. For example, Listing 1 is a segment of the data structure from /linux/fs.h. The driver
in this article provides an implementation for the read, write, open, and releasesystem call file
operations. If you do not provide an implementation for one of the entries in this data structure
then it will simply point to NULL, making it inaccessible. Listing 1 is somewhat intimidating,
given the number of operations available. However, to build the ebbchar LKM we only need to
provide an implementation for four of the entries. Therefore, Listing 1 is provided mainly as a
reference that you can use if you need to provide additional functionality within the driver
framework.
The Device Driver Source Code
The source code for the ebbchar device driver is provided in Listing 2. Similar to the code in the
first article in this series, there is an init() function and an exit() function. However, there are
additional file_operations functions that are required for the character device:
Drivers have a class name and a device name. In Listing 2, ebb (Exploring BeagleBone) is used
3. as the class name, andebbchar as the device name. This results in the creation of a device that
appears on the file system at/sys/class/ebb/ebbchar.
Building and Testing the LKM
A Makefile is required to build the LKM, as provided in Listing 3. This Makefile is very similar
to the Makefile in the first article in the series, with the exception that it also builds a user-space
C program that interacts with the LKM.
Listing 3: The Makefile for the LKM and the User-space Program
(/extras/kernel/ebbchar/Makefile)
1
2
3
4
5
6
7
obj-m+=ebbchar.o
all:
make -C /lib/modules/$(shell uname -r)/build/ M=$(PWD) modules
$(CC) testebbchar.c -o test
clean:
make -C /lib/modules/$(shell uname -r)/build/ M=$(PWD) clean
User Access to the Device using Udev Rules
Throughout this article, the program that interfaces to the LKM device is executed using sudo. It
would be very useful to set up our LKM device so that it can be accessed by a particular user or
group, while still protecting the file system. To address this issue, you can use an advanced
feature of Linux called udev rules that enables you to customize the behavior of the udevd
service. This service gives you some user-space control over devices on your Linux system.
For example, to give user-level access to the ebbchar device, the first step is to identify the sysfs
entry for the device. You can achieve this by using a simple find:
root@beaglebone:/sys# find . -name "ebbchar"
./devices/virtual/ebb/ebbchar
./class/ebb/ebbchar
./module/ebbchar
We then need to identify the KERNEL and SUBSYSTEM values about which to write the rule.
You can use the udevadmcommand to perform this task:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ udevadm info -a -p
4. /sys/class/ebb/ebbchar
Udevadm info starts with the device specified by the devpath and then walks up the chain of
parent
devices. It prints for every device found, all possible attributes in the udev rules key format. A
rule to match, can be composed by the attributes of the device and the attributes from one single
parent device.
looking at device '/devices/virtual/ebb/ebbchar':
KERNEL=="ebbchar"
SUBSYSTEM=="ebb"
DRIVER==""
The rules are contained in the /etc/udev/rules.d directory. A new rule can be added as a file using
these values, where the file begins with a priority number. Using a name such as 99-
ebbchar.rules creates a new rule with the lowest priority, so that it does not interfere with other
device rules. The rule can be written as in Listing 5 and placed in the/etc/udev/rules.d directory
as follows:
molloyd@beaglebone:/etc/udev/rules.d$ ls
50-hidraw.rules 50-spi.rules 60-omap-tty.rules 70-persistent-net.rules 99-ebbchar.rules
molloyd@beaglebone:/etc/udev/rules.d$ more 99-ebbchar.rules
#Rules file for the ebbchar device driver
KERNEL=="ebbchar", SUBSYSTEM=="ebb", MODE="0666"
1
2
#Rules file for the ebbchar device driver
KERNEL=="ebbchar", SUBSYSTEM=="ebb", MODE="0666"
Once the rules file is added to your system, you can test it using:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ sudo insmod ebbchar.ko
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ls -l /dev/ebbchar
crw-rw-rwT 1 root root 240, 0 Apr 9 00:44 /dev/ebbchar
You can see that user and group now have the permissions required to read from and write to
this device. Interestingly, the sticky bit is also set. The sticky bit usually means that write
permission is not sufficient to delete files. Therefore, in the/tmp directory any user can create
files, but no user can delete another user’s files. The sticky bit is represented by a capital T in the
final character place. This usually appears as a lower-case t unless the executable (x) bit for
others is set; however, when the x bit is not set it appears as a capital T. However, it is not
entirely clear why the sticky bit is being set by udev — it appears to be unusual to udev rules
under Debian. At this point the test application can be executed without requiring superuser
5. permissions.
The strace Command
The strace command is a very useful debugging tool that can execute a program in order to
intercept and record the system calls that it performs. The system call name, the arguments
passed, and the resulting return value are all visible, which makes it a valuable tool for solving
runtime issues. Importantly, you do not need the source code for the executable in order to view
the output of strace. For example, you can utilize strace on your user-space application in order
to view the communication between the user-space program and the kernel module, which
results in the following for the test application:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ sudo apt-get install strace
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ strace -v
usage: strace [-dffhiqrtttTvVxx] [-a column] [-e expr] … [-o file]
[-p pid] … [-s strsize] [-u username] [-E var=val] …
[command [arg …]]
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ sudo strace ./test
execve("./test", ["./test"], [/* 15 vars */]) = 0
…
write(1, "Starting device test code exampl"..., 37Starting de…) = 37
open("/dev/ebbchar", O_RDWR) = 3
write(1, "Writing message to the device [T"..., 60Writing message …) = 60
write(3, "Testing the EBBChar device", 26) = 26
write(1, "Reading from the device... ", 27Reading from the device…) = 27
read(3, "", 100) = 0
write(1, "The received message is: [Testin"..., 66The received …) = 66
write(1, "End of the program ", 19End of the program) = 19
exit_group(0) = ?
The system call output gives us impressive insight into the communication that takes place
between the user-space program test and the /dev/ebbchar device driver.
LKM Synchronization Problems
There is a serious problem with the LKM that is described in Listing 2. In the first article in this
series I pointed out that LKMs do not execute sequentially and that they can be interrupted.
Those facts have important consequences for the code that is written in this article, which can be
demonstrated as follows:
Step 1: At the first terminal window shell you can execute the test application, but do not allow it
to run to completion (by not pressing ENTER when prompted), as follows:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
6. Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the first terminal window
Writing message to the device [This is the message from the first terminal window].
Press ENTER to read back from the device...
Step 2: Then at a second terminal window shell you can execute the same test application
simultaneously as a second process on the Linux device. For example:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the second terminal window
Writing message to the device [This is the message from the second terminal window].
Press ENTER to read back from the device...
Step 3: You can then return to the first terminal window shell and press ENTER to run the
program to completion, which results in the following output (shaded output is the repeated
output from Step 1):
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the first terminal window
Writing message to the device [This is the message from the first terminal window].
Press ENTER to read back from the device...
Reading from the device...
The received message is: [This is the message from the second terminal window(51 letters)]
End of the program
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$
You can see that the received message is actually the message that was sent by the test
application from Step 2, which is running in the second terminal window shell (not the first as
might be expected). This is because the message that was sent in Step 2 overwrote the string
message that was being stored by the LKM as a result of Step 1.
Step 4: You can return to the second terminal shell and run it to completion by pressing ENTER,
which results in (the shaded output is the repeated output from Step 2):
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the second terminal window
7. Writing message to the device [This is the message from the second terminal window].
Press ENTER to read back from the device...
Reading from the device...
The received message is: []
End of the program
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$
No string is received. That is because the LKM is not storing any messages at that point in time.
It has already delivered the stored message to the first terminal window test application and reset
the buffer index to 0
Adding Mutex Locks
The Linux kernel provides a full implementation of semaphores — a data type (struct
semaphore) that is used for controlling access by multiple processes to a shared resource. The
easiest way to use this semaphore code within the kernel is to use mutexes, as there is a full set
of helper functions and macros available.
A simple way to prevent the problems described above is to prevent two processes from using
the /dev/ebbchar device at the same time. A mutex is a lock that can set (put down) before a
process begins using a shared resource. The lock can then be released (brought up) when the
process is finished using the shared resource. When the lock has been set, no other process can
access the locked code region. Once the mutex lock has been released by the process that locked
it, the shared region of code is once again available to be accessed by the other process, which in
turn locks the resource.
1
2
3
4
5
6
7
obj-m+=ebbchar.o
all:
make -C /lib/modules/$(shell uname -r)/build/ M=$(PWD) modules
$(CC) testebbchar.c -o test
clean:
make -C /lib/modules/$(shell uname -r)/build/ M=$(PWD) clean
User Access to the Device using Udev Rules
8. Throughout this article, the program that interfaces to the LKM device is executed using sudo. It
would be very useful to set up our LKM device so that it can be accessed by a particular user or
group, while still protecting the file system. To address this issue, you can use an advanced
feature of Linux called udev rules that enables you to customize the behavior of the udevd
service. This service gives you some user-space control over devices on your Linux system.
For example, to give user-level access to the ebbchar device, the first step is to identify the
sysfs entry for the device. You can achieve this by using a simple find:
root@beaglebone:/sys# find . -name "ebbchar"
./devices/virtual/ebb/ebbchar
./class/ebb/ebbchar
./module/ebbchar
We then need to identify the KERNEL and SUBSYSTEM values about which to write the rule.
You can use the udevadmcommand to perform this task:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ udevadm info -a -p
/sys/class/ebb/ebbchar
Udevadm info starts with the device specified by the devpath and then walks up the chain of
parent
devices. It prints for every device found, all possible attributes in the udev rules key format. A
rule to match, can be composed by the attributes of the device and the attributes from one
single
parent device.
looking at device '/devices/virtual/ebb/ebbchar':
KERNEL=="ebbchar"
SUBSYSTEM=="ebb"
DRIVER==""
The rules are contained in the /etc/udev/rules.d directory. A new rule can be added as a file
using these values, where the file begins with a priority number. Using a name such as 99-
ebbchar.rules creates a new rule with the lowest priority, so that it does not interfere with other
device rules. The rule can be written as in Listing 5 and placed in the/etc/udev/rules.d directory
as follows:
molloyd@beaglebone:/etc/udev/rules.d$ ls
50-hidraw.rules 50-spi.rules 60-omap-tty.rules 70-persistent-net.rules 99-ebbchar.rules
molloyd@beaglebone:/etc/udev/rules.d$ more 99-ebbchar.rules
#Rules file for the ebbchar device driver
KERNEL=="ebbchar", SUBSYSTEM=="ebb", MODE="0666"Listing 5: Udev rules file
for the ebbchar device driver (/extras/kernel/ebbchar/99-ebbchar.rules)
9. 1
2
#Rules file for the ebbchar device driver
KERNEL=="ebbchar", SUBSYSTEM=="ebb", MODE="0666"
Once the rules file is added to your system, you can test it using:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ sudo insmod ebbchar.ko
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ls -l /dev/ebbchar
crw-rw-rwT 1 root root 240, 0 Apr 9 00:44 /dev/ebbchar
You can see that user and group now have the permissions required to read from and write to
this device. Interestingly, the sticky bit is also set. The sticky bit usually means that write
permission is not sufficient to delete files. Therefore, in the/tmp directory any user can create
files, but no user can delete another user’s files. The sticky bit is represented by a capital T in
the final character place. This usually appears as a lower-case t unless the executable (x) bit for
others is set; however, when the x bit is not set it appears as a capital T. However, it is not
entirely clear why the sticky bit is being set by udev — it appears to be unusual to udev rules
under Debian. At this point the test application can be executed without requiring superuser
permissions.
The strace Command
The strace command is a very useful debugging tool that can execute a program in order to
intercept and record the system calls that it performs. The system call name, the arguments
passed, and the resulting return value are all visible, which makes it a valuable tool for solving
runtime issues. Importantly, you do not need the source code for the executable in order to view
the output of strace. For example, you can utilize strace on your user-space application in order
to view the communication between the user-space program and the kernel module, which
results in the following for the test application:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ sudo apt-get install strace
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ strace -v
usage: strace [-dffhiqrtttTvVxx] [-a column] [-e expr] … [-o file]
[-p pid] … [-s strsize] [-u username] [-E var=val] …
[command [arg …]]
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ sudo strace ./test
execve("./test", ["./test"], [/* 15 vars */]) = 0
…
write(1, "Starting device test code exampl"..., 37Starting de…) = 37
open("/dev/ebbchar", O_RDWR) = 3
write(1, "Writing message to the device [T"..., 60Writing message …) = 60
10. write(3, "Testing the EBBChar device", 26) = 26
write(1, "Reading from the device... ", 27Reading from the device…) = 27
read(3, "", 100) = 0
write(1, "The received message is: [Testin"..., 66The received …) = 66
write(1, "End of the program ", 19End of the program) = 19
exit_group(0) = ?
The system call output gives us impressive insight into the communication that takes place
between the user-space program test and the /dev/ebbchar device driver.
LKM Synchronization Problems
There is a serious problem with the LKM that is described in Listing 2. In the first article in this
series I pointed out that LKMs do not execute sequentially and that they can be interrupted.
Those facts have important consequences for the code that is written in this article, which can be
demonstrated as follows:
Step 1: At the first terminal window shell you can execute the test application, but do not allow
it to run to completion (by not pressing ENTER when prompted), as follows:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the first terminal window
Writing message to the device [This is the message from the first terminal window].
Press ENTER to read back from the device...
Step 2: Then at a second terminal window shell you can execute the same test application
simultaneously as a second process on the Linux device. For example:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the second terminal window
Writing message to the device [This is the message from the second terminal window].
Press ENTER to read back from the device...
Step 3: You can then return to the first terminal window shell and press ENTER to run the
program to completion, which results in the following output (shaded output is the repeated
output from Step 1):
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the first terminal window
11. Writing message to the device [This is the message from the first terminal window].
Press ENTER to read back from the device...
Reading from the device...
The received message is: [This is the message from the second terminal window(51 letters)]
End of the program
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$
You can see that the received message is actually the message that was sent by the test
application from Step 2, which is running in the second terminal window shell (not the first as
might be expected). This is because the message that was sent in Step 2 overwrote the string
message that was being stored by the LKM as a result of Step 1.
Step 4: You can return to the second terminal shell and run it to completion by pressing
ENTER, which results in (the shaded output is the repeated output from Step 2):
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the second terminal window
Writing message to the device [This is the message from the second terminal window].
Press ENTER to read back from the device...
Reading from the device...
The received message is: []
End of the program
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$
No string is received. That is because the LKM is not storing any messages at that point in
time. It has already delivered the stored message to the first terminal window test application
and reset the buffer index to 0
Adding Mutex Locks
The Linux kernel provides a full implementation of semaphores — a data type (struct
semaphore) that is used for controlling access by multiple processes to a shared resource. The
easiest way to use this semaphore code within the kernel is to use mutexes, as there is a full set
of helper functions and macros available.
A simple way to prevent the problems described above is to prevent two processes from using
the /dev/ebbchar device at the same time. A mutex is a lock that can set (put down) before a
process begins using a shared resource. The lock can then be released (brought up) when the
process is finished using the shared resource. When the lock has been set, no other process can
access the locked code region. Once the mutex lock has been released by the process that locked
it, the shared region of code is once again available to be accessed by the other process, which
12. in turn locks the resource.
Solution
A character device typically transfers data to and from a user application — they behave like
pipes or serial ports, instantly reading or writing the byte data in a character-by-character stream.
They provide the framework for many typical drivers, such as those that are required for
interfacing to serial communications, video capture, and audio devices. The main alternative to a
character device is a block device. Block devices behave in a similar fashion to regular files,
allowing a buffered array of cached data to be viewed or manipulated with operations such as
reads, writes, and seeks. Both device types can be accessed through device files that are attached
to the file system tree. For example, the program code that is presented in this article builds to
become a device /dev/ebbchar, which appears on your Linux system as follows:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ lsmod
Module Size Used by
ebbchar 2754 0
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ls -l /dev/ebb*
crw-rw-rwT 1 root root 240, 0 Apr 11 15:34 /dev/ebbchar
A straightforward character driver that can be used to pass information between a Linux user-
space program and a loadable kernel module (LKM), which is running in Linux kernel space. In
this example, a C user-space application sends a string to the LKM. The LKM then responds
with the message that was sent along with the number of letters that the sent message contains.
Later in the article I describe why we need to solve synchronization problems that arise with this
approach, and I provide a version of the program that uses mutex locks to provide a solution.
Before describing the source code for the driver in this article, there are a few concepts that need
to be discussed, such as device driver major and minor numbers, and the File Operations data
structure.
Major and Minor Numbers
Device drivers have an associated major and minor number. For example, /dev/ram0 and
/dev/null are associated with a driver with major number 1, and /dev/tty0 and /dev/ttyS0 are
associated with a driver with major number 4. The major number is used by the kernel to identify
the correct device driver when the device is accessed. The role of the minor number is device
dependent, and is handled internally within the driver. You can see the major/minor number pair
for each device if you perform a listing in the /dev directory. For example:
molloyd@beaglebone:/dev$ ls -l
13. crw-rw---T 1 root i2c 89, 0 Jan 1 2000 i2c-0
brw-rw---T 1 root disk 1, 0 Mar 1 20:46 ram0
brw-rw---T 1 root floppy 179, 0 Mar 1 20:46 mmcblk0
crw-rw-rw- 1 root root 1, 3 Mar 1 20:46 null
crw------- 1 root root 4, 0 Mar 1 20:46 tty0
crw-rw---T 1 root dialout 4, 64 Mar 1 20:46 ttyS0
…
Character devices are identified by a ‘c‘ in the first column of a listing, and block devices are
identified by a ‘b‘. The access permissions, owner, and group of the device is provided for each
device. Regular user accounts on the BeagleBone are members of some of these groups and
therefore have permissions to use the i2c-0 and ttyS0 devices etc. See:
molloyd@beaglebone:/dev$ groups
molloyd dialout cdrom floppy audio video plugdev users i2c spi
The device that is developed in this article appears as a device (/dev/ebbchar) in the /dev
directory.
It is possible to manually create a block or character device file entry and later associate it with
your device (e.g., sudo mknod /dev/test c 92 1), but this approach is prone to problems. One such
problem is that you have to ensure that the number you choose (e.g., 92 in this case) is not
already in use. On the BeagleBone, you could examine the file/usr/src/linux-headers-3.8.13-
bone70/include/uapi/linux/major.h for a list of all system device major numbers. However, a
device that idenfies a “unique” major number using this approach would not be very portable, as
the major number of the device could clash with that of another device on another Linux SBC or
Linux distribution. The code that is provided in this article automatically identifies an
appropriate major number to use.
The File Operations Data Structure
The file_operations data structure that is defined in /linux/fs.h holds pointers to functions
(function pointers) within a driver that allows you to define the behavior of certain file
operations. For example, Listing 1 is a segment of the data structure from /linux/fs.h. The driver
in this article provides an implementation for the read, write, open, and releasesystem call file
operations. If you do not provide an implementation for one of the entries in this data structure
then it will simply point to NULL, making it inaccessible. Listing 1 is somewhat intimidating,
given the number of operations available. However, to build the ebbchar LKM we only need to
provide an implementation for four of the entries. Therefore, Listing 1 is provided mainly as a
reference that you can use if you need to provide additional functionality within the driver
framework.
The Device Driver Source Code
14. The source code for the ebbchar device driver is provided in Listing 2. Similar to the code in the
first article in this series, there is an init() function and an exit() function. However, there are
additional file_operations functions that are required for the character device:
Drivers have a class name and a device name. In Listing 2, ebb (Exploring BeagleBone) is used
as the class name, andebbchar as the device name. This results in the creation of a device that
appears on the file system at/sys/class/ebb/ebbchar.
Building and Testing the LKM
A Makefile is required to build the LKM, as provided in Listing 3. This Makefile is very similar
to the Makefile in the first article in the series, with the exception that it also builds a user-space
C program that interacts with the LKM.
Listing 3: The Makefile for the LKM and the User-space Program
(/extras/kernel/ebbchar/Makefile)
1
2
3
4
5
6
7
obj-m+=ebbchar.o
all:
make -C /lib/modules/$(shell uname -r)/build/ M=$(PWD) modules
$(CC) testebbchar.c -o test
clean:
make -C /lib/modules/$(shell uname -r)/build/ M=$(PWD) clean
User Access to the Device using Udev Rules
Throughout this article, the program that interfaces to the LKM device is executed using sudo. It
would be very useful to set up our LKM device so that it can be accessed by a particular user or
group, while still protecting the file system. To address this issue, you can use an advanced
feature of Linux called udev rules that enables you to customize the behavior of the udevd
service. This service gives you some user-space control over devices on your Linux system.
For example, to give user-level access to the ebbchar device, the first step is to identify the sysfs
entry for the device. You can achieve this by using a simple find:
root@beaglebone:/sys# find . -name "ebbchar"
./devices/virtual/ebb/ebbchar
./class/ebb/ebbchar
15. ./module/ebbchar
We then need to identify the KERNEL and SUBSYSTEM values about which to write the rule.
You can use the udevadmcommand to perform this task:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ udevadm info -a -p
/sys/class/ebb/ebbchar
Udevadm info starts with the device specified by the devpath and then walks up the chain of
parent
devices. It prints for every device found, all possible attributes in the udev rules key format. A
rule to match, can be composed by the attributes of the device and the attributes from one single
parent device.
looking at device '/devices/virtual/ebb/ebbchar':
KERNEL=="ebbchar"
SUBSYSTEM=="ebb"
DRIVER==""
The rules are contained in the /etc/udev/rules.d directory. A new rule can be added as a file using
these values, where the file begins with a priority number. Using a name such as 99-
ebbchar.rules creates a new rule with the lowest priority, so that it does not interfere with other
device rules. The rule can be written as in Listing 5 and placed in the/etc/udev/rules.d directory
as follows:
molloyd@beaglebone:/etc/udev/rules.d$ ls
50-hidraw.rules 50-spi.rules 60-omap-tty.rules 70-persistent-net.rules 99-ebbchar.rules
molloyd@beaglebone:/etc/udev/rules.d$ more 99-ebbchar.rules
#Rules file for the ebbchar device driver
KERNEL=="ebbchar", SUBSYSTEM=="ebb", MODE="0666"
1
2
#Rules file for the ebbchar device driver
KERNEL=="ebbchar", SUBSYSTEM=="ebb", MODE="0666"
Once the rules file is added to your system, you can test it using:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ sudo insmod ebbchar.ko
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ls -l /dev/ebbchar
crw-rw-rwT 1 root root 240, 0 Apr 9 00:44 /dev/ebbchar
You can see that user and group now have the permissions required to read from and write to
this device. Interestingly, the sticky bit is also set. The sticky bit usually means that write
permission is not sufficient to delete files. Therefore, in the/tmp directory any user can create
files, but no user can delete another user’s files. The sticky bit is represented by a capital T in the
16. final character place. This usually appears as a lower-case t unless the executable (x) bit for
others is set; however, when the x bit is not set it appears as a capital T. However, it is not
entirely clear why the sticky bit is being set by udev — it appears to be unusual to udev rules
under Debian. At this point the test application can be executed without requiring superuser
permissions.
The strace Command
The strace command is a very useful debugging tool that can execute a program in order to
intercept and record the system calls that it performs. The system call name, the arguments
passed, and the resulting return value are all visible, which makes it a valuable tool for solving
runtime issues. Importantly, you do not need the source code for the executable in order to view
the output of strace. For example, you can utilize strace on your user-space application in order
to view the communication between the user-space program and the kernel module, which
results in the following for the test application:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ sudo apt-get install strace
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ strace -v
usage: strace [-dffhiqrtttTvVxx] [-a column] [-e expr] … [-o file]
[-p pid] … [-s strsize] [-u username] [-E var=val] …
[command [arg …]]
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ sudo strace ./test
execve("./test", ["./test"], [/* 15 vars */]) = 0
…
write(1, "Starting device test code exampl"..., 37Starting de…) = 37
open("/dev/ebbchar", O_RDWR) = 3
write(1, "Writing message to the device [T"..., 60Writing message …) = 60
write(3, "Testing the EBBChar device", 26) = 26
write(1, "Reading from the device... ", 27Reading from the device…) = 27
read(3, "", 100) = 0
write(1, "The received message is: [Testin"..., 66The received …) = 66
write(1, "End of the program ", 19End of the program) = 19
exit_group(0) = ?
The system call output gives us impressive insight into the communication that takes place
between the user-space program test and the /dev/ebbchar device driver.
LKM Synchronization Problems
There is a serious problem with the LKM that is described in Listing 2. In the first article in this
series I pointed out that LKMs do not execute sequentially and that they can be interrupted.
Those facts have important consequences for the code that is written in this article, which can be
17. demonstrated as follows:
Step 1: At the first terminal window shell you can execute the test application, but do not allow it
to run to completion (by not pressing ENTER when prompted), as follows:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the first terminal window
Writing message to the device [This is the message from the first terminal window].
Press ENTER to read back from the device...
Step 2: Then at a second terminal window shell you can execute the same test application
simultaneously as a second process on the Linux device. For example:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the second terminal window
Writing message to the device [This is the message from the second terminal window].
Press ENTER to read back from the device...
Step 3: You can then return to the first terminal window shell and press ENTER to run the
program to completion, which results in the following output (shaded output is the repeated
output from Step 1):
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the first terminal window
Writing message to the device [This is the message from the first terminal window].
Press ENTER to read back from the device...
Reading from the device...
The received message is: [This is the message from the second terminal window(51 letters)]
End of the program
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$
You can see that the received message is actually the message that was sent by the test
application from Step 2, which is running in the second terminal window shell (not the first as
might be expected). This is because the message that was sent in Step 2 overwrote the string
message that was being stored by the LKM as a result of Step 1.
Step 4: You can return to the second terminal shell and run it to completion by pressing ENTER,
which results in (the shaded output is the repeated output from Step 2):
18. molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the second terminal window
Writing message to the device [This is the message from the second terminal window].
Press ENTER to read back from the device...
Reading from the device...
The received message is: []
End of the program
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$
No string is received. That is because the LKM is not storing any messages at that point in time.
It has already delivered the stored message to the first terminal window test application and reset
the buffer index to 0
Adding Mutex Locks
The Linux kernel provides a full implementation of semaphores — a data type (struct
semaphore) that is used for controlling access by multiple processes to a shared resource. The
easiest way to use this semaphore code within the kernel is to use mutexes, as there is a full set
of helper functions and macros available.
A simple way to prevent the problems described above is to prevent two processes from using
the /dev/ebbchar device at the same time. A mutex is a lock that can set (put down) before a
process begins using a shared resource. The lock can then be released (brought up) when the
process is finished using the shared resource. When the lock has been set, no other process can
access the locked code region. Once the mutex lock has been released by the process that locked
it, the shared region of code is once again available to be accessed by the other process, which in
turn locks the resource.
1
2
3
4
5
6
7
obj-m+=ebbchar.o
all:
make -C /lib/modules/$(shell uname -r)/build/ M=$(PWD) modules
19. $(CC) testebbchar.c -o test
clean:
make -C /lib/modules/$(shell uname -r)/build/ M=$(PWD) clean
User Access to the Device using Udev Rules
Throughout this article, the program that interfaces to the LKM device is executed using sudo. It
would be very useful to set up our LKM device so that it can be accessed by a particular user or
group, while still protecting the file system. To address this issue, you can use an advanced
feature of Linux called udev rules that enables you to customize the behavior of the udevd
service. This service gives you some user-space control over devices on your Linux system.
For example, to give user-level access to the ebbchar device, the first step is to identify the
sysfs entry for the device. You can achieve this by using a simple find:
root@beaglebone:/sys# find . -name "ebbchar"
./devices/virtual/ebb/ebbchar
./class/ebb/ebbchar
./module/ebbchar
We then need to identify the KERNEL and SUBSYSTEM values about which to write the rule.
You can use the udevadmcommand to perform this task:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ udevadm info -a -p
/sys/class/ebb/ebbchar
Udevadm info starts with the device specified by the devpath and then walks up the chain of
parent
devices. It prints for every device found, all possible attributes in the udev rules key format. A
rule to match, can be composed by the attributes of the device and the attributes from one
single
parent device.
looking at device '/devices/virtual/ebb/ebbchar':
KERNEL=="ebbchar"
SUBSYSTEM=="ebb"
DRIVER==""
The rules are contained in the /etc/udev/rules.d directory. A new rule can be added as a file
using these values, where the file begins with a priority number. Using a name such as 99-
ebbchar.rules creates a new rule with the lowest priority, so that it does not interfere with other
device rules. The rule can be written as in Listing 5 and placed in the/etc/udev/rules.d directory
as follows:
molloyd@beaglebone:/etc/udev/rules.d$ ls
50-hidraw.rules 50-spi.rules 60-omap-tty.rules 70-persistent-net.rules 99-ebbchar.rules
20. molloyd@beaglebone:/etc/udev/rules.d$ more 99-ebbchar.rules
#Rules file for the ebbchar device driver
KERNEL=="ebbchar", SUBSYSTEM=="ebb", MODE="0666"Listing 5: Udev rules file
for the ebbchar device driver (/extras/kernel/ebbchar/99-ebbchar.rules)
1
2
#Rules file for the ebbchar device driver
KERNEL=="ebbchar", SUBSYSTEM=="ebb", MODE="0666"
Once the rules file is added to your system, you can test it using:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ sudo insmod ebbchar.ko
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ls -l /dev/ebbchar
crw-rw-rwT 1 root root 240, 0 Apr 9 00:44 /dev/ebbchar
You can see that user and group now have the permissions required to read from and write to
this device. Interestingly, the sticky bit is also set. The sticky bit usually means that write
permission is not sufficient to delete files. Therefore, in the/tmp directory any user can create
files, but no user can delete another user’s files. The sticky bit is represented by a capital T in
the final character place. This usually appears as a lower-case t unless the executable (x) bit for
others is set; however, when the x bit is not set it appears as a capital T. However, it is not
entirely clear why the sticky bit is being set by udev — it appears to be unusual to udev rules
under Debian. At this point the test application can be executed without requiring superuser
permissions.
The strace Command
The strace command is a very useful debugging tool that can execute a program in order to
intercept and record the system calls that it performs. The system call name, the arguments
passed, and the resulting return value are all visible, which makes it a valuable tool for solving
runtime issues. Importantly, you do not need the source code for the executable in order to view
the output of strace. For example, you can utilize strace on your user-space application in order
to view the communication between the user-space program and the kernel module, which
results in the following for the test application:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ sudo apt-get install strace
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ strace -v
usage: strace [-dffhiqrtttTvVxx] [-a column] [-e expr] … [-o file]
[-p pid] … [-s strsize] [-u username] [-E var=val] …
[command [arg …]]
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ sudo strace ./test
execve("./test", ["./test"], [/* 15 vars */]) = 0
21. …
write(1, "Starting device test code exampl"..., 37Starting de…) = 37
open("/dev/ebbchar", O_RDWR) = 3
write(1, "Writing message to the device [T"..., 60Writing message …) = 60
write(3, "Testing the EBBChar device", 26) = 26
write(1, "Reading from the device... ", 27Reading from the device…) = 27
read(3, "", 100) = 0
write(1, "The received message is: [Testin"..., 66The received …) = 66
write(1, "End of the program ", 19End of the program) = 19
exit_group(0) = ?
The system call output gives us impressive insight into the communication that takes place
between the user-space program test and the /dev/ebbchar device driver.
LKM Synchronization Problems
There is a serious problem with the LKM that is described in Listing 2. In the first article in this
series I pointed out that LKMs do not execute sequentially and that they can be interrupted.
Those facts have important consequences for the code that is written in this article, which can be
demonstrated as follows:
Step 1: At the first terminal window shell you can execute the test application, but do not allow
it to run to completion (by not pressing ENTER when prompted), as follows:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the first terminal window
Writing message to the device [This is the message from the first terminal window].
Press ENTER to read back from the device...
Step 2: Then at a second terminal window shell you can execute the same test application
simultaneously as a second process on the Linux device. For example:
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the second terminal window
Writing message to the device [This is the message from the second terminal window].
Press ENTER to read back from the device...
Step 3: You can then return to the first terminal window shell and press ENTER to run the
program to completion, which results in the following output (shaded output is the repeated
output from Step 1):
22. molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the first terminal window
Writing message to the device [This is the message from the first terminal window].
Press ENTER to read back from the device...
Reading from the device...
The received message is: [This is the message from the second terminal window(51 letters)]
End of the program
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$
You can see that the received message is actually the message that was sent by the test
application from Step 2, which is running in the second terminal window shell (not the first as
might be expected). This is because the message that was sent in Step 2 overwrote the string
message that was being stored by the LKM as a result of Step 1.
Step 4: You can return to the second terminal shell and run it to completion by pressing
ENTER, which results in (the shaded output is the repeated output from Step 2):
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$ ./test
Starting device test code example...
Type in a short string to send to the kernel module:
This is the message from the second terminal window
Writing message to the device [This is the message from the second terminal window].
Press ENTER to read back from the device...
Reading from the device...
The received message is: []
End of the program
molloyd@beaglebone:~/exploringBB/extras/kernel/ebbchar$
No string is received. That is because the LKM is not storing any messages at that point in
time. It has already delivered the stored message to the first terminal window test application
and reset the buffer index to 0
Adding Mutex Locks
The Linux kernel provides a full implementation of semaphores — a data type (struct
semaphore) that is used for controlling access by multiple processes to a shared resource. The
easiest way to use this semaphore code within the kernel is to use mutexes, as there is a full set
of helper functions and macros available.
A simple way to prevent the problems described above is to prevent two processes from using
the /dev/ebbchar device at the same time. A mutex is a lock that can set (put down) before a
23. process begins using a shared resource. The lock can then be released (brought up) when the
process is finished using the shared resource. When the lock has been set, no other process can
access the locked code region. Once the mutex lock has been released by the process that locked
it, the shared region of code is once again available to be accessed by the other process, which
in turn locks the resource.