CS 300 : Operating Systems

Adding a new system call to the Linux kernel


Though adding system calls to the kernel should never be your first choice to expand the kernel's functionality in the real world (applications and developers expect a certain set of calls), for the purpose of this class it will be a rather common habit. Below we describe how to add a new system call to the Linux kernel v3.0.0 from the kernel repository. For information on the underlying mechanisms, refer here, as well as in the suggested class resources and of course, any internet search engine.

We will be adding a system call named which takes an integer as an argument and prints it out along with a greeting.

Adding a new system call - the Linux-kernel side

  • cd linux-source
    • Change your current working directory to the kernel directory. From now on we assume you are in the kernel directory

  • mkdir cs300
    • Make a new directory to host your new system call. We will call this directory cs300/

  • vi cs300/cs300_test.c
    • Open a new file and add your system call there. As described, ours will be called cs300_test, it will take a single argument (int) and print it along with a simple greeting:

      #include <linux/kernel.h>

      asmlinkage long sys_cs300_test(int arg0)
        printk(" Hello World !");
        printk("--syscall arg %d", arg0);

        return((long) arg0);

      Save your file under cs300/
    • Function printk is the kernel's analogue of printf
    • Due to space limitations we have jmacs in the virtual disk image but not emacs - it is Joe's editor with emacs key-bindings and should be familiar for emacs users.

  • vi cs300/Makefile
    • Open a new Makefile to inform the kernel how to compile your new file(s). You only need to define how the single file you gave is turned into an object file, so in this simple case

      #Makefile start

      obj-y := cs300_test.o

      #Makefile end

      Save your file under cs300/

  • vi Makefile
    • Open the topmost Makefile and inform it of the new directory. The Makefile will descend into it and recursively build whatever it can. To do so, find the line where core-y is defined (~610) and add your new directory to it:

      core-y        += kernel/ mm/ fs/ ipc/ security/ crypto/ block/ cs300/

      Save the Makefile where it was.
  • vi arch/x86/kernel/syscall_table_32.S
    • Edit (with vi or any editor of your preference) syscall_table_32.S to add a new system call function pointer. Go to the last line and add the line

      .long sys_cs300_test           /* 327 */

      By this line you add the pointer to your function in the system-calls table; this is system call number 327 for i386 processors.

  • vi  include/asm-x86/unistd_32.h
    • Edit unistd_32.h to add your  new system call number. Find where the rest of the system call numbers are, go to the last one (326) and add the line

      #define __NR_cs300_test        327

      Now you defined how your system call function pointer can be referenced from the system calls table.

Adding a new system call - the user-level side

Since all the libraries we have are unaware of our new system call, in order to test it we need to provide the user with a library call to trigger the syscall. Boot your GNU/Linux distribution (Debian in QDGL's case), login as a simple user and follow the procedure below.
  • mkdir cs300
    • Make a new directory to host your new library call. We will call this directory cs300/

  • cd cs300
    • Move to the new directory

  • vi cs300_test.c
    • Open a new file and add your library call declaration there.

      #include <stdio.h>
      #include <linux/unistd.h>
      #include <sys/syscall.h>

      #define _CS2456_TEST_ 327

      int main(int argc, char *argv[])
        printf("\nDiving to kernel level\n\n");
        syscall(_CS2456_TEST_ , 2456);
        printf("\nRising to user level\n\n");
         return 0;

      Save your file.

  • gcc cs300_test.c - o cs300_test
    • Compile your user level syscall interface

Testing your new system call

  • Boot your new kernel
    • qemu-system-x86_64 -m 64M -hda debian_squeeze_amd64_standard.qcow2 -append "root=/dev/hda1 console=ttyS0,115200n8 console=tty0" -kernel linux-source/arch/i386/boot/bzImage
    • You should be able to verify that this is a new kernel by checking its revision ( ex. uname -a )
  • Go the directory where you stored your library interface to the syscall and execute the test you wrote
  • Your system call should now work if you try it :
    • The example above printed the following output to our virtual machine:

      Diving to kernel level                                                          
       [   76.703723]  Hello World !--syscall arg 2456                                 
       Rising to user level
  • If you try the same system call in the kernel that does not support it (like the default Debian kernel), you would just see the user level printouts:
    • Diving to kernel level                                                          
       Rising to user level