Chapter 6. Working with Processes

Table of Contents

Process Trees
Parent and Child Relationships
Process Ownership
Viewing Process Information
Backgrounding Processes
Process Behaviour
Command Return Codes
Priority and Niceness
Sending Signals (and Killing Processes)
Exercises
Further Resources

Process Trees

Parent and Child Relationships

Each Linux (and Unix) process has a parent (except for the top process) and can have one or more children. The relationship is crafted when a process is launched: the process that launched the new process becomes the parent of that process. As a user, you might not know what process you are currently working in. Every program is a process, being it the shell you're typing the commands in or the graphical environment you're working with.

For instance, a user who has a terminal open can have the following process structure for this terminal:

init
`- xterm
   `- bash

You can obtain a tree of running processes using the pstree command:

$ pstree
init-+-acpid
     |-4*[agetty]
     |-agiletrack---java---19*[{java}]
     |-apache2---8*[apache2]
     |-bonobo-activati---{bonobo-activati}
     |-5*[dbus-daemon]
     |-dhcpcd
     |-gconfd-2
     |-gnome-keyring-d
     |-gnome-power-man
     |-gnome-screensav
     |-gnome-settings----{gnome-settings-}
     |-4*[gnome-vfs-daemo]
     |-gnome-volume-ma
     |-gpg-agent
     |-hald---hald-runner-+-hald-addon-acpi
     |                    |-hald-addon-cpuf
     |                    `-hald-addon-stor
     |-java---15*[{java}]
     |-login---bash---startx---xinit-+-X
     |                               `-gnome-session-+-gnome-panel
     |                                               |-metacity
     |                                               |-nautilus
     |                                               `-{gnome-session}
[...]

Now, not every process launched immediately becomes a child of the process where it was launched from. Some processes might immediately become child of the root process, most often called init. The root process is the first process launched by the kernel when it boots up. It is responsible for starting the necessary services at boot time and prepare the system for its duties.

Processes that become child of the root process usually do this because they don't want to be terminated when their parent process exits or dies: when this happens, the child processes become orphaned and the init process will terminate these processes as well. So, becoming a child of the init process will ensure that the process remains available. In the above example you'll find a good example: the dhcpcd command governs the IP address of the network interface through the DHCP protocol. If the process didn't continuously run, your IP address would be dismissed after a few minutes (or hours).

Process Ownership

When a process is launched (usually through a command the user entered) it, by default, obtains the user id and group id of its parent process. When a user logs on to the system, the login process launches a shell process with the user id and group id of the user that logged on, so every command the user launches takes the user id and group id of that user, since the parent process of every launched command is either the before mentioned shell process or one of its child processes.

Some processes however explicitly ask the Linux kernel to use a different user id and group id. This is accomplished by setting the setuid or setgid flags on the process file itself. With setuid (set user id) and setgid (set group id) the owner of the process is the owner of the file rather than the user that launched the process.

An example is the passwd command, used to change the password of a user:

$ ls -l /bin/passwd
-rws--x--x 1 root root  28956 Jul 15  2007 passwd

As you can see, the command file itself is owned by root. It also has the setuid bit set (see the s in -rws--x--x). If a user runs the passwd command, the command itself has root privileges rather than the privileges for the user. For the passwd command, this is necessary because it needs to update the password files (/etc/passwd and /etc/shadow) which are only writable by the root user (the /etc/shadow file is not even readable for regular users).

Viewing Process Information

Various tools exist to obtain process information. The next few chapters give a nice overview of these tools...

Process Lists

The main program to create a process list is the ps command. If ran inside a shell, it shows the processes that are running inside the session (meaning the processes launched from the shell, including the shell itself):

$ ps
  PID TTY          TIME CMD
24064 pts/3    00:00:00 bash
24116 pts/3    00:00:00 ps

The columns shown are:

  1. PID - process id of the process

  2. TTY - controlling terminal (this is Unix heritage where users were logged on through terminals, pts is a pseudoterminal)

  3. TIME - the execution time the process took. In the above example, both commands hardly took any CPU time on the system (bash is the shell, which is most of the time waiting for input so not consuming any CPU time, the other one is ps which gave its results in less than a second)

  4. CMD - the process name itself (the command)

Of course, several arguments to ps exist which change its behaviour. For instance, with ps -e you see the same information, but for all processes running on the system. With ps -f a few more columns are added, including the parent process id and the time the process started.

You can also limit the processes to see based on the user (ps -u user name), command name (ps -C command), really running processes (taking cpu time at the moment: ps -r) and more. For more information, see the ps manual page.

Another command that is often used to obtain process list information is the top program. The top command is an interactive command that shows you a process list, sorted by one or more values (default is CPU usage) and refreshes this list every 5 seconds (this is of course configurable):

top - 10:19:47 up 6 days,  6:41,  5 users,  load average: 1.00, 1.27, 0.92
Tasks: 120 total,   1 running, 119 sleeping,   0 stopped,   0 zombie
Cpu(s):  3.2%us,  0.7%sy,  0.0%ni, 95.6%id,  0.3%wa,  0.1%hi,  0.0%si,  0.0%st
Mem:   1545408k total,  1490968k used,    54440k free,   177060k buffers
Swap:  2008084k total,      132k used,  2007952k free,   776060k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4458 haldaemo  16   0  5488 3772 2388 S  2.0  0.2   4:23.69 hald
27255 swift     15   0  2272 1064  768 R  2.0  0.1   0:00.01 top
    1 root      15   0  1612  544  468 S  0.0  0.0   0:00.48 init
    2 root      12  -5     0    0    0 S  0.0  0.0   0:00.00 kthreadd
    3 root      39  19     0    0    0 S  0.0  0.0   0:00.45 ksoftirqd/0
    4 root      10  -5     0    0    0 S  0.0  0.0   0:01.95 events/0
    5 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 khelper
   60 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 kblockd/0
   61 root      11  -5     0    0    0 S  0.0  0.0   0:25.77 kacpid
   62 root      11  -5     0    0    0 S  0.0  0.0   0:09.60 kacpi_notify
  171 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 ata/0
  172 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 ata_aux
  173 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 ksuspend_usbd
  176 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 khubd
  178 root      10  -5     0    0    0 S  0.0  0.0   0:00.01 kseriod
  196 root      10  -5     0    0    0 S  0.0  0.0   0:01.13 kswapd0
  197 root      20  -5     0    0    0 S  0.0  0.0   0:00.00 aio/0

There is plenty of information in the top screen...

top - 10:19:47 up 6 days,  6:41,  5 users,  load average: 1.00, 1.27, 0.92

The first line shows you the uptime of the system (this system is running for 6 days, 6 hours and 41 minutes), the number of logged on users (beware, this is not the number of different users - if a user launches 3 xterms inside a graphical session he will be shown as four logged on users) and the load average.

The load average is something many people misinterpret. The load average shows the number of processes that were running or asking for CPU time during the given interval. In the above example, this means that:

  • in the last minute, an average of 1 process was asking for or using CPU time

  • in the last 5 minutes, an average of 1.27 processes were asking for or using CPU time

  • in the last 15 minutes, an average of 0.92 processes were asking for or using CPU time

For a single CPU system, you most likely don't want a number higher than 1 in the long run (for instance, the 15-minute span). The more CPUs, the higher the load average can become.

Tasks: 120 total,   1 running, 119 sleeping,   0 stopped,   0 zombie

The number of processes running on this system (120) of which 119 are sleeping (not performing any duties), 1 running (the top command itself), 0 stopped (a process in the stopped state can still be revived but is, at this moment, not accepting input or performing any tasks) and 0 zombie.

A zombie process is not really a real process: the process itself has already finished, but its parent process doesn't know this yet, so the kernel is keeping some process information until the parent process asks for the child process state.

Cpu(s):  3.2%us,  0.7%sy,  0.0%ni, 95.6%id,  0.3%wa,  0.1%hi,  0.0%si,  0.0%st

CPU state information, showing the CPU usage percentages: user processes (us), system/kernel CPU usage (sy), niced processes (ni), idle CPU (id), waiting for I/O (wa), hardware interrupts (hi), software interrupts (si) and virtual cpu stealing (st).

Most of the states are self-explanatory. The niced processes is for processes the user reniced and is a subset of the user processes percentage. The virtual CPU stealing is the percentage of time a virtual CPU waits for a real CPU and is not interesting for regular Linux/Unix users (as they don't work with virtualization).

Mem:   1545408k total,  1490968k used,    54440k free,   177060k buffers
Swap:  2008084k total,      132k used,  2007952k free,   776060k cached

Memory usage: of the 1.5 Gbyte of memory available, 1.45Gbyte is in use and 54Mbyte is free. Of the used memory, 177 Mbyte is used by the kernel for internal buffers. Also, 776 Mbyte of the used memory actually consists out of cached data which can potentially be cleared if a process would require more memory than currently available.

The swap space itself is hardly used: of the 2Gbyte of swap space defined, only 132 kbyte is in use.

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4458 haldaemo  16   0  5488 3772 2388 S  2.0  0.2   4:23.69 hald
...

The rest of the screen gives the process listing itself. The columns shown are:

  1. Process ID (PID) of the process

  2. Username (USER) showing the owner of the process

  3. Priority value (PR) of the process (the higher the value, the higher the priority). Priorities are exclusively determined by the Linux kernel.

  4. Nice value (NI) of the process (is a user sets a nice value, or renices a tool, it tells the Linux kernel how "nice" the program is - the higher the nice value, the nicer it is so (generally) the lower the priority should be).

  5. The virtual memory (VIRT) the process is occupying. This includes the memory it is actually using, mapped memory from devices, files mapped into memory and shared memory.

  6. The resident (really used) memory (RES) the process is using.

  7. The amount of possibly shared memory (SHR). It is "possibly" because the memory is shareable, but not automatically used by others already.

  8. Process state (S), which can be any of S (sleeping), R (running), D (uninterruptible sleep), T (traced or stopped) or Z (zombie).

  9. CPU usage (%CPU)

  10. Memory usage (%MEM - based on RES)

  11. Runtime (TIME+)

  12. Command (COMMAND)

Process Information

You can also be interested in more detailed process information such as the files (or connections) the process has currently open.

With lsof you can view this information. Just give the process id with it (lsof -p PID) and you get all this information. However, lsof can do much more. For instance, with lsof you can see what process is listening on a particular port:

# lsof -i :443
COMMAND   PID   USER   FD   TYPE DEVICE SIZE NODE NAME
apache2  4346   root    3u  IPv4  11484       TCP *:https (LISTEN)

Another tool that can do the same is fuser:

# fuser -v 443/tcp
                     USER        PID ACCESS COMMAND
443/tcp:             root       4346 F.... apache2

The same can be accomplished with files. For instance, to see what processes are using a particular file with fuser, just give the file name (fuser -v /path/to/file).

Backgrounding Processes

Processes can be started in the background, either because the process immediately detaches it from the running session (daemons) or because the user asks to run it in the background.

Daemons are processes that do not stay in the running session. The moment you launch a daemon process, you immediately get your prompt back as if the process has finished. However, this isn't true: the process is still running, but it is running in the background. Most daemons do not have the possibility to reattach to the session. Whether or not a process is a daemon depends on the process itself as this is a pure programmatic decision.

Backgrounded processes however are processes that stay in the running session, but do not "lock" the input devices (keyboard). As a result, the user gets the prompt back and can continue launching other processes or do other tasks. To background a process, a user can add a "&" sign at the end of the command line. For instance, to put the command "eix-update" in the background:

# eix-update &

You can see what processes are running in your session in the background using the jobs command:

# jobs
[1]- Running           eix-update &

You can put a job back into the foreground using the fg command. If you just enter fg, it'll put the last job put in the background back. If you want to select a different job, use the number that jobs returned. For instance, to return the 3rd job back to the foreground:

# fg %3

If you want to put a process that you are running in the background, use Ctrl-Z. Ctrl-Z also pauses the process, so if you want to continue the process in the background, enter "bg" afterwards as well:

# eix-update
(...)
(Press Ctrl-Z)
[1]+ Stopped         eix-update
# bg
[1]+ eix-update &

There are a few things you must keep in mind when using jobs:

  • A (non-daemon) process is attached to a running session. The moment you terminate your session, all jobs that were running in that session (both foreground and background processes) are terminated as well.

  • Although processes can be ran in the background, their output is still forwarded to your terminal. If you do not want this, you can redirect the output of a command using the > redirect. For instance, to redirect the standard output (default - 1) of update-eix to a log file and do the same for the error output (2):

    # eix-update > /var/tmp/update-eix.log 2>&1 &

    Another popular redirect is to ignore output completely:

    # eix-update > /dev/null 2>&1 &

Process Behaviour

Programs are most often launched when a user selects a tool or executes a command. They can also be invoked automatically by a running program or by the Linux kernel (although the init tool is probably the only one ever invoked by the kernel autonomously).

The next few sections give pointers on process behaviour and how you can modify it (if appropriate).

Command Return Codes

The simplest example of launching a program is a simple command you enter in the command line. Once the program has finished, it leaves behind its return code (or exit code) informing you how well it did its job.

A return code is always an integer in the range of 0 to 255. Some programs might attempt to return a code larger than 255 (or even negative). Although not technically restricted, this is not a good idea as some applications only expect a return code between 0 to 255 and might even "wrap" return codes to this range. If a program would ever have a return code of 512 for instance, it might be mapped into 0.

Every program that has successfully finished its job will (or should) return code 0. A non-zero return code means that the application has failed to finish its tasks (completely).

Inside any POSIX-compliant shell (POSIX has a standard for Unix environments, including how a shell should function) such as ksh or bash you can obtain the return code of the last command using $?:

$ ls -l
...
$ echo $?
0
$ ls -z
ls: invalid option -- z
Try `ls --help' for more information
$ echo $?
2

These return codes are important as they are the means to investigate if all commands went successfully or not, allowing you to write quite intelligent shell scripts which trigger several commands and include logic to handle command failures.

Priority and Niceness

On Linux, you can't set the priority of a process yourself: the Linux kernel does that for you, based on information about the process itself, including but not limited to if the process is I/O-bound (as such programs are most of the time user-interactive), its previous CPU consumption, possible locks it is holding and more.

You can, however, inform the kernel on what you think the process' priority ought to be. For this, you can set a nice value for the application. The value, in the range of -20 to 19, informs the Linux kernel about how nice the program should be towards the rest of the system. Negative numbers (-1 to -20) are not that nice; the Linux kernel will thus assign those a larger time slice and you'll notice that such programs usually get a higher priority. However, only the root user can assign a negative nice number to a program. Positive numbers (1 to 19) make a process more nice to the system; they will receive a lower time slice and usually a lower priority.

Thanks to this system you can launch long-lasting, non-interactive commands in the background without worrying about the inpact to your interactive user experience. The nice tool allows you to start up a command with a particular nice value.

For instance, to start a Gentoo system upgrade with the highest possible nice value (as this is something you usually want to perform in the background):

# nice -n 19 emerge -uDN @world

If a process is already running, you can change its nice value with the renice tool (for instance, to increase the nice value of the process with process id 219 with 5):

# renice +5 219

Sending Signals (and Killing Processes)

Some processes allow you to send certain signals to them. A signal is a simple integer between 0 and 64; each of them is also given a particular name. The kill tool can be used to send signals to processes, but also to obtain a list of available signals:

$ kill -l
 1) SIGHUP       2) SIGINT        3) SIGQUIT      4) SIGILL
 5) SIGTRAP      6) SIGABRT       7) SIGBUS       8) SIGFPE
 9) SIGKILL     10) SIGUSR1      11) SIGSEGV     12) SIGUSR2
13) SIGPIPE     14) SIGALRM      15) SIGTERM     16) SIGSTKFLT
17) SIGCHLD     18) SIGCONT      19) SIGSTOP     20) SIGTSTP
21) SIGTTIN     22) SIGTTOU      23) SIGURG      24) SIGXCPU
25) SIGXFSZ     26) SIGVTALRM    27) SIGPROF     28) SIGWINCH
29) SIGIO       30) SIGPWR       31) SIGSYS      34) SIGRTMIN
35) SIGRTMIN+1  36) SIGRTMIN+2   37) SIGRTMIN+3  38) SIGRTMIN+4
39) SIGRTMIN+5  40) SIGRTMIN+6   41) SIGRTMIN+7  42) SIGRTMIN+8
43) SIGRTMIN+9  44) SIGRTMIN+10  45) SIGRTMIN+11 46) SIGRTMIN+12
47) SIGRTMIN+13 48) SIGRTMIN+14  49) SIGRTMIN+15 50) SIGRTMAX-14
51) SIGRTMAX-13 52) SIGRTMAX-12  53) SIGRTMAX-11 54) SIGRTMAX-10
55) SIGRTMAX-9  56) SIGRTMAX-8   57) SIGRTMAX-7  58) SIGRTMAX-6
59) SIGRTMAX-5  60) SIGRTMAX-4   61) SIGRTMAX-3  62) SIGRTMAX-2
63) SIGRTMAX-1  64) SIGRTMAX

Its name might already inform you about its usual task: killing processes. By default, if you want to terminate a process but you can't communicate with the process directly (like hitting "Quit" or "Exit"), you should send a signal 15 (SIGTERM) to the program. This is also what kill does by default.

However, if the process doesn't listen to this signal or has gone haywire, you can use the SIGKILL signal. The SIGKILL signal doesn't actually reach the application (ever) but immediately terminates the process. Its number is 9:

$ kill -9 219

Exercises

  1. How do you obtain the process ID of a running process?

  2. How can you run a process in background and still be able to terminate the session without terminating the process (without the process being a daemon)?

  3. What is a <defunct> process?

Further Resources