The library functions in the standard C library such as open, read, etc. are NOT system calls. Those are normal functions which cannot change the mode of execution of the system.

These functions invoke a special instruction which makes the system change its execution mode to kernel mode and start executing the system call code. The instruction is termed as an operating system trap.

The system calls are a special case of interrupt handling. The library routines pass a unique number for each system call, as the parameter to the operating system trap through a specific register or on the stack. Using that number, the kernel takes a decision of which system call to execute.

The algorithm for executing system calls is given below:

/* Algorithm: syscall
* Input: system call number
* Output: system call result
*/
{
    determine entry in the system call table related to the system call number;
    decides how many parameters should be given to the system call;
    copy parameters from the user address space to u-area;
    store current context for abortive return; // studied later
    invoke system call code in kernel;
    if (error during execution of system call)
    {
        set register 0 in user saved register context to error number;
        turn on carry bit in PS register in user saved register context;
    }
    else
        set register 0, 1 in user saved register context to return values from system call;
}

Register 0 and 1 are used to exchange information between user mode and kernel mode. As being machine independent, the terms register 0 and 1 are used.

The kernel calculates the address of the parameters according to operations (adding or subtracting, according to the direction of growth of the stack) an offset to the user stack pointer, related to the number of parameters to the system call.

The setting of carry bit shows that there was an error in the system call execution. After execution of this algorithm, the library function traces the return value from registers 0 and 1 and reverts it to the user.