[SOLVED] Linux kernel function returns 1 instead of EINVAL

Issue

I'm trying to add a new system call to linux kernel:

asmlinkage long sys_set_status(int status) {
    if ((status != 0) && (status != 1))
        return -EINVAL; //-22
    current->status = status;
    return 0;
}

in syscall_64.tbl it is declared:

334 common  set_status      sys_set_status

in syscalls.h it is declared:

asmlinkage long sys_set_status(int status);

but when i test the return value:

int set_status(int status) {
    long r = syscall(334, status);
    return (int)r;
}

int main() {
    int res = set_status(-1); //illegal status, should return -EINVAL whish is -22
    cout << "return value is: " << res << endl;
    return 0;
}

i get:

return value is: -1

Solution

long r = syscall(334, status);

From man syscall:

The return value is defined by the system call being invoked. In general, a 0 return value indicates success. A -1 return value indicates an error, and an error number is stored in errno.

You are not calling the system call directly, you are calling it via the libc syscall wrapper, which performs approximately this:

int syscall(num, ...)
{
  /* architecture-specific code to put system call number and args into
     appropriate registers */
  /* arch-specific code to execute the system call instruction */
  int rc = /* arch-specific code to get the result of the system call */ ;
  if (rc < 0) { errno = -rc; return -1; }
  return 0;
}

If you don't want this translation to happen, you would have to perform the architecture-specific parts yourself (in assembly), and then you would have the actual system call return value.

Answered By - Employed Russian

Answer Checked By - Cary Denson (WPSolving Admin)