Linux kernel interfaces: Difference between revisions

Revision as of 10:42, 27 July 2019

Latest revision as of 17:36, 2 August 2025

(70 intermediate revisions by 38 users not shown)

Line 1:

{{Short description|Linux kernel APIs and ABIs}}

{{Multiple issues|section=|

{{essay-like|date=July 2014}}

{{Advert|date=March 2019}}

}}

[[File:Linux kernel interfaces.svg|thumb|upright=1.5|Linux API, Linux ABI, and in-kernel APIs and ABIs]]

The [[Linux kernel]] provides severalmultiple interfaces to user-[[User space applicationsand that arekernel used for different purposesspace|user-space and thatkernel-mode]] have different properties by designcode. The Thereinterfaces arecan twobe classified typesas ofeither [[application programming interface]] (API) in theor [[Linuxapplication kernelbinary interface]] that(ABI), areand notthey tocan be confused:classified theas either "kernel–user space" APIor and the "kernel -internal" API.

== {{Anchor|Linux API}}Linux API ==

[[File:Linux API.svg|thumb|upright=1.5|The '''Linux API''' is composed out of the system call interface of the Linux kernel, the [[GNU C Library]] (by [[GNU]]), [[cgroups|libcgroup]],<ref name="libcgroup">{{cite web|url=https://www.freedesktop.org/wiki/Software/systemd/ControlGroupInterface/|title=ControlGroupInterface|publisher=[[freedesktop.org]]}}</ref> [[Direct Rendering Manager|libdrm]], [[Advanced Linux Sound Architecture|libalsa]] and [[evdev|libevdev]]<ref name="libevdev">{{cite web |url=https://www.freedesktop.org/wiki/Software/libevdev/ |title=libevdev |publisher=[[freedesktop.org]]}}</ref> (by [[freedesktop.org]]).]]

The Linux API is the kernel&ndash;user space API, which allows programs in user space to access system resources and services of the Linux kernel. It is composed out of the System Call Interface of the Linux kernel and the subroutines in the GNU C Library (glibc). The focus of the development of the Linux API has been to provide the ''usable features'' of the specifications defined in POSIX in a way which is reasonably compatible, robust and performant, and to provide additional useful features not defined in POSIX, just as the kernel&ndash;user space APIs of other systems implementing the POSIX API also provide additional features not defined in POSIX.

[[File:Linux kernel API.svg|thumb|upright=1.5|Linux API vs. [[POSIX]] API]]

The Linux API includes the kernel&ndash;user space API, which allows code in user space to access system resources and services of the Linux kernel.<ref>{{cite web

The Linux API, by choice, has been kept stable over the decades and never breaks; this stability guarantees the portability of source code. At the same time, Linux kernel developers have historically been conservative and meticulous about introducing new system calls.

| url = https://www.linux.it/~rubini/docs/ksys/ksys.html

| title = Kernel System Calls

| date = 2006-11-02 | access-date = 2014-11-11

| author = Alessandro Rubini | website = linux.it

The Linux API is the kernel&ndash;user space API, which allows programs in user space to access system resources and services of the Linux kernel.}}</ref> It is composed out of the Systemsystem Callcall Interfaceinterface of the Linux kernel and the subroutines in the GNU [[C Librarystandard (glibc)library]]. The focus of the development of the Linux API has been to provide the ''usable features'' of the specifications defined in [[POSIX]] in a way which is reasonably compatible, robust and performant, and to provide additional useful features not defined in POSIX, just as the kernel&ndash;user space APIs of other systems implementing the POSIX API also provide additional features not defined in POSIX.

The Linux API, by choice, has been kept stable over the decades through a policy of not introducing breaking changes; this stability guarantees the portability of [[source code]].<ref>{{cite web

Much available free and open-source software is written for the POSIX API. Since so much more development flows into the Linux kernel as compared to the other POSIX-compliant combinations of kernel and C standard library, the Linux kernel and its API have been augmented with additional features. As far as these additional features provide a technical advantage, programming for the Linux API is preferred over the POSIX-API. Well-known current examples are udev, systemd and Weston. People such as Lennart Poettering openly advocate to prefer the Linux API over the POSIX API, where this offers advantages.

| url = https://lkml.org/lkml/2012/12/23/75

| title = Re: [Regression w/ patch] Media commit causes user space to misbahave (was: Re: Linux 3.8-rc1)

| date = 2012-12-23 | access-date = 2014-08-26

| author = Linus Torvalds | publisher = [[Linux kernel mailing list]]

| quote = If a change results in user programs breaking, it's a bug in the kernel. We never EVER blame the user programs.

The Linux API, by choice, has been kept stable over the decades and never breaks; this stability guarantees the portability of source code. }}</ref> At the same time, Linux kernel developers have historically been conservative and meticulous about introducing new system calls.{{citation needed|date=November 2014}}

Much available [[free and open-source software]] is written for the POSIX API. Since so much more development flows into the Linux kernel as compared to the other POSIX-compliant combinations of kernel and C standard library,{{citation needed|date=July 2014}} the Linux kernel and its API have been augmented with additional features. Programming for the full Linux API, rather than just the POSIX API, may provide advantages in cases where those additional features are useful. Well-known current examples are [[udev]], [[systemd]] and [[Weston (software)|Weston]].<ref>{{cite web |url=https://lwn.net/Articles/430598/ |title=Choosing between portability and innovation |publisher=[[LWN.net]] |date=2011-03-02}}</ref> People such as [[Lennart Poettering]] openly advocate to prefer the Linux API over the POSIX API, where this offers advantages.<ref name="fosdem1">{{cite web|url=https://archive.fosdem.org/2011/interview/lennart-poettering.html |publisher=fosdem.org |title=Interview: Lennart Poettering - Lennart Poettering will give a talk about "Systemd: beyond init" at FOSDEM 2011.|year=2011 |access-date=2014-06-16 |quote=''In fact, the way I see things the '''[[Linux kernel API|Linux API]]''' has been taking the role of the '''POSIX API''' and Linux is the focal point of all Free Software development. Due to that I can only recommend developers to try to hack with only Linux in mind and experience the freedom and the opportunities this offers you. So, get yourself a copy of [[The Linux Programming Interface]], ignore everything it says about [[POSIX]] compatibility and hack away your amazing Linux software. It's quite relieving!''}}</ref>

At FOSDEM 2016, Michael Kerrisk explained some of the perceived issues with the Linux kernel's user-space API, describing that it contains multiple design errors by being non-extensible, unmaintainable, overly complex, of limited purpose, in violation of standards, and inconsistent. Most of those mistakes cannot be fixed because doing so would break the ABI that the kernel presents to the user space.

At [[FOSDEM]] 2016, [[Michael Kerrisk]] explained some of the perceived issues with the Linux kernel's user-space API, describing that it contains multiple design errors by being non-extensible, unmaintainable, overly complex, of limited purpose, in violation of standards, and inconsistent. Most of those mistakes cannot be fixed because doing so would break the ABI that the kernel presents to the user space.<ref>{{cite web|url=https://archive.fosdem.org/2016/schedule/event/design_linux_kernel_api/|title=How to design a Linux kernel API|author=Michael Kerrisk|author-link=Michael Kerrisk|date=2016-01-31|access-date=2016-02-04}}</ref>

=== System Call Interface of the Linux kernel===

''System Call Interface'' is the denomination for the entirety of all implemented and available system calls in a kernel. Various subsystems, such as e.g. the DRM define their own system calls and the entirety is called System Call Interface.

=== {{Anchor|System Call Interface|SCI}} System call interface of the Linux kernel===

Various issues with the organization of the Linux kernel system calls are being publicly discussed. Issues have been pointed out by Andy Lutomirski, Michael Kerrisk and others.

The ''Systemsystem Callcall Interfaceinterface'' isof thea denominationkernel foris the entiretyset of all implemented and available [[system calls]] in a kernel. VariousIn the Linux kernel, various subsystems, such as e.g. the [[Direct Rendering Manager]] (DRM), define their own system calls, andall theof entiretywhich isare calledpart Systemof the system Callcall Interfaceinterface.

Various issues with the organization of the Linux kernel system calls are being publicly discussed. Issues have been pointed out by Andy Lutomirski, [[Michael Kerrisk]] and others.<ref>{{cite web |url=https://www.linux-magazine.com/Issues/2014/165/Kernel-News |title=System Call Organization}}</ref><ref>{{cite web |url=https://lkml.org/lkml/2014/2/27/571 |title= Making a universal list of syscalls? |publisher=[[LKML]] |date=2014-02-27}}</ref><ref>{{cite web |url=https://lwn.net/Articles/585415/ |title=Flags as a system call API design pattern |publisher=[[LWN.net]] |date=2014-02-12}}</ref><ref>{{cite web |url=https://lwn.net/Articles/446528/ |title=On vsyscalls and the vDSO |publisher=[[LWN.net]] |date=2011-06-08}}</ref>

=== The C standard library ===

[[File:Linux kernel System Call Interface and glibc.svg|thumb|upright=1.5|The [[GNU C Library]] is a wrapper around the system calls of the Linux kernel; thesystem combination of the Linux kernel System Call Interface and glibc is what builds the Linuxcall APIinterface.]]

A [[C standard library]] for Linux includes wrappers around the system calls of the Linux kernel; the combination of the Linux kernel system call interface and a C standard library is what builds the Linux API. Some popular implementations of the C standard library are

* [[glibc]]

* [[uClibc]]

* [[klibc]]

* [[Newlib]]

* [[musl]]

* [[dietlibc]]

* [[Bionic (software)|libbionic]] and [[Hybris (software)|libhybris]]

Although the landscape is shifting, amongst these options, glibc remains the most popular implementation, to the point of many treating it as the default and the term equivalent to libc.

* GNU C Library (glibc)

* Embedded GLIBC

* uClibc

* klibc

* Newlib

* musl

* dietlibc

* libbionic and libhybris

==== Additions to POSIX ====

As in other [[Unix-like]] systems, additional capabilities of the Linux kernel exist that are not part of POSIX:

* [[cgroups]] subsystem, the system calls it introduces and libcgroup<ref name="libcgroup" />

* The system calls of the [[Direct Rendering Manager]], especially the driver-private ioctls for the command submission, are ''not'' part of the POSIX specifications.

* [[Advanced Linux Sound Architecture]] ''could'' set system calls, which are not part of the POSIX specifications

* The system calls <code>[[futex]]</code> (fast userspace mutex), <code>[[epoll]]</code>, <code>[[splice (system call)|splice]]</code>, <code>[[dnotify]]</code>, <code>[[fanotify]]</code>, and <code>[[inotify]]</code> have been exclusive to the Linux kernel so far.

* The system call <code>[[getrandom]]</code> was introduced in version 3.17 of the [[Linux kernel mainline]]<ref>{{cite web |url=https://lkml.org/lkml/2014/7/17/145 |title=[PATCH, RFC] random: introduce getrandom(2) system call |publisher=[[Linux kernel mailing list|LKML]] |date=2014-07-17}}</ref>

* <code>[[memfd]]</code> was proposed by the [[kdbus]] developers<ref>{{cite web |url=https://github.com/gregkh/kdbus/blob/master/memfd.c |archive-url=https://archive.today/20140422172420/https://github.com/gregkh/kdbus/blob/master/memfd.c |url-status=dead |archive-date=2014-04-22 |title=memfd.c |website=[[GitHub]] }}</ref>

* <code>memfd</code> was proposed by the kdbus developers

** <code>[[memfd_create]]</code> was merged into the Linux kernel mainline in kernel version 3.17

* <code>[[readahead]]</code> initiates a file "read-ahead" into page cache

[[Direct Rendering Manager|DRM]] has been paramount for the development and implementations of well-defined and performant [[Free and open-source graphics device driver#Software architecture|free and open-source graphics device drivers]] without which no rendering acceleration would be available at all, or even worse, only the 2D drivers would be available in the [[X.Org Server]]. DRM was developed for Linux, and since has been ported to other operating systems as well.<ref>{{cite web |url=https://www.phoronix.com/scan.php?page=news_item&px=MTYzNTQ |title=NetBSD 7.0 Will Finally Have DRM/KMS Drivers |publisher=[[Phoronix]] |date=2014-03-19}}</ref>

=== Further libraries ===

* libdrm (for [[Direct Rendering Manager]])

* libnl (The libnl suite is a collection of libraries providing APIs to netlink protocol based Linux kernel interfaces.)

* libevdev (for [[evdev]])

* libasound ([[Advanced Linux Sound Architecture]])

* …

== Linux ABI ==

{{Expert needed|free and open-source software|ex2=software|ex3=computing|reason=This section mostly ignores the kernel-userspace ABI (that is very real and important thing) and jumps into userspace-to-userspace APIs|date=February 2018}}

The term Linux ABI refers to a kernel–user space ABI. The Application binary interface refers to the compiled binaries, in machine code. Any such ABI is therefore bound to the instruction set. Defining a useful ABI and keeping it stable is less the responsibility of the Linux kernel developers or of the developers of the GNU C Library, and more the task for Linux distributions and Independent software vendor (ISVs) who wish to sell and provide support for their proprietary software as binaries only for such a single Linux ABI, as opposed to supporting multiple Linux ABIs.

[[File:Linux API and Linux ABI.svg|thumb|upright=1.5|The Linux API and ABI ]]

{{Main article|x32 ABI|Linux Standard Base}}

The term Linux ABI refers tois a kernel–user space ABI. TheAs ApplicationABI binaryis interfacea refers[[machine tocode]] the compiled binariesinterface, inthe machine code. Any suchLinux ABI is therefore bound to the [[instruction set]]. Defining a useful ABI and keeping it stable is less the responsibility of the Linux kernel developers or of the developers of the GNU C Library, and more the task for [[Linux distributionsdistribution]]s and Independent[[independent software vendor]]s (ISVs) who wish to sell and provide support for their proprietary software as binaries only for such a single Linux ABI, as opposed to supporting multiple Linux ABIs.

An ABI has to be defined for every instruction set, such as x86, x86-64, MIPS, ARMv7-A (32-Bit), ARMv8-A (64-Bit), etc. with the endianness, if both are supported.

An ABI has to be defined for every instruction set, such as [[x86]], [[x86-64]], [[MIPS architecture|MIPS]], [[ARMv7-A]] (32-Bit), [[ARMv8-A]] (64-Bit), etc. with the [[endianness]], if both are supported.

It should be able to compile the software with different compilers against the definitions specified in the ABI and achieve full binary compatibility. Compilers that are free and open-source software are e.g. GNU Compiler Collection, LLVM/Clang.

It should be able to compile the software with different compilers against the definitions specified in the ABI and achieve full binary compatibility. Compilers that are [[free and open-source software]] are e.g. [[GNU Compiler Collection]], [[LLVM]]/[[Clang]].

End-users are in fact not all interested in the Linux API (or the Windows API), but in the ABIs.

== In-kernel APIs ==

There are a lot ofMany kernel-internal APIs forexist, allallowing thekernel subsystems to interface with one another. These are being kept fairly stable, but there is no guarantee for stability. InA casekernel-internal newAPI researchcan orbe insightschanged makewhen such a changeneed seemis favorable,indicated anby APInew isresearch changed,or insights; all necessary rewritemodifications and testing have to be done by the author.

The Linux kernel is a monolithic kernel, hence device drivers are kernel components. To ease the burden of companies maintaining their (proprietary) device drivers out-outside of- the main kernel tree, stable APIs for the device drivers have been repeatedly requested. The Linux kernel developers have repeatedly denied guaranteeing stable in-kernel APIs for device drivers. Guaranteeing such would have faltered the development of the Linux kernel in the past and would still in the future and, due to the nature of free and open-source software, are not necessary. Ergo, by choice, the Linux kernel has no ''stable'' in-kernel API.<ref>{{cite web|url=https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/stable-api-nonsense.rst |title=The Linux Kernel Driver Interface}}</ref>

== In-kernel ABIs ==

Since there are no stable in-kernel APIs, there cannot be stable in-kernel ABIs.<ref>{{cite web|url=httphttps://abi-laboratory.pro/tracker/timeline/linux/ | title=Analysis of ABI changes in the Linux kernel |publisher=Andrey Ponomarenko's ABI laboratory|date=2016-03-15}}</ref>

== Abstraction APIs ==

[[File:Linux kernel and OpenGL video games.svg|thumb|OpenGL is indeed an abstraction API to make use of diverse GPUs of multiple vendors without the need to program for each one specifically.]]

[[File:Division of labor cpu and gpu.svg|thumb|But the implementation of the OpenGL-specification is executed on the CPU in the context of the running operating system. One design goal of [[Vulkan (API)|Vulkan]] was to make the "graphics driver", i.e. the implementation of the graphics API, do less.]]

For severalmany use cases, the Linux API is considered too low-level, andso APIs of higher abstraction APIsmust arebe used. SuchHigher-level ofAPIs coursemust stillbe need to workimplemeted on top of the lowlower-level Linux APIs. Examples:

* implementationImplementation of the [[OpenGL]] and [[Vulkan (API)|Vulkan]] specifications in proprietary Linux graphics drivers and the free and open-source implementation in [[Mesa (computer graphics)|Mesa]].

* implementationImplementation of the [[OpenAL]] specification.

* [[Simple DirectMedia Layer]]: abstraction API for input/sound/etc. available for many operating systems.

* [[Simple and Fast Multimedia Library]]: like above.

== See also ==

{{Portal|Linux}}

* {{Annotated link|File descriptor}}

* ''[[The Linux Programming Interface]]'' by [[Michael Kerrisk]]

* {{Annotated link|Hybris (software)}}

* [[Semaphore (programming)]]

* ''[[{{Annotated link|The Linux Programming Interface]]}}'' by [[Michael Kerrisk]]

* [[system call]]{{snd}} is a function to facilitate programs to request services from the kernel

* {{Annotated link|netlink}}

** <tt>[[File descriptor|eventfd()]]</tt>

* [[{{Annotated link|Semaphore (programming)]]}}

** [[netlink]]{{snd}} socket family used for IPC between kernel and user space processes, designed as the successor of <tt>[[ioctl]]</tt>; Netlink was added by [[Alan Cox]] during Linux kernel 1.3 development as a character driver interface to provide multiple kernel and user-space bidirectional communications links. Then, Alexey Kuznetsov extended it during Linux kernel 2.1 development to provide a flexible and extensible messaging interface to the new advanced routing infrastructure. Since then, Netlink sockets have become one of the main interfaces that kernel subsystems provide to user-space applications in Linux. Modern [[Wireless network interface controller|WNIC]] [[Comparison of open-source wireless drivers|drivers]] use it to communicate with user-space.

* {{Annotated link|system call}}

* [[Windows API]]{{snd}} article on various API available on Microsoft Windows operating systems

* {{Annotated link|Windows API}}

** [[windows.h]]{{snd}} header file for the [[C (programming language)|C programming language]] which contains declarations for ''all'' of the functions in the Windows API

* {{Annotated link|windows.h}}

* [[Wine (software)|Wine]]{{snd}} a compatibility layer between Linux and programs written for Microsoft Windows

* {{Annotated link|Wine (software)}}

* [[Hybris (software)|libhybris]] – compatibility layer between Linux and programs written for Android

== References ==

Line 101 ⟶ 117:

*[https://web.archive.org/web/20070227215533/http://www.gnugeneration.com/books/linux/2.6.20/kernel-api/ The API of Linux kernel 2.6.20] and [https://www.kernel.org/doc/htmldocs/kernel-api/ 4.12] (in deprecated htmldocs format)

*[https://abi-laboratory.pro/?view=timeline&l=linux API/ABI changes review for Linux]

* [httphttps://man7.org/tlpi/ The Linux Programming Interface] book, [httphttps://man7.org/tlpi/api_changes/ Linux and ''glibc'' API changes] since [[The Linux Programming Interface]] was released in 2010

*{{usurped|1=[https://archive.today/20121220032539/http://www.makelinux.net/kernel_map/ Interactive Linux kernel map]}} with main API functions and structures, {{usurped|1=[https://web.archive.org/web/20180827040646/http://www.makelinux.net/kernel_map/LKM.pdf PDF]}} version

* {{usurped|1=[https://archive.today/20130221193824/http://www.makelinux.net/ldd3 Linux Device Drivers]}} by Jonathan Corbet, Greg Kroah-Hartman and Alessandro Rubini, 3rd edition

* [http://isis.poly.edu/kulesh/stuff/src/klist/ Linux Kernel Linked List Explained] {{Webarchive|url=https://web.archive.org/web/20090925130610/http://isis.poly.edu/kulesh/stuff/src/klist/ |date=2009-09-25 }}

{{Linux kernel}}