Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 2b188cc1 authored by Jens Axboe's avatar Jens Axboe
Browse files

Add io_uring IO interface

The submission queue (SQ) and completion queue (CQ) rings are shared
between the application and the kernel. This eliminates the need to
copy data back and forth to submit and complete IO.

IO submissions use the io_uring_sqe data structure, and completions
are generated in the form of io_uring_cqe data structures. The SQ
ring is an index into the io_uring_sqe array, which makes it possible
to submit a batch of IOs without them being contiguous in the ring.
The CQ ring is always contiguous, as completion events are inherently
unordered, and hence any io_uring_cqe entry can point back to an
arbitrary submission.

Two new system calls are added for this:

io_uring_setup(entries, params)
	Sets up an io_uring instance for doing async IO. On success,
	returns a file descriptor that the application can mmap to
	gain access to the SQ ring, CQ ring, and io_uring_sqes.

io_uring_enter(fd, to_submit, min_complete, flags, sigset, sigsetsize)
	Initiates IO against the rings mapped to this fd, or waits for
	them to complete, or both. The behavior is controlled by the
	parameters passed in. If 'to_submit' is non-zero, then we'll
	try and submit new IO. If IORING_ENTER_GETEVENTS is set, the
	kernel will wait for 'min_complete' events, if they aren't
	already available. It's valid to set IORING_ENTER_GETEVENTS
	and 'min_complete' == 0 at the same time, this allows the
	kernel to return already completed events without waiting
	for them. This is useful only for polling, as for IRQ
	driven IO, the application can just check the CQ ring
	without entering the kernel.

With this setup, it's possible to do async IO with a single system
call. Future developments will enable polled IO with this interface,
and polled submission as well. The latter will enable an application
to do IO without doing ANY system calls at all.

For IRQ driven IO, an application only needs to enter the kernel for
completions if it wants to wait for them to occur.

Each io_uring is backed by a workqueue, to support buffered async IO
as well. We will only punt to an async context if the command would
need to wait for IO on the device side. Any data that can be accessed
directly in the page cache is done inline. This avoids the slowness
issue of usual threadpools, since cached data is accessed as quickly
as a sync interface.

Sample application: http://git.kernel.dk/cgit/fio/plain/t/io_uring.c



Reviewed-by: default avatarHannes Reinecke <hare@suse.com>
Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
parent 594b9a89
Loading
Loading
Loading
Loading
+2 −0
Original line number Diff line number Diff line
@@ -398,3 +398,5 @@
384	i386	arch_prctl		sys_arch_prctl			__ia32_compat_sys_arch_prctl
385	i386	io_pgetevents		sys_io_pgetevents		__ia32_compat_sys_io_pgetevents
386	i386	rseq			sys_rseq			__ia32_sys_rseq
425	i386	io_uring_setup		sys_io_uring_setup		__ia32_sys_io_uring_setup
426	i386	io_uring_enter		sys_io_uring_enter		__ia32_sys_io_uring_enter
+2 −0
Original line number Diff line number Diff line
@@ -343,6 +343,8 @@
332	common	statx			__x64_sys_statx
333	common	io_pgetevents		__x64_sys_io_pgetevents
334	common	rseq			__x64_sys_rseq
425	common	io_uring_setup		__x64_sys_io_uring_setup
426	common	io_uring_enter		__x64_sys_io_uring_enter

#
# x32-specific system call numbers start at 512 to avoid cache impact
+1 −0
Original line number Diff line number Diff line
@@ -30,6 +30,7 @@ obj-$(CONFIG_TIMERFD) += timerfd.o
obj-$(CONFIG_EVENTFD)		+= eventfd.o
obj-$(CONFIG_USERFAULTFD)	+= userfaultfd.o
obj-$(CONFIG_AIO)               += aio.o
obj-$(CONFIG_IO_URING)		+= io_uring.o
obj-$(CONFIG_FS_DAX)		+= dax.o
obj-$(CONFIG_FS_ENCRYPTION)	+= crypto/
obj-$(CONFIG_FILE_LOCKING)      += locks.o

fs/io_uring.c

0 → 100644
+1255 −0

File added.

Preview size limit exceeded, changes collapsed.

+9 −0
Original line number Diff line number Diff line
@@ -3517,4 +3517,13 @@ extern void inode_nohighmem(struct inode *inode);
extern int vfs_fadvise(struct file *file, loff_t offset, loff_t len,
		       int advice);

#if defined(CONFIG_IO_URING)
extern struct sock *io_uring_get_socket(struct file *file);
#else
static inline struct sock *io_uring_get_socket(struct file *file)
{
	return NULL;
}
#endif

#endif /* _LINUX_FS_H */
Loading