Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 747e94ae authored by Steven Rostedt's avatar Steven Rostedt Committed by Steven Rostedt
Browse files

ring-buffer: Make write slow path out of line 



Gcc inlines the slow path of the ring buffer write which can
hurt performance. This patch simply forces the slow path function
rb_move_tail() to always be a function.

The ring_buffer_benchmark module with reader_disabled=1 shows that
this patch changes the time to record an event from 135 ns to
132 ns. (3 ns or 2.22% improvement)

Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
parent 51ea8a88

kernel/trace/ring_buffer.c

+5 −2
Original line number Diff line number Diff line
@@ -1823,7 +1823,10 @@ rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, 
	local_sub(length, &tail_page->write);

}



static struct ring_buffer_event *

/*

 * This is the slow path, force gcc not to inline it.

 */

static noinline struct ring_buffer_event *

rb_move_tail(struct ring_buffer_per_cpu *cpu_buffer,

	     unsigned long length, unsigned long tail,

	     struct buffer_page *tail_page, u64 *ts)
@@ -1943,7 +1946,7 @@ __rb_reserve_next(struct ring_buffer_per_cpu *cpu_buffer, 
	tail = write - length;



	/* See if we shot pass the end of this buffer page */

	if (write > BUF_PAGE_SIZE)

	if (unlikely(write > BUF_PAGE_SIZE))

		return rb_move_tail(cpu_buffer, length, tail,

				    tail_page, ts);