Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit d6a2f903 authored by David Carrillo-Cisneros's avatar David Carrillo-Cisneros Committed by Ingo Molnar
Browse files

perf/core: Introduce PMU_EV_CAP_READ_ACTIVE_PKG 



Introduce the flag PMU_EV_CAP_READ_ACTIVE_PKG, useful for uncore events,
that allows a PMU to signal the generic perf code that an event is readable
in the current CPU if the event is active in a CPU in the same package as
the current CPU.

This is an optimization that avoids a unnecessary IPI for the common case
where uncore events are run and read in the same package but in
different CPUs.

As an example, the IPI removal speeds up perf_read() in my Haswell system
as follows:

  - For event UNC_C_LLC_LOOKUP: From 260 us to 31 us.
  - For event RAPL's power/energy-cores/: From to 255 us to 27 us.

For the optimization to work, all events in the group must have it
(similarly to PERF_EV_CAP_SOFTWARE).

Signed-off-by: default avatarDavid Carrillo-Cisneros <davidcc@google.com>
Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Carrillo-Cisneros <davidcc@google.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Kan Liang <kan.liang@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vegard Nossum <vegard.nossum@gmail.com>
Cc: Vince Weaver <vincent.weaver@maine.edu>
Link: http://lkml.kernel.org/r/1471467307-61171-4-git-send-email-davidcc@google.com


Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
parent 4ff6a8de

include/linux/perf_event.h

+3 −0
Original line number Original line Diff line number Diff line
@@ -514,8 +514,11 @@ typedef void (*perf_overflow_handler_t)(struct perf_event *, 
 * Event capabilities. For event_caps and groups caps.

 * Event capabilities. For event_caps and groups caps.

 *

 *

 * PERF_EV_CAP_SOFTWARE: Is a software event.

 * PERF_EV_CAP_SOFTWARE: Is a software event.

 * PERF_EV_CAP_READ_ACTIVE_PKG: A CPU event (or cgroup event) that can be read

 * from any CPU in the package where it is active.

 */

 */

#define PERF_EV_CAP_SOFTWARE		BIT(0)

#define PERF_EV_CAP_SOFTWARE		BIT(0)

#define PERF_EV_CAP_READ_ACTIVE_PKG	BIT(1)





#define SWEVENT_HLIST_BITS		8

#define SWEVENT_HLIST_BITS		8

#define SWEVENT_HLIST_SIZE		(1 << SWEVENT_HLIST_BITS)

#define SWEVENT_HLIST_SIZE		(1 << SWEVENT_HLIST_BITS)

kernel/events/core.c

+23 −2
Original line number Original line Diff line number Diff line
@@ -3424,6 +3424,22 @@ struct perf_read_data { 
	int ret;

	int ret;

};

};





static int find_cpu_to_read(struct perf_event *event, int local_cpu)

{

	int event_cpu = event->oncpu;

	u16 local_pkg, event_pkg;



	if (event->group_caps & PERF_EV_CAP_READ_ACTIVE_PKG) {

		event_pkg =  topology_physical_package_id(event_cpu);

		local_pkg =  topology_physical_package_id(local_cpu);



		if (event_pkg == local_pkg)

			return local_cpu;

	}



	return event_cpu;

}



/*

/*

 * Cross CPU call to read the hardware event

 * Cross CPU call to read the hardware event

 */

 */
@@ -3545,7 +3561,7 @@ u64 perf_event_read_local(struct perf_event *event) 




static int perf_event_read(struct perf_event *event, bool group)

static int perf_event_read(struct perf_event *event, bool group)

{

{

	int ret = 0;

	int ret = 0, cpu_to_read, local_cpu;





	/*

	/*

	 * If event is enabled and currently active on a CPU, update the

	 * If event is enabled and currently active on a CPU, update the
@@ -3557,7 +3573,12 @@ static int perf_event_read(struct perf_event *event, bool group) 
			.group = group,

			.group = group,

			.ret = 0,

			.ret = 0,

		};

		};

		ret = smp_call_function_single(event->oncpu, __perf_event_read, &data, 1);



		local_cpu = get_cpu();

		cpu_to_read = find_cpu_to_read(event, local_cpu);

		put_cpu();



		ret = smp_call_function_single(cpu_to_read, __perf_event_read, &data, 1);

		/* The event must have been read from an online CPU: */

		/* The event must have been read from an online CPU: */

		WARN_ON_ONCE(ret);

		WARN_ON_ONCE(ret);

		ret = ret ? : data.ret;

		ret = ret ? : data.ret;