Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 2a31b9db authored by Paolo Bonzini's avatar Paolo Bonzini
Browse files

kvm: introduce manual dirty log reprotect 



There are two problems with KVM_GET_DIRTY_LOG.  First, and less important,
it can take kvm->mmu_lock for an extended period of time.  Second, its user
can actually see many false positives in some cases.  The latter is due
to a benign race like this:

  1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
     them.
  2. The guest modifies the pages, causing them to be marked ditry.
  3. Userspace actually copies the pages.
  4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
     they were not written to since (3).

This is especially a problem for large guests, where the time between
(1) and (3) can be substantial.  This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns.  Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page.  The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.

Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
parent 8fe65a82

Documentation/virtual/kvm/api.txt

+67 −0
Original line number Diff line number Diff line
@@ -305,6 +305,9 @@ the address space for which you want to return the dirty bitmap. 
They must be less than the value that KVM_CHECK_EXTENSION returns for

the KVM_CAP_MULTI_ADDRESS_SPACE capability.



The bits in the dirty bitmap are cleared before the ioctl returns, unless

KVM_CAP_MANUAL_DIRTY_LOG_PROTECT is enabled.  For more information,

see the description of the capability.



4.9 KVM_SET_MEMORY_ALIAS
@@ -3758,6 +3761,46 @@ Coalesced pio is based on coalesced mmio. There is little difference 
between coalesced mmio and pio except that coalesced pio records accesses

to I/O ports.



4.117 KVM_CLEAR_DIRTY_LOG (vm ioctl)



Capability: KVM_CAP_MANUAL_DIRTY_LOG_PROTECT

Architectures: x86

Type: vm ioctl

Parameters: struct kvm_dirty_log (in)

Returns: 0 on success, -1 on error



/* for KVM_CLEAR_DIRTY_LOG */

struct kvm_clear_dirty_log {

	__u32 slot;

	__u32 num_pages;

	__u64 first_page;

	union {

		void __user *dirty_bitmap; /* one bit per page */

		__u64 padding;

	};

};



The ioctl clears the dirty status of pages in a memory slot, according to

the bitmap that is passed in struct kvm_clear_dirty_log's dirty_bitmap

field.  Bit 0 of the bitmap corresponds to page "first_page" in the

memory slot, and num_pages is the size in bits of the input bitmap.

Both first_page and num_pages must be a multiple of 64.  For each bit

that is set in the input bitmap, the corresponding page is marked "clean"

in KVM's dirty bitmap, and dirty tracking is re-enabled for that page

(for example via write-protection, or by clearing the dirty bit in

a page table entry).



If KVM_CAP_MULTI_ADDRESS_SPACE is available, bits 16-31 specifies

the address space for which you want to return the dirty bitmap.

They must be less than the value that KVM_CHECK_EXTENSION returns for

the KVM_CAP_MULTI_ADDRESS_SPACE capability.



This ioctl is mostly useful when KVM_CAP_MANUAL_DIRTY_LOG_PROTECT

is enabled; for more information, see the description of the capability.

However, it can always be used as long as KVM_CHECK_EXTENSION confirms

that KVM_CAP_MANUAL_DIRTY_LOG_PROTECT is present.





5. The kvm_run structure

------------------------
@@ -4652,6 +4695,30 @@ and injected exceptions. 
* For the new DR6 bits, note that bit 16 is set iff the #DB exception

  will clear DR6.RTM.



7.18 KVM_CAP_MANUAL_DIRTY_LOG_PROTECT



Architectures: all

Parameters: args[0] whether feature should be enabled or not



With this capability enabled, KVM_GET_DIRTY_LOG will not automatically

clear and write-protect all pages that are returned as dirty.

Rather, userspace will have to do this operation separately using

KVM_CLEAR_DIRTY_LOG.



At the cost of a slightly more complicated operation, this provides better

scalability and responsiveness for two reasons.  First,

KVM_CLEAR_DIRTY_LOG ioctl can operate on a 64-page granularity rather

than requiring to sync a full memslot; this ensures that KVM does not

take spinlocks for an extended period of time.  Second, in some cases a

large amount of time can pass between a call to KVM_GET_DIRTY_LOG and

userspace actually using the data in the page.  Pages can be modified

during this time, which is inefficint for both the guest and userspace:

the guest will incur a higher penalty due to write protection faults,

while userspace can see false reports of dirty pages.  Manual reprotection

helps reducing this time, improving guest performance and reducing the

number of dirty log false positives.





8. Other capabilities.

----------------------

arch/mips/kvm/mips.c

+23 −0
Original line number Diff line number Diff line
@@ -1023,6 +1023,29 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log) 
	return r;

}



int kvm_vm_ioctl_clear_dirty_log(struct kvm *kvm, struct kvm_clear_dirty_log *log)

{

	struct kvm_memslots *slots;

	struct kvm_memory_slot *memslot;

	bool flush = false;

	int r;



	mutex_lock(&kvm->slots_lock);



	r = kvm_clear_dirty_log_protect(kvm, log, &flush);



	if (flush) {

		slots = kvm_memslots(kvm);

		memslot = id_to_memslot(slots, log->slot);



		/* Let implementation handle TLB/GVA invalidation */

		kvm_mips_callbacks->flush_shadow_memslot(kvm, memslot);

	}



	mutex_unlock(&kvm->slots_lock);

	return r;

}



long kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)

{

	long r;

arch/x86/kvm/x86.c

+27 −0
Original line number Diff line number Diff line
@@ -4418,6 +4418,33 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log) 
	return r;

}



int kvm_vm_ioctl_clear_dirty_log(struct kvm *kvm, struct kvm_clear_dirty_log *log)

{

	bool flush = false;

	int r;



	mutex_lock(&kvm->slots_lock);



	/*

	 * Flush potentially hardware-cached dirty pages to dirty_bitmap.

	 */

	if (kvm_x86_ops->flush_log_dirty)

		kvm_x86_ops->flush_log_dirty(kvm);



	r = kvm_clear_dirty_log_protect(kvm, log, &flush);



	/*

	 * All the TLBs can be flushed out of mmu lock, see the comments in

	 * kvm_mmu_slot_remove_write_access().

	 */

	lockdep_assert_held(&kvm->slots_lock);

	if (flush)

		kvm_flush_remote_tlbs(kvm);



	mutex_unlock(&kvm->slots_lock);

	return r;

}



int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_event,

			bool line_status)

{

include/linux/kvm_host.h

+5 −0
Original line number Diff line number Diff line
@@ -449,6 +449,7 @@ struct kvm { 
#endif

	long tlbs_dirty;

	struct list_head devices;

	bool manual_dirty_log_protect;

	struct dentry *debugfs_dentry;

	struct kvm_stat_data **debugfs_stat_data;

	struct srcu_struct srcu;
@@ -754,6 +755,8 @@ int kvm_get_dirty_log(struct kvm *kvm, 


int kvm_get_dirty_log_protect(struct kvm *kvm,

			      struct kvm_dirty_log *log, bool *flush);

int kvm_clear_dirty_log_protect(struct kvm *kvm,

				struct kvm_clear_dirty_log *log, bool *flush);



void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm,

					struct kvm_memory_slot *slot,
@@ -762,6 +765,8 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm, 


int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm,

				struct kvm_dirty_log *log);

int kvm_vm_ioctl_clear_dirty_log(struct kvm *kvm,

				  struct kvm_clear_dirty_log *log);



int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_level,

			bool line_status);

include/uapi/linux/kvm.h

+15 −0
Original line number Diff line number Diff line
@@ -492,6 +492,17 @@ struct kvm_dirty_log { 
	};

};



/* for KVM_CLEAR_DIRTY_LOG */

struct kvm_clear_dirty_log {

	__u32 slot;

	__u32 num_pages;

	__u64 first_page;

	union {

		void __user *dirty_bitmap; /* one bit per page */

		__u64 padding2;

	};

};



/* for KVM_SET_SIGNAL_MASK */

struct kvm_signal_mask {

	__u32 len;
@@ -975,6 +986,7 @@ struct kvm_ppc_resize_hpt { 
#define KVM_CAP_HYPERV_ENLIGHTENED_VMCS 163

#define KVM_CAP_EXCEPTION_PAYLOAD 164

#define KVM_CAP_ARM_VM_IPA_SIZE 165

#define KVM_CAP_MANUAL_DIRTY_LOG_PROTECT 166



#ifdef KVM_CAP_IRQ_ROUTING
@@ -1421,6 +1433,9 @@ struct kvm_enc_region { 
#define KVM_GET_NESTED_STATE         _IOWR(KVMIO, 0xbe, struct kvm_nested_state)

#define KVM_SET_NESTED_STATE         _IOW(KVMIO,  0xbf, struct kvm_nested_state)



/* Available with KVM_CAP_MANUAL_DIRTY_LOG_PROTECT */

#define KVM_CLEAR_DIRTY_LOG          _IOWR(KVMIO, 0xc0, struct kvm_clear_dirty_log)



/* Secure Encrypted Virtualization command */

enum sev_cmd_id {

	/* Guest initialization commands */