Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 32fad281 authored by Paul Mackerras's avatar Paul Mackerras Committed by Alexander Graf
Browse files

KVM: PPC: Book3S HV: Make the guest hash table size configurable



This adds a new ioctl to enable userspace to control the size of the guest
hashed page table (HPT) and to clear it out when resetting the guest.
The KVM_PPC_ALLOCATE_HTAB ioctl is a VM ioctl and takes as its parameter
a pointer to a u32 containing the desired order of the HPT (log base 2
of the size in bytes), which is updated on successful return to the
actual order of the HPT which was allocated.

There must be no vcpus running at the time of this ioctl.  To enforce
this, we now keep a count of the number of vcpus running in
kvm->arch.vcpus_running.

If the ioctl is called when a HPT has already been allocated, we don't
reallocate the HPT but just clear it out.  We first clear the
kvm->arch.rma_setup_done flag, which has two effects: (a) since we hold
the kvm->lock mutex, it will prevent any vcpus from starting to run until
we're done, and (b) it means that the first vcpu to run after we're done
will re-establish the VRMA if necessary.

If userspace doesn't call this ioctl before running the first vcpu, the
kernel will allocate a default-sized HPT at that point.  We do it then
rather than when creating the VM, as the code did previously, so that
userspace has a chance to do the ioctl if it wants.

When allocating the HPT, we can allocate either from the kernel page
allocator, or from the preallocated pool.  If userspace is asking for
a different size from the preallocated HPTs, we first try to allocate
using the kernel page allocator.  Then we try to allocate from the
preallocated pool, and then if that fails, we try allocating decreasing
sizes from the kernel page allocator, down to the minimum size allowed
(256kB).  Note that the kernel page allocator limits allocations to
1 << CONFIG_FORCE_MAX_ZONEORDER pages, which by default corresponds to
16MB (on 64-bit powerpc, at least).

Signed-off-by: default avatarPaul Mackerras <paulus@samba.org>
[agraf: fix module compilation]
Signed-off-by: default avatarAlexander Graf <agraf@suse.de>
parent 2e1ae9c0
Loading
Loading
Loading
Loading
+36 −0
Original line number Diff line number Diff line
@@ -1930,6 +1930,42 @@ The "pte_enc" field provides a value that can OR'ed into the hash
PTE's RPN field (ie, it needs to be shifted left by 12 to OR it
into the hash PTE second double word).


4.75 KVM_PPC_ALLOCATE_HTAB

Capability: KVM_CAP_PPC_ALLOC_HTAB
Architectures: powerpc
Type: vm ioctl
Parameters: Pointer to u32 containing hash table order (in/out)
Returns: 0 on success, -1 on error

This requests the host kernel to allocate an MMU hash table for a
guest using the PAPR paravirtualization interface.  This only does
anything if the kernel is configured to use the Book 3S HV style of
virtualization.  Otherwise the capability doesn't exist and the ioctl
returns an ENOTTY error.  The rest of this description assumes Book 3S
HV.

There must be no vcpus running when this ioctl is called; if there
are, it will do nothing and return an EBUSY error.

The parameter is a pointer to a 32-bit unsigned integer variable
containing the order (log base 2) of the desired size of the hash
table, which must be between 18 and 46.  On successful return from the
ioctl, it will have been updated with the order of the hash table that
was allocated.

If no hash table has been allocated when any vcpu is asked to run
(with the KVM_RUN ioctl), the host kernel will allocate a
default-sized hash table (16 MB).

If this ioctl is called when a hash table has already been allocated,
the kernel will clear out the existing hash table (zero all HPTEs) and
return the hash table order in the parameter.  (If the guest is using
the virtualized real-mode area (VRMA) facility, the kernel will
re-create the VMRA HPTEs on the next KVM_RUN of any vcpu.)


5. The kvm_run structure
------------------------

+2 −5
Original line number Diff line number Diff line
@@ -36,11 +36,8 @@ static inline void svcpu_put(struct kvmppc_book3s_shadow_vcpu *svcpu)
#define SPAPR_TCE_SHIFT		12

#ifdef CONFIG_KVM_BOOK3S_64_HV
/* For now use fixed-size 16MB page table */
#define HPT_ORDER	24
#define HPT_NPTEG	(1ul << (HPT_ORDER - 7))	/* 128B per pteg */
#define HPT_NPTE	(HPT_NPTEG << 3)		/* 8 PTEs per PTEG */
#define HPT_HASH_MASK	(HPT_NPTEG - 1)
#define KVM_DEFAULT_HPT_ORDER	24	/* 16MB HPT by default */
extern int kvm_hpt_order;		/* order of preallocated HPTs */
#endif

#define VRMA_VSID	0x1ffffffUL	/* 1TB VSID reserved for VRMA */
+4 −0
Original line number Diff line number Diff line
@@ -237,6 +237,10 @@ struct kvm_arch {
	unsigned long vrma_slb_v;
	int rma_setup_done;
	int using_mmu_notifiers;
	u32 hpt_order;
	atomic_t vcpus_running;
	unsigned long hpt_npte;
	unsigned long hpt_mask;
	spinlock_t slot_phys_lock;
	unsigned long *slot_phys[KVM_MEM_SLOTS_NUM];
	int slot_npages[KVM_MEM_SLOTS_NUM];
+2 −1
Original line number Diff line number Diff line
@@ -119,7 +119,8 @@ extern void kvmppc_core_destroy_mmu(struct kvm_vcpu *vcpu);
extern int kvmppc_kvm_pv(struct kvm_vcpu *vcpu);
extern void kvmppc_map_magic(struct kvm_vcpu *vcpu);

extern long kvmppc_alloc_hpt(struct kvm *kvm);
extern long kvmppc_alloc_hpt(struct kvm *kvm, u32 *htab_orderp);
extern long kvmppc_alloc_reset_hpt(struct kvm *kvm, u32 *htab_orderp);
extern void kvmppc_free_hpt(struct kvm *kvm);
extern long kvmppc_prepare_vrma(struct kvm *kvm,
				struct kvm_userspace_memory_region *mem);
+95 −28
Original line number Diff line number Diff line
@@ -37,56 +37,121 @@
/* POWER7 has 10-bit LPIDs, PPC970 has 6-bit LPIDs */
#define MAX_LPID_970	63

long kvmppc_alloc_hpt(struct kvm *kvm)
/* Power architecture requires HPT is at least 256kB */
#define PPC_MIN_HPT_ORDER	18

long kvmppc_alloc_hpt(struct kvm *kvm, u32 *htab_orderp)
{
	unsigned long hpt;
	long lpid;
	struct revmap_entry *rev;
	struct kvmppc_linear_info *li;
	long order = kvm_hpt_order;

	if (htab_orderp) {
		order = *htab_orderp;
		if (order < PPC_MIN_HPT_ORDER)
			order = PPC_MIN_HPT_ORDER;
	}

	/* Allocate guest's hashed page table */
	/*
	 * If the user wants a different size from default,
	 * try first to allocate it from the kernel page allocator.
	 */
	hpt = 0;
	if (order != kvm_hpt_order) {
		hpt = __get_free_pages(GFP_KERNEL|__GFP_ZERO|__GFP_REPEAT|
				       __GFP_NOWARN, order - PAGE_SHIFT);
		if (!hpt)
			--order;
	}

	/* Next try to allocate from the preallocated pool */
	if (!hpt) {
		li = kvm_alloc_hpt();
		if (li) {
		/* using preallocated memory */
			hpt = (ulong)li->base_virt;
			kvm->arch.hpt_li = li;
	} else {
		/* using dynamic memory */
			order = kvm_hpt_order;
		}
	}

	/* Lastly try successively smaller sizes from the page allocator */
	while (!hpt && order > PPC_MIN_HPT_ORDER) {
		hpt = __get_free_pages(GFP_KERNEL|__GFP_ZERO|__GFP_REPEAT|
				       __GFP_NOWARN, HPT_ORDER - PAGE_SHIFT);
				       __GFP_NOWARN, order - PAGE_SHIFT);
		if (!hpt)
			--order;
	}

	if (!hpt) {
		pr_err("kvm_alloc_hpt: Couldn't alloc HPT\n");
	if (!hpt)
		return -ENOMEM;
	}

	kvm->arch.hpt_virt = hpt;
	kvm->arch.hpt_order = order;
	/* HPTEs are 2**4 bytes long */
	kvm->arch.hpt_npte = 1ul << (order - 4);
	/* 128 (2**7) bytes in each HPTEG */
	kvm->arch.hpt_mask = (1ul << (order - 7)) - 1;

	/* Allocate reverse map array */
	rev = vmalloc(sizeof(struct revmap_entry) * HPT_NPTE);
	rev = vmalloc(sizeof(struct revmap_entry) * kvm->arch.hpt_npte);
	if (!rev) {
		pr_err("kvmppc_alloc_hpt: Couldn't alloc reverse map array\n");
		goto out_freehpt;
	}
	kvm->arch.revmap = rev;
	kvm->arch.sdr1 = __pa(hpt) | (order - 18);

	lpid = kvmppc_alloc_lpid();
	if (lpid < 0)
		goto out_freeboth;

	kvm->arch.sdr1 = __pa(hpt) | (HPT_ORDER - 18);
	kvm->arch.lpid = lpid;
	pr_info("KVM guest htab at %lx (order %ld), LPID %x\n",
		hpt, order, kvm->arch.lpid);

	pr_info("KVM guest htab at %lx, LPID %lx\n", hpt, lpid);
	if (htab_orderp)
		*htab_orderp = order;
	return 0;

 out_freeboth:
	vfree(rev);
 out_freehpt:
	free_pages(hpt, HPT_ORDER - PAGE_SHIFT);
	if (kvm->arch.hpt_li)
		kvm_release_hpt(kvm->arch.hpt_li);
	else
		free_pages(hpt, order - PAGE_SHIFT);
	return -ENOMEM;
}

long kvmppc_alloc_reset_hpt(struct kvm *kvm, u32 *htab_orderp)
{
	long err = -EBUSY;
	long order;

	mutex_lock(&kvm->lock);
	if (kvm->arch.rma_setup_done) {
		kvm->arch.rma_setup_done = 0;
		/* order rma_setup_done vs. vcpus_running */
		smp_mb();
		if (atomic_read(&kvm->arch.vcpus_running)) {
			kvm->arch.rma_setup_done = 1;
			goto out;
		}
	}
	if (kvm->arch.hpt_virt) {
		order = kvm->arch.hpt_order;
		/* Set the entire HPT to 0, i.e. invalid HPTEs */
		memset((void *)kvm->arch.hpt_virt, 0, 1ul << order);
		/*
		 * Set the whole last_vcpu array to an invalid vcpu number.
		 * This ensures that each vcpu will flush its TLB on next entry.
		 */
		memset(kvm->arch.last_vcpu, 0xff, sizeof(kvm->arch.last_vcpu));
		*htab_orderp = order;
		err = 0;
	} else {
		err = kvmppc_alloc_hpt(kvm, htab_orderp);
		order = *htab_orderp;
	}
 out:
	mutex_unlock(&kvm->lock);
	return err;
}

void kvmppc_free_hpt(struct kvm *kvm)
{
	kvmppc_free_lpid(kvm->arch.lpid);
@@ -94,7 +159,8 @@ void kvmppc_free_hpt(struct kvm *kvm)
	if (kvm->arch.hpt_li)
		kvm_release_hpt(kvm->arch.hpt_li);
	else
		free_pages(kvm->arch.hpt_virt, HPT_ORDER - PAGE_SHIFT);
		free_pages(kvm->arch.hpt_virt,
			   kvm->arch.hpt_order - PAGE_SHIFT);
}

/* Bits in first HPTE dword for pagesize 4k, 64k or 16M */
@@ -119,6 +185,7 @@ void kvmppc_map_vrma(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
	unsigned long psize;
	unsigned long hp0, hp1;
	long ret;
	struct kvm *kvm = vcpu->kvm;

	psize = 1ul << porder;
	npages = memslot->npages >> (porder - PAGE_SHIFT);
@@ -127,8 +194,8 @@ void kvmppc_map_vrma(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
	if (npages > 1ul << (40 - porder))
		npages = 1ul << (40 - porder);
	/* Can't use more than 1 HPTE per HPTEG */
	if (npages > HPT_NPTEG)
		npages = HPT_NPTEG;
	if (npages > kvm->arch.hpt_mask + 1)
		npages = kvm->arch.hpt_mask + 1;

	hp0 = HPTE_V_1TB_SEG | (VRMA_VSID << (40 - 16)) |
		HPTE_V_BOLTED | hpte0_pgsize_encoding(psize);
@@ -138,7 +205,7 @@ void kvmppc_map_vrma(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
	for (i = 0; i < npages; ++i) {
		addr = i << porder;
		/* can't use hpt_hash since va > 64 bits */
		hash = (i ^ (VRMA_VSID ^ (VRMA_VSID << 25))) & HPT_HASH_MASK;
		hash = (i ^ (VRMA_VSID ^ (VRMA_VSID << 25))) & kvm->arch.hpt_mask;
		/*
		 * We assume that the hash table is empty and no
		 * vcpus are using it at this stage.  Since we create
Loading