Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 8ea4a5d8 authored by Linus Torvalds's avatar Linus Torvalds
Browse files

Merge tag 'for-linus-4.17-ofs' of git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux 

Pull orangefs updates from Mike Marshall:
 "Fixes and cleanups:

   - Documentation cleanups

   - removal of unused code

   - make some structs static

   - implement Orangefs vm_operations fault callout

   - eliminate two single-use functions and put their cleaned up code in
     line.

   - replace a vmalloc/memset instance with vzalloc

   - fix a race condition bug in wait code"

* tag 'for-linus-4.17-ofs' of git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux:
  Orangefs: documentation updates
  orangefs: document package install and xfstests procedure
  orangefs: remove unused code
  orangefs: make several *_operations structs static
  orangefs: implement vm_ops->fault
  orangefs: open code short single-use functions
  orangefs: replace vmalloc and memset with vzalloc
  orangefs: bug fix for a race condition when getting a slot
parents 190f2ace 8e9ba5c4

Documentation/filesystems/orangefs.txt

+112 −25
Original line number Diff line number Diff line
@@ -21,10 +21,16 @@ Orangefs features include: 
  * Stateless





MAILING LIST

============

MAILING LIST ARCHIVES

=====================



http://beowulf-underground.org/mailman/listinfo/pvfs2-users

http://lists.orangefs.org/pipermail/devel_lists.orangefs.org/





MAILING LIST SUBMISSIONS

========================



devel@lists.orangefs.org





DOCUMENTATION
@@ -42,12 +48,59 @@ Orangefs versions prior to 2.9.3 would not be compatible with the 
upstream version of the kernel client.





BUILDING THE USERSPACE FILESYSTEM ON A SINGLE SERVER

====================================================

RUNNING ORANGEFS ON A SINGLE SERVER

===================================



OrangeFS is usually run in large installations with multiple servers and

clients, but a complete filesystem can be run on a single machine for

development and testing.



On Fedora, install orangefs and orangefs-server.



dnf -y install orangefs orangefs-server



There is an example server configuration file in

/etc/orangefs/orangefs.conf.  Change localhost to your hostname if

necessary.



To generate a filesystem to run xfstests against, see below.



There is an example client configuration file in /etc/pvfs2tab.  It is a

single line.  Uncomment it and change the hostname if necessary.  This

controls clients which use libpvfs2.  This does not control the

pvfs2-client-core.



Create the filesystem.



pvfs2-server -f /etc/orangefs/orangefs.conf



Start the server.



systemctl start orangefs-server



Test the server.



pvfs2-ping -m /pvfsmnt



Start the client.  The module must be compiled in or loaded before this

point.



systemctl start orangefs-client



Mount the filesystem.



mount -t pvfs2 tcp://localhost:3334/orangefs /pvfsmnt





You can omit --prefix if you don't care that things are sprinkled around in

/usr/local. As of version 2.9.6, Orangefs uses Berkeley DB by default, we

will probably be changing the default to lmdb soon.

BUILDING ORANGEFS ON A SINGLE SERVER

====================================



Where OrangeFS cannot be installed from distribution packages, it may be

built from source.



You can omit --prefix if you don't care that things are sprinkled around

in /usr/local.  As of version 2.9.6, OrangeFS uses Berkeley DB by

default, we will probably be changing the default to LMDB soon.



./configure --prefix=/opt/ofs --with-db-backend=lmdb
@@ -55,35 +108,69 @@ make 


make install



Create an orangefs config file:

Create an orangefs config file.



/opt/ofs/bin/pvfs2-genconfig /etc/pvfs2.conf



  for "Enter hostnames", use the hostname, don't let it default to

  localhost.

Create an /etc/pvfs2tab file.



echo tcp://localhost:3334/orangefs /pvfsmnt pvfs2 defaults,noauto 0 0 > \

    /etc/pvfs2tab



Create the mount point you specified in the tab file if needed.



create a pvfs2tab file in /etc:

cat /etc/pvfs2tab

tcp://myhostname:3334/orangefs /mymountpoint pvfs2 defaults,noauto 0 0

mkdir /pvfsmnt



create the mount point you specified in the tab file if needed:

mkdir /mymountpoint

Bootstrap the server.



bootstrap the server:

/opt/ofs/sbin/pvfs2-server /etc/pvfs2.conf -f

/opt/ofs/sbin/pvfs2-server -f /etc/pvfs2.conf



Start the server.



start the server:

/opt/osf/sbin/pvfs2-server /etc/pvfs2.conf



Now the server is running. At this point you might like to

prove things are working with:

Now the server should be running. Pvfs2-ls is a simple

test to verify that the server is running.



/opt/ofs/bin/pvfs2-ls /pvfsmnt



/opt/osf/bin/pvfs2-ls /mymountpoint

If stuff seems to be working, load the kernel module and

turn on the client core.



If stuff seems to be working, turn on the client core:

/opt/osf/sbin/pvfs2-client -p /opt/osf/sbin/pvfs2-client-core

/opt/ofs/sbin/pvfs2-client -p /opt/osf/sbin/pvfs2-client-core



Mount your filesystem.

mount -t pvfs2 tcp://myhostname:3334/orangefs /mymountpoint



mount -t pvfs2 tcp://localhost:3334/orangefs /pvfsmnt





RUNNING XFSTESTS

================



It is useful to use a scratch filesystem with xfstests.  This can be

done with only one server.



Make a second copy of the FileSystem section in the server configuration

file, which is /etc/orangefs/orangefs.conf.  Change the Name to scratch.

Change the ID to something other than the ID of the first FileSystem

section (2 is usually a good choice).



Then there are two FileSystem sections: orangefs and scratch.



This change should be made before creating the filesystem.



pvfs2-server -f /etc/orangefs/orangefs.conf



To run xfstests, create /etc/xfsqa.config.



TEST_DIR=/orangefs

TEST_DEV=tcp://localhost:3334/orangefs

SCRATCH_MNT=/scratch

SCRATCH_DEV=tcp://localhost:3334/scratch



Then xfstests can be run



./check -pvfs2





OPTIONS

fs/orangefs/devorangefs-req.c

+27 −28
Original line number Diff line number Diff line
@@ -463,11 +463,10 @@ static ssize_t orangefs_devreq_write_iter(struct kiocb *iocb, 
	if (op->downcall.type != ORANGEFS_VFS_OP_READDIR)

		goto wakeup;



	op->downcall.trailer_buf = vmalloc(op->downcall.trailer_size);

	op->downcall.trailer_buf = vzalloc(op->downcall.trailer_size);

	if (!op->downcall.trailer_buf)

		goto Enomem;



	memset(op->downcall.trailer_buf, 0, op->downcall.trailer_size);

	if (!copy_from_iter_full(op->downcall.trailer_buf,

			         op->downcall.trailer_size, iter)) {

		gossip_err("%s: failed to copy trailer.\n", __func__);
@@ -779,9 +778,35 @@ static long orangefs_devreq_compat_ioctl(struct file *filp, unsigned int cmd, 


#endif /* CONFIG_COMPAT is in .config */



static __poll_t orangefs_devreq_poll(struct file *file,

				      struct poll_table_struct *poll_table)

{

	__poll_t poll_revent_mask = 0;



	poll_wait(file, &orangefs_request_list_waitq, poll_table);



	if (!list_empty(&orangefs_request_list))

		poll_revent_mask |= EPOLLIN;

	return poll_revent_mask;

}



/* the assigned character device major number */

static int orangefs_dev_major;



static const struct file_operations orangefs_devreq_file_operations = {

	.owner = THIS_MODULE,

	.read = orangefs_devreq_read,

	.write_iter = orangefs_devreq_write_iter,

	.open = orangefs_devreq_open,

	.release = orangefs_devreq_release,

	.unlocked_ioctl = orangefs_devreq_ioctl,



#ifdef CONFIG_COMPAT		/* CONFIG_COMPAT is in .config */

	.compat_ioctl = orangefs_devreq_compat_ioctl,

#endif

	.poll = orangefs_devreq_poll

};



/*

 * Initialize orangefs device specific state:

 * Must be called at module load time only
@@ -814,29 +839,3 @@ void orangefs_dev_cleanup(void) 
		     "*** /dev/%s character device unregistered ***\n",

		     ORANGEFS_REQDEVICE_NAME);

}
 


static __poll_t orangefs_devreq_poll(struct file *file,

				      struct poll_table_struct *poll_table)

{

	__poll_t poll_revent_mask = 0;



	poll_wait(file, &orangefs_request_list_waitq, poll_table);



	if (!list_empty(&orangefs_request_list))

		poll_revent_mask |= EPOLLIN;

	return poll_revent_mask;

}



const struct file_operations orangefs_devreq_file_operations = {

	.owner = THIS_MODULE,

	.read = orangefs_devreq_read,

	.write_iter = orangefs_devreq_write_iter,

	.open = orangefs_devreq_open,

	.release = orangefs_devreq_release,

	.unlocked_ioctl = orangefs_devreq_ioctl,



#ifdef CONFIG_COMPAT		/* CONFIG_COMPAT is in .config */

	.compat_ioctl = orangefs_devreq_compat_ioctl,

#endif

	.poll = orangefs_devreq_poll

};

fs/orangefs/file.c

+47 −78
Original line number Diff line number Diff line
@@ -41,70 +41,6 @@ static int flush_racache(struct inode *inode) 
	return ret;

}



/*

 * Copy to client-core's address space from the buffers specified

 * by the iovec upto total_size bytes.

 * NOTE: the iovector can either contain addresses which

 *       can futher be kernel-space or user-space addresses.

 *       or it can pointers to struct page's

 */

static int precopy_buffers(int buffer_index,

			   struct iov_iter *iter,

			   size_t total_size)

{

	int ret = 0;

	/*

	 * copy data from application/kernel by pulling it out

	 * of the iovec.

	 */





	if (total_size) {

		ret = orangefs_bufmap_copy_from_iovec(iter,

						      buffer_index,

						      total_size);

		if (ret < 0)

		gossip_err("%s: Failed to copy-in buffers. Please make sure that the pvfs2-client is running. %ld\n",

			   __func__,

			   (long)ret);

	}



	if (ret < 0)

		gossip_err("%s: Failed to copy-in buffers. Please make sure that the pvfs2-client is running. %ld\n",

			__func__,

			(long)ret);

	return ret;

}



/*

 * Copy from client-core's address space to the buffers specified

 * by the iovec upto total_size bytes.

 * NOTE: the iovector can either contain addresses which

 *       can futher be kernel-space or user-space addresses.

 *       or it can pointers to struct page's

 */

static int postcopy_buffers(int buffer_index,

			    struct iov_iter *iter,

			    size_t total_size)

{

	int ret = 0;

	/*

	 * copy data to application/kernel by pushing it out to

	 * the iovec. NOTE; target buffers can be addresses or

	 * struct page pointers.

	 */

	if (total_size) {

		ret = orangefs_bufmap_copy_to_iovec(iter,

						    buffer_index,

						    total_size);

		if (ret < 0)

			gossip_err("%s: Failed to copy-out buffers. Please make sure that the pvfs2-client is running (%ld)\n",

				__func__,

				(long)ret);

	}

	return ret;

}



/*

 * Post and wait for the I/O upcall to finish

 */
@@ -157,15 +93,16 @@ static ssize_t wait_for_direct_io(enum ORANGEFS_io_type type, struct inode *inod 
		     total_size);

	/*

	 * Stage 1: copy the buffers into client-core's address space

	 * precopy_buffers only pertains to writes.

	 */

	if (type == ORANGEFS_IO_WRITE) {

		ret = precopy_buffers(buffer_index,

				      iter,

	if (type == ORANGEFS_IO_WRITE && total_size) {

		ret = orangefs_bufmap_copy_from_iovec(iter, buffer_index,

		    total_size);

		if (ret < 0)

		if (ret < 0) {

			gossip_err("%s: Failed to copy-in buffers. Please make sure that the pvfs2-client is running. %ld\n",

			    __func__, (long)ret);

			goto out;

		}

	}



	gossip_debug(GOSSIP_FILE_DEBUG,

		     "%s(%pU): Calling post_io_request with tag (%llu)\n",
@@ -260,15 +197,21 @@ static ssize_t wait_for_direct_io(enum ORANGEFS_io_type type, struct inode *inod 


	/*

	 * Stage 3: Post copy buffers from client-core's address space

	 * postcopy_buffers only pertains to reads.

	 */

	if (type == ORANGEFS_IO_READ) {

		ret = postcopy_buffers(buffer_index,

				       iter,

	if (type == ORANGEFS_IO_READ && new_op->downcall.resp.io.amt_complete) {

		/*

		 * NOTE: the iovector can either contain addresses which

		 *       can futher be kernel-space or user-space addresses.

		 *       or it can pointers to struct page's

		 */

		ret = orangefs_bufmap_copy_to_iovec(iter, buffer_index,

		    new_op->downcall.resp.io.amt_complete);

		if (ret < 0)

		if (ret < 0) {

			gossip_err("%s: Failed to copy-out buffers. Please make sure that the pvfs2-client is running (%ld)\n",

			    __func__, (long)ret);

			goto out;

		}

	}

	gossip_debug(GOSSIP_FILE_DEBUG,

	    "%s(%pU): Amount %s, returned by the sys-io call:%d\n",

	    __func__,
@@ -585,6 +528,28 @@ static long orangefs_ioctl(struct file *file, unsigned int cmd, unsigned long ar 
	return ret;

}



static int orangefs_fault(struct vm_fault *vmf)

{

	struct file *file = vmf->vma->vm_file;

	int rc;

	rc = orangefs_inode_getattr(file->f_mapping->host, 0, 1,

	    STATX_SIZE);

	if (rc == -ESTALE)

		rc = -EIO;

	if (rc) {

		gossip_err("%s: orangefs_inode_getattr failed, "

		    "rc:%d:.\n", __func__, rc);

		return rc;

	}

	return filemap_fault(vmf);

}



const struct vm_operations_struct orangefs_file_vm_ops = {

	.fault = orangefs_fault,

	.map_pages = filemap_map_pages,

	.page_mkwrite = filemap_page_mkwrite,

};



/*

 * Memory map a region of a file.

 */
@@ -596,12 +561,16 @@ static int orangefs_file_mmap(struct file *file, struct vm_area_struct *vma) 
			(char *)file->f_path.dentry->d_name.name :

			(char *)"Unknown"));



	if ((vma->vm_flags & VM_SHARED) && (vma->vm_flags & VM_MAYWRITE))

		return -EINVAL;



	/* set the sequential readahead hint */

	vma->vm_flags |= VM_SEQ_READ;

	vma->vm_flags &= ~VM_RAND_READ;



	/* Use readonly mmap since we cannot support writable maps. */

	return generic_file_readonly_mmap(file, vma);

	file_accessed(file);

	vma->vm_ops = &orangefs_file_vm_ops;

	return 0;

}



#define mapping_nrpages(idata) ((idata)->nrpages)

fs/orangefs/inode.c

+2 −2
Original line number Diff line number Diff line
@@ -138,7 +138,7 @@ static ssize_t orangefs_direct_IO(struct kiocb *iocb, 
}



/** ORANGEFS2 implementation of address space operations */

const struct address_space_operations orangefs_address_operations = {

static const struct address_space_operations orangefs_address_operations = {

	.readpage = orangefs_readpage,

	.readpages = orangefs_readpages,

	.invalidatepage = orangefs_invalidatepage,
@@ -307,7 +307,7 @@ int orangefs_update_time(struct inode *inode, struct timespec *time, int flags) 
}



/* ORANGEDS2 implementation of VFS inode operations for files */

const struct inode_operations orangefs_file_inode_operations = {

static const struct inode_operations orangefs_file_inode_operations = {

	.get_acl = orangefs_get_acl,

	.set_acl = orangefs_set_acl,

	.setattr = orangefs_setattr,

fs/orangefs/orangefs-bufmap.c

+2 −2
Original line number Diff line number Diff line
@@ -71,9 +71,9 @@ static void put(struct slot_map *m, int slot) 
	spin_lock(&m->q.lock);

	__clear_bit(slot, m->map);

	v = ++m->c;

	if (unlikely(v == 1))	/* no free slots -> one free slot */

	if (v > 0)

		wake_up_locked(&m->q);

	else if (unlikely(v == -1))	/* finished dying */

	if (unlikely(v == -1))     /* finished dying */

		wake_up_all_locked(&m->q);

	spin_unlock(&m->q.lock);

}