Orangefs: kernel client part 6

What:			/sys/fs/orangefs/perf_counters/*

Date:			Jun 2015

Contact:		Mike Marshall <hubcap@omnibond.com>

Description:

			Counters and settings for various caches.

			Read only.

What:			/sys/fs/orangefs/perf_counter_reset

Date:			June 2015

Contact:		Mike Marshall <hubcap@omnibond.com>

Description:

			echo a 0 or a 1 into perf_counter_reset to

			reset all the counters in

			/sys/fs/orangefs/perf_counters

			except ones with PINT_PERF_PRESERVE set.

What:			/sys/fs/orangefs/perf_time_interval_secs

Date:			Jun 2015

Contact:		Mike Marshall <hubcap@omnibond.com>

Description:

			Length of perf counter intervals in

			seconds.

What:			/sys/fs/orangefs/perf_history_size

Date:			Jun 2015

Contact:		Mike Marshall <hubcap@omnibond.com>

Description:

			The perf_counters cache statistics have N, or

			perf_history_size, samples. The default is

			one.

			Every perf_time_interval_secs the (first)

			samples are reset.

			If N is greater than one, the "current" set

			of samples is reset, and the samples from the

			other N-1 intervals remain available.

What:			/sys/fs/orangefs/op_timeout_secs

Date:			Jun 2015

Contact:		Mike Marshall <hubcap@omnibond.com>

Description:

			Service operation timeout in seconds.

What:			/sys/fs/orangefs/slot_timeout_secs

Date:			Jun 2015

Contact:		Mike Marshall <hubcap@omnibond.com>

Description:

			"Slot" timeout in seconds. A "slot"

			is an indexed buffer in the shared

			memory segment used for communication

			between the kernel module and userspace.

			Slots are requested and waited for,

			the wait times out after slot_timeout_secs.

What:			/sys/fs/orangefs/acache/*

Date:			Jun 2015

Contact:		Mike Marshall <hubcap@omnibond.com>

Description:

			Attribute cache configurable settings.

What:			/sys/fs/orangefs/ncache/*

Date:			Jun 2015

Contact:		Mike Marshall <hubcap@omnibond.com>

Description:

			Name cache configurable settings.

What:			/sys/fs/orangefs/capcache/*

Date:			Jun 2015

Contact:		Mike Marshall <hubcap@omnibond.com>

Description:

			Capability cache configurable settings.

What:			/sys/fs/orangefs/ccache/*

Date:			Jun 2015

Contact:		Mike Marshall <hubcap@omnibond.com>

Description:

			Credential cache configurable settings.

ORANGEFS

========

OrangeFS is an LGPL userspace scale-out parallel storage system. It is ideal

for large storage problems faced by HPC, BigData, Streaming Video,

Genomics, Bioinformatics.

Orangefs, originally called PVFS, was first developed in 1993 by

Walt Ligon and Eric Blumer as a parallel file system for Parallel

Virtual Machine (PVM) as part of a NASA grant to study the I/O patterns

of parallel programs.

Orangefs features include:

  * Distributes file data among multiple file servers

  * Supports simultaneous access by multiple clients

  * Stores file data and metadata on servers using local file system

    and access methods

  * Userspace implementation is easy to install and maintain

  * Direct MPI support

  * Stateless

MAILING LIST

============

http://beowulf-underground.org/mailman/listinfo/pvfs2-users

DOCUMENTATION

=============

http://www.orangefs.org/documentation/

USERSPACE FILESYSTEM SOURCE

===========================

http://www.orangefs.org/download

Orangefs versions prior to 2.9.3 would not be compatible with the

upstream version of the kernel client.

BUILDING THE USERSPACE FILESYSTEM ON A SINGLE SERVER

====================================================

When Orangefs is upstream, "--with-kernel" shouldn't be needed, but

until then the path to where the kernel with the Orangefs kernel client

patch was built is needed to ensure that pvfs2-client-core (the bridge

between kernel space and user space) will build properly. You can omit

--prefix if you don't care that things are sprinkled around in

/usr/local.

./configure --prefix=/opt/ofs --with-kernel=/path/to/orangefs/kernel

make

make install

Create an orangefs config file:

/opt/ofs/bin/pvfs2-genconfig /etc/pvfs2.conf

  for "Enter hostnames", use the hostname, don't let it default to

  localhost.

create a pvfs2tab file in /etc:

cat /etc/pvfs2tab

tcp://myhostname:3334/orangefs /mymountpoint pvfs2 defaults,noauto 0 0

create the mount point you specified in the tab file if needed:

mkdir /mymountpoint

bootstrap the server:

/opt/ofs/sbin/pvfs2-server /etc/pvfs2.conf -f

start the server:

/opt/osf/sbin/pvfs2-server /etc/pvfs2.conf

Now the server is running. At this point you might like to

prove things are working with:

/opt/osf/bin/pvfs2-ls /mymountpoint

You might not want to enforce selinux, it doesn't seem to matter by

linux 3.11...

If stuff seems to be working, turn on the client core:

/opt/osf/sbin/pvfs2-client -p /opt/osf/sbin/pvfs2-client-core

Mount your filesystem.

mount -t pvfs2 tcp://myhostname:3334/orangefs /mymountpoint

OPTIONS

=======

The following mount options are accepted:

  acl

    Allow the use of Access Control Lists on files and directories.

  intr

    Some operations between the kernel client and the user space

    filesystem can be interruptible, such as changes in debug levels

    and the setting of tunable parameters.

  local_lock

    Enable posix locking from the perspective of "this" kernel. The

    default file_operations lock action is to return ENOSYS. Posix

    locking kicks in if the filesystem is mounted with -o local_lock.

    Distributed locking is being worked on for the future.

DEBUGGING

=========

If you want the debug (GOSSIP) statments in a particular

source file (inode.c for example) go to syslog:

  echo inode > /sys/kernel/debug/orangefs/kernel-debug

No debugging (the default):

  echo none > /sys/kernel/debug/orangefs/kernel-debug

Debugging from several source files:

  echo inode,dir > /sys/kernel/debug/orangefs/kernel-debug

All debugging:

  echo all > /sys/kernel/debug/orangefs/kernel-debug

Get a list of all debugging keywords:

  cat /sys/kernel/debug/orangefs/debug-help