Legacy File System

From CCI User Wiki
Revision as of 16:19, 26 February 2014 by Toddr (talk | contribs) (moved File System to Legacy File System: We have a new file system page; this is now legacy.)

Jump to: navigation, search

As of August 9, 2012, CCI has moved to a new GPFS filesystem. The organization of the filesystem is different than the original and the details can be found below.

General Layout

The CCI GPFS filesystem is built using two block sizes. The small block (sb) portion of the filesystem uses 128 KB blocks. The large block (lb) portion uses 4 MB blocks. There are three main areas of the filesystem: home, data, and provisioned. Each area of the filesystem has its own purpose and merits.

Tree

/gpfs
 /sb
  /home
   /PROJ
    /shared
    /USER
  /data
   /PROJ
    /shared
    /USER
/gpfs
 /lb
  /data
   /PROJ
    /shared
    /USER
  /provisioned
   /PROJ
    /shared
    /USER

Home (project and per-user)

Home can mean either a project home directory or the associated user home directories. User home directories are only writable by the user while project home directories can be written to by any user within the project. This area of the filesystem has a 10 GB quota and is the only area of the filesystem that uses replication to protect data.

Please note: In GPFS, files are counted twice during quota calculations when they are replicated. This means the home directory limit is effectively 5 GB per project (shared + every user's home).

Data (short-term storage)

There are data directories for each project and the associated users on both lb and sb with corresponding links in each user's home directory. This space is meant as a temporary staging area for performing computation. Two block sizes are provided as one may perform better depending on the specific application and the system on which it is executing.

Important: This space will periodically be purged of files older than 14 days. This policy is subject to change based on filesystem demands. If longer-term storage of data is necessary it should be stored in the provisioned area.

Provisioned (long-term storage)

There is a provisioned directory for each project and the associated users on lb only with corresponding links in each user's home directory. This space is meant to allow for longer-term storage of working data than allowed by the data area. It is not meant for long-term storage of results. This space has a 5G quota and is not replicated.

Additional space may be allocated to a project at the discretion of the CCI Director upon written request by the project PI. Any extended quotas are subject to periodic review and potential reduction at the discretion of the Director.