The University of Arizona
    For questions, please open a UAService ticket and assign to the Tools Team.
Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 57 Next »

 

 

Disk Storage

 

When you obtain a new HPC account, you will be provided with the following storage:

  • /home/uxx/netid - 15GB    (backed up nightly)  
  • /extra/netid  -   200GB (no backups) Ocelote and ElGato only 
  • /tmp - some clusters have /tmp space which is on a disk in each compute node. Ocelote has about 800GB available on each node. It is often faster and more efficient to use this during your jobs and then do the final write to the shared array.

Additional storage:

  • /xdisk/netid  -  200GB to 1TB available on request. The time limit is 45 days with one extension. This allocation is deleted after 45 days if no extension is requested. There are directory level quotas so any files in this directory count against this quota not matter who created it.  The data is not backed up.
  • /rsgrps/netid -  rented / purchased space  (no backups)

File count limit:
 As of this year file count limits have been imposed. The lack of limits created performance problems on the storage array for reasons that get very detailed. This means you may not be able to create files while there is space seemingly available.

  • /extra and /rsgrps are limited to 600 files / GB
  • /home and /xdisk do not have limits 


We strongly recommend that you do some regular housekeeping of your allocated space. Millions of files are hard to keep organized and even more difficult to migrate. Archiving or using a tool like tar will help keep our disk arrays efficient.

xdisk

Use this link for details on xdisk usage

extra

/extra is something new with Ocelote. When you log in to Ocelote for the first time, an allocation of 200GB will be created for you. It takes an hour or two to show up, and then it is permanent. Remember that it is not backed up like /home. The number of files within the 200GB is limited to 120,000.

uquota is the command to display how much space you have used / remaining


Job Limits

Job Time Limits

 Each group is allocated a base of 24,000 hours of compute time, this allocation is refreshed monthly.  The allocation can be used on either the htc/cluster/smp clusters or on the new cluster, Ocelote., or a combination.

The command va will display your remaining time

ElGato has a different allocation method of time as it is funded through an NSF MRI grant with usage time provided for campus researchers outside of the grant recipients. Reference the www.elgato.arizona.edu web site.

 

 

 

 

 

 

 

 

 PBS Batch Queue Limits

The batch queues on the different systems have the following memory, time and core limits.

 

 System 

 Queue Name

 # of Compute Nodes

 Max Wallclock Hrs / Job

 Largest job

Total cores in use / group

 Largest job / memory

 Max # of Running Jobs

Max Queued Jobs
Ocelotestandard3312401344**
cores 

1344**


8064GB5003000

windfall3312401344**
cores 

8064GB500

high_pri367201344**
cores 

8064GB500









 Cluster (8400)

 standard

 124

 240

 504 cores

1008 (user limit) ***
 

 1008 GB

 512

500

 

 windfall

 229 (all)

 240

 256 cores

512 (user limit)

 512 GB

 512

 

 

 cluster_high

 105 (buy-in)

 720

 512 cores

buy-in

 1024 GB

 1148

 

 

 

 

 

 

 

 

 

 

 SMP (UV1000)

 standard

 

 240

 256 cores

512***

 512 GB

 512

500

 

 windfall

 

 240

 256 cores

 

 512 GB

 256

 

 

 smp_high

 

 720

 512 cores

 

 1280 GB

240

 

 

 

 

 

 

 

 

 

 

HTC 

 standard

 104

 720

 256 cores

512***

 512 GB

 512

500

 

 windfall

 104

 720

 256 cores

 

 512 GB

 512

 

 

 htc_high

 10

 720

 512 cores

 

 512 GB

 240

 

 

**  This represents 48 physical nodes, and 9.2TB of memory
***  This limit is shared by all members of a group across all queues. So you can use the system 1000 core limit by one user on the standard queue or share it across multiple users or queues. 

  • No labels