The University of Arizona
    For questions, please open a UAService ticket and assign to the Tools Team.
UA HPC
Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 72 Next »

 

 

Disk Storage


When you obtain a new HPC account, you will be provided with the following storage:

  • /home/uxx/netid - 15GB    (backed up nightly)  
  • /extra/netid  -   200GB (no backups)
  • /tmp - some clusters have /tmp space which is on a disk in each compute node. Ocelote has about 800GB available on each node. It is often faster and more efficient to use this during your jobs and then do the final write to the shared array.

Additional storage:

  • /xdisk/netid  -  200GB to 1TB available on request. The time limit is 45 days with one extension. This allocation is deleted after 45 days if no extension is requested. There are directory level quotas so any files in this directory count against this quota not matter who created it.  The data is not backed up.
  • /rsgrps/netid -  rented / purchased space  (no backups)

File count limit:
 As of this year file count limits have been imposed. The lack of limits created performance problems on the storage array for reasons that get very detailed. This means you may not be able to create files while there is space seemingly available.

  • /extra and /rsgrps are limited to 600 files / GB
  • /home and /xdisk do not have limits 


We strongly recommend that you do some regular housekeeping of your allocated space. Millions of files are hard to keep organized and even more difficult to migrate. Archiving or using a tool like tar will help keep our disk arrays efficient and potentially free up more space for you to use.

xdisk

Use this link for details on xdisk usage

extra

/extra is something new with Ocelote. When you log in to Ocelote for the first time, an allocation of 200GB will be created for you. It takes an hour or two to show up, and then it is permanent. Remember that it is not backed up like /home. The number of files within the 200GB is limited to 120,000.

uquota is the command to display how much space you have used / remaining

                              used  soft limit  hard limit       files/limit
Filesets with group access:
/rsgrps/me                    75.45G          2T          2T      539741/1228800


Job Limits

Job Time Limits

 Each group is allocated a base of 24,000 hours of compute time. This allocation is refreshed monthly. 

The command va will display your remaining time

You can use this time on either the standard nodes which do not require special attributes in the scheduler script, or on the GPU nodes which do require special attributes. The queues are setup so that jobs that do not request GPU's will not run there.

ElGato has a different allocation method of time as it is funded through an NSF MRI grant with usage time provided for campus researchers outside of the grant recipients. Reference the www.elgato.arizona.edu web site.



 PBS Batch Queue Limits

The batch queues on the different systems have the following memory, time and core limits.

QueueDescription
standardUsed to consume the monthly allocation of hours provided to each group
windfallUsed when standard is depleted but subject to preemption
high_priorityUsed by 'buy-in' users for purchased nodes



 System 

 Queue Name

 # of Compute Nodes

 Max Wallclock Hrs / Job

 Largest job

Total cores in use / group

 Largest job / memory

 Max # of Running Jobs

Max Queued Jobs
Ocelotestandard3482401344**
cores 

2016


8064GB5003000

windfall4002401344**
cores 
8064GB5003000

high_pri527201344**
cores 
8064GB5005000


***  This limit is shared by all members of a group across all queues. So you can use the system 1000 core limit by one user on the standard queue or share it across multiple users or queues. 

  • No labels