Caltech Home > Caltech HPC Homepage > Documentation > Storage > Migration to VAST storage
Search open search form
Caltech
Resnick High Performance Computing Center
open menu close menu

Migration to VAST storage

IMSS has partnered with Vast to deploy a next generation filesystem to The Resnick HPCC. This new storage solution is no in product You can read more about Vast here:

https://www.vastdata.com/platform/overview

The new guidelines for the Vast file system will be as follows:

  • Up to 20T of free group storage
  • 50G home directories (same as GPFS)
  • $8.00 per terabyte, per month above and beyond the free tier

There will be no free investor tier for Vast as there was with the GPFS filesystem.

Users/groups will be in charge of moving their own data to Vast

All HPC Groups and/or Individual Users will be solely responsible for migrating their own group data to Vast. IMSS will not be managing the migration of this data from the GPFS file system. Typical ways to move data would be using cp,scp,rsync from one file system to the next. (see below)

The current IBM ESS GPFS solution will remain viable through at least the end of the calendar year (end of 2025) and its existing policies will remain intact. We have plans in place to continue this offering as second tier storage. However, plans for second tier storage will not be firmed up until we approach the end of year. Please note that this file system is almost full.

The Vast filesystem will retain the same basic layout as the previous GPFS filesystem, simply replace "central" with "resnick". For example, central/groups/group_name will become /resnick/groups/group_name. A user's home directory will live in /resnick/home/username and unlike the unmanaged group directories, will be migrated on your behalf by IMSS HPCC staff. 

The date and time for the final sync of home directories will be laid out in an upcoming email.

* All HPC Groups will be solely responsible for migrating their own /central/group data to Vast. IMSS will not be managing the migration of this data from GPFS.

Example migration:

SSH to the login nodes:

ssh login.hpc.caltech.edu

Run TMUX to assure connection is stable: (tmux will allow your copy operation to continue to run should the ssh connection to the login node become disrupted)

tmux

Start copy:

rsync -av /central/groups/<group_name>/<your_group_data> /resnick/groups/<group_name>/<your_group_data>