NBCS: Managing Disk Space on RCI

 
This page is intended to help the RCI user work within the limits of a quota system. The following topics are available: 

Quotas and Unix File System Basics

Computer files are stored on computer disks, which are storage devices attached to the computer. A disk can hold thousands of files belonging to many different users. Usually, limits of some sort are placed on how much can be stored on a computer by a particular user, normally in their "home" directory. The normal login procedure connects you to your "home" directory, which is referred to by your user name. When you refer to a filename without any special qualification, you will get a file by that name in the current directory. The ls command lists the files in the current directory and the pwd command gives you the name of the current directory. 

 

There are many different directories on a disk. A user can have a home directory that itself contains a directory (known as a subdirectory). For example, John Q. Smith can have a main directory, "jqsmith", and a subdirectory, "work". The subdirectory "work" is contained within "jqsmith"; "work" could also be referred to as a child directory of "jqsmith". 

 

This, in turn, falls within the structure of the entire UNIX file system, which can be thought of as an upside-down tree (as in a family tree). The top directory is called root and is noted as "/". Next come other directory names, such as usr, u1, u2, etc. On Rutgers's systems, the user directories are subdirectories of u1, u2, or some other u#. On larger Rutgers systems, the u# directories are placed under a directory named for the system (eden or rci). As shown here, the directory jqsmith is a subdirectory of u2, which in turn is a subdirectory of rci, which in turn is a subdirectory of the root directory. 

Since disk space is both limited and shared, it is imperative that users manage disk space wisely. There are a number of UNIX commands available to help you manage the disk space that you have been allotted. 


Commands to Help Manage Disk Space


Each of these commands requires you to telnet into your RCI account and type them at the system prompt (usually amenti> or niflheim>, although knowledgable users may have customized the prompt's appearance).

The quota Command

The amount of disk space that you are allowed for permanent storage is called your "quota". On RCI, you have one quota which is shared among your incoming mail and file storage.  Currently, most RCI users are set to a 100 megabyte quota.

Once you exceed your quota, you will have to bring down your usage to below your quota. If you don't do anything, you will be unable to write any new files, including making changes to reduce existing files, and new mail will be unable to arrive for you. (Usually, the only thing you can do at that point is to delete files with the rm command.)

To check the status of your quota, use the command 

quota

If you have not exceeded your quota, the quota command will not produce any response. If you have exceeded your quota, the quota command will give a simple message stating how much over quota you are, as follows. 

quota

Block limit reached on /pandemonium/vol1
If you want the quota command to respond in more detail, showing your current usage, even if you are not over quota, use the verbose (-v) form of the command. 
quota -v

Filesystem     usage  quota  limit    timeleft  files  quota  limit  timeleft
/pandemonium/vol1
              102400 102400 102400    EXPIRED     35      -1     -1
On RCI, "/pandemonium" is the disk name for the user file areas in /rci/u# and /var/mail (the incoming mail area)  However, you should avoid using "/pandemonium" in your commands, as we occasionally replace the user disks with bigger versions.  Instead, use "/rci/u#" in your commands, replacing the "#" character with the specific "u" directory your account is in.  You can see this directory with the pwd (see above) command. 

If you are over quota because of incoming mail messages , you may be able to bring down your usage by reading your mail and either deleting large messages or storing them in a folder on your desktop computer (which will also delete them from the incoming mail area).  

(By the way, RCI has a limit of about 15 MB for any single mail message.  In general, you should not use mail to send large (for example, over 1 megabyte) files, no matter how convenient it may appear.  Quotas may prevent you or the recipient from seeing such files. You can send an e-mail message to help@rci.rutgers.edu for suggestions on how to send large files from one place to another. )  See the pine document for details on reading mail. 

The du Command

The command du tells you how much disk space is used in the designated directory (the current directory by default) and all subdirectories in it. Therefore, if you issue the command du -k from your home directory (to get the usage in kilobytes), the response could be something like: 
>du -k

4       ./oldMail
20      ./Mail
4       ./.nn
2       ./.mosaic-personal-annotations
5       ./.msg
955     .
The last entry is the total space used in this directory and all subdirectories. This allows you to see where your usage is. 

The ls Command

To list the contents of a directory, you use the command ls. Here are some options to ls:
ls -l 
Give protection, size (in bytes), date, etc., information on listed files 
ls -A 
List all of the files including the "hidden" files that start with a "." 
ls -s 
Show the size (in kilobytes) of the listed files 
ls -R 
List the contents of the current directory and all subdirectories "recursively" (that is, ...) 
ls -AsR 
The last three above used together. 
The output from ls may be sorted using what is called a "pipe" (the vertical bar character, "|") and the sort command. Type 

ls -As | sort -n

to cause the current directory contents to be displayed in size order. 

The rm Command

To remove a file, you type rm filename where filename is the name of the file you want to delete. If you want to remove all files with a common attribute in their name you can use the "wild card" character, "*". 

One word of advice: since "*" matches anything, use the interactive (-i) option of rm to prevent inadvertent deletion of desired files. For instance: you think that you want to delete all files that begin with "tes", so you type 

rm -i tes*

rm asks if you want to remove test.dat. You respond with a yto say yes. rm now asks if you want to remove test.c. This you want to keep, so you type nfor no. 

When looking for files to delete, consider the following: 

Files ending in ~or .BAK 
These are the previous versions of the similarly named files. For example, test.dat~ is the previous version of test.dat
Files ending in .CKP or surrounded by #...# 
These are "checkpoints", intermediate versions of files written to retain some changes in case of a system crash. These are usually automatically removed by the programs once you've successfully saved the changes, but under some circumstances they may get left behind. 
core 
This is a snapshot of system memory created when a program fails. This is not useful to the average user and can safely be deleted. 
Files ending in .dvi, .log, and .ps, if you have the original ending in .tex 
These are TeX/LaTeX output files and can easily be recreated from the .tex files. 

The gzip Command

The gzip command is used to reduce the space occupied by a file without removing it. The command would be 

gzip filename

The "wild card" character "*" may be used with gzip, as in gzip *.c to compress all files that end with ".c". The result of this command is a smaller file that will have the original name with the extension .gz added to it. 

The gunzip and zcat Commands

To work with a file that has been compressed with gzip, you either gunzip it or view it with zcat. The gunzip command will restore the original file with the original file name: 

gunzip filename.gz

The zcat command will leave the file compressed but will send an uncompressed version to the screen. 


Mail Messages

One of the most common quota problems is caused by people saving unnecessary old mail. If, in using the commands above, you notice a large mbox or mail.txt file, you should enter your mail reading program and review your messages and delete some of them. Refer to the appropriate man pages, documents, and internal help files for how to do this in your mail reader. (Information on the supported UNIX mail reader, pine, is available.) 


Quota Increases and the check-criteria command

We can, at your request, increase your quota to 500 Mb. At the same time, we can open a directory for you on the scratch disk, so that you have a place to put large, temporary files, and on the file archive disk, so that you can move large, stable files out of your login directory. (We use "over 3 Mb" and "doesn't change over a month" as our rules of thumb for "large" and "stable", but these are not fixed criteria.) The scratch and archive services are described elsewhere. You should send an e-mail message to help@rci.rutgers.edu to request this increase. 

However, before we can do this, we have to be sure you're using the space you have efficiently. 

This page gives some tips on efficient usage. We've also written a program to help people requesting quota increases to see which files of theirs might be considered to be "inefficient use" of the space. The program is called check-criteria, and to use it, you type 

check-criteria -v

at the system prompt. It will go through your home directory and tell you whether or not you meet the criteria for a quota increase. The -v option will show you which files should be removed.  You can also type 

check-criteria -h

for a short description of what it's looking for. 


Alternate Storage Locations on the RCI System

Scratch Space

The scratch area currently provides a total of about 350 gigabytes of storage (to be shared among all RCI users). A directory created in the scratch area will have the following path: /rci/scratch/users/yourusername (e.g., /rci/scratch/users/janedoe). Files may be copied or moved to and from the scratch area with the cp and mv commands, e.g., 

cp /rci/u2/janedoe/myfile /rci/scratch/users/janedoe/myfile 

Use of scratch space is on a first-come-first-served basis. Please be a good citizen and 

  1. delete files you no longer need, and 
  2. compress your files using gzip. 
Active files stored in the scratch area remain available for 10 days, after which they are automatically deleted. Files stored in the scratch area are not backed-up, so if you need to keep an archival copy, you should do so on a tape

Archive Space

Files on the archive are stored on a very large (350 Gigabyte, at writing) disk. This replaced the former optical/CD-ROM archive, which had performance problems.

A directory created on the archive will have the following path: /rci/archive/users/yourusername (e.g., /rci/archive/users/janedoe). Files may be copied or moved to-and-from the archive area with the cp and mv commands, e.g., 

cp /rci/u2/janedoe/myfile /rci/archive/users/janedoe/myfile 

Please conserve space (for yourself and for others) by compressing your files with gzip before copying or moving them to the archive. Files that have been gzip'd may be read with the zcat command or, alternatively, with gzip -c.

Tapes

The RCI system has a variety of tape drives available for long-term storage and/or for importing/exporting data from/to other institutions. These tapes drives accomodate the following types of tapes: 
  • round (reel) tapes (6250 and 1600 BPI): 2 drives 
  • square 3480 cartridges: 4 drives 
  • 4mm digital audio tape (DAT) (regular & high density): 1 drive 
  • 8mm Exabyte (regular & high density): 1 drive 
  • QIC cartridges: 1 drive 
  • Digital Linear Tape (DLT): 3 drives 
Supported tape formats, written in either the ASCII or EBCDIC character sets, are 
  • IBM Standard Label 
  • IBM non-labeled 
  • ANSI label 
  • TAR 
  • CPIO 

  • (we can also handle raw tapes) 
The various tape media and drives provide different performance tradeoffs (capacity vs. speed).

Return to RU Main Site
Please direct all questions regarding RCI to help@rci.rutgers.edu
Please direct all questions regarding this website to webmaster@rci.rutgers.edu

Last updated: May 23, 2005, 2:24 pm