The Linux File System

Every file on the SCC lives somewhere in one big tree. Learning to read that tree, and to move around inside it, is the one skill the rest of this guide assumes.

A Tree, Not a Pile

Linux organizes files into a hierarchical tree of directories. Every directory except one sits inside another directory:

/
`-- project
    `-- scv
        `-- examples
            `-- imaging
                `-- tut_cnc_intro_scc
                    |-- materials
                    |   |-- code.py
                    |   |-- message.txt
                    |   |-- notes.txt
                    |   |-- participants.tsv
                    |   |-- scan_protocol.txt
                    |   `-- task_timing.csv
                    `-- students
                        |-- kylekurk
                        `-- lbakst

The one exception is the directory at the very top. It is called the root and it is written as a single slash, /. Everything on the system hangs off it.

This is a real directory on the SCC, and you will walk through it yourself in the exercise further down this page.

A directory is identified uniquely by its path — the list of directories you descend through to reach it, starting from the root. So the path to kylekurk above is:

/project/scv/examples/imaging/tut_cnc_intro_scc/students/kylekurk

Read it left to right as a set of directions: start at the root, go into project, then scv, then examples, then imaging, then tut_cnc_intro_scc, then students, and you have arrived.

A path that starts with / is absolute — it works from anywhere, because it starts at the root. A path that does not is relative: it starts from wherever you happen to be standing. From inside tut_cnc_intro_scc, the same directory is just students/kylekurk.

Where Your Files Live

The SCC gives you three everyday places to put your own files, and they are not interchangeable. Putting data in the wrong one is the most common mistake new users make — usually by filling up a 10 GB home directory with imaging data.

Imaging data belongs in /projectnb. It is the largest space, and it is shared with everyone on your project, which is what you want when a collaborator needs to look at your derivatives.

Confidential Data: /restricted

There is a fourth space. /restricted holds highly confidential data — RCS describes it as dbGaP compliant, and it is used primarily by BUMC researchers. It repeats the same split as above inside itself, so a project with restricted data gets both:

You cannot simply move data into /restricted. A project is granted access to it deliberately. If your data carries confidentiality or regulatory requirements, talk to RCS at help@scc.bu.edu before you put it anywhere on the SCC.

Finding Your Way Around

Three commands do almost all the navigating. At any moment you are standing in one directory, called your working directory.

Three shorthands are worth learning straight away:

cd with nothing after it takes you home, wherever you are.

Exercise: Navigate the Tree

Work through this in a terminal. Everything it visits is readable by every SCC user, so it will behave the same for you as it does here.

Where are you right now?

[scc4]$ pwd
/usr4/tutorial/janedoe

A fresh terminal starts in your home directory.

Move to the exercise directory, using an absolute path.

[scc4]$ cd /project/scv/examples/imaging/tut_cnc_intro_scc
[scc4]$ pwd
/project/scv/examples/imaging/tut_cnc_intro_scc

Look at what is there.

[scc4]$ ls
materials  students

The same shape as the tree at the top of this page: a materials directory holding files everyone shares, and a students directory holding one directory per person.

Go into materials, this time with a relative path.

[scc4]$ cd materials
[scc4]$ pwd
/project/scv/examples/imaging/tut_cnc_intro_scc/materials

No leading slash, so materials was read from where you were standing rather than from the root.

List what is in it.

[scc4]$ ls
code.py      notes.txt          scan_protocol.txt
message.txt  participants.tsv   task_timing.csv

Read the message.

[scc4]$ cat message.txt

cat prints a text file to your screen. Read what it says — that is the answer to the exercise.

Go back up one level.

[scc4]$ cd ..
[scc4]$ pwd
/project/scv/examples/imaging/tut_cnc_intro_scc

.. moved you from materials into the directory that contains it.

Go home.

[scc4]$ cd
[scc4]$ pwd
/usr4/tutorial/janedoe

cd with no path at all always returns you to your home directory, which is the quickest way out of being lost.

Making Your Own Directory

Give yourself somewhere to work inside your project space:

[scc4]$ cd /projectnb/your_project
[scc4]$ mkdir your_username
[scc4]$ cd your_username

Reading ls -l

[scc4]$ ls -l
total 1
drwxr-xr-x 3 tuta1 tutorial 4096 Oct 28 16:03 tuta1

Permissions

That drwxr-xr-x string is four pieces run together:

Within each group of three, r is read, w is write and x is execute. A - means that permission is off. So r-x is "read and execute, but not write".

For a directory, x means "can enter". A directory you can read but not execute is one you can list but cannot cd into.

Changing Permissions

chmod takes a who, a + or -, and a permission:

[scc4]$ chmod g+w my_directory

That adds write permission for the group — which is how you let the rest of your project contribute to a directory you made.

[scc4]$ chmod u+x test.sh

That adds execute permission for you, which is what makes a shell script runnable.

Demonstration: Permissions in Action

Permissions are easier to believe once you have been locked out of something. This is a live demonstration: the instructor changes one file, and the room watches what happens to everybody else.

Everyone: read the file. Move into the workshop materials and print notes.txt.

[scc4]$ cd /project/scv/examples/imaging/tut_cnc_intro_scc/materials
[scc4]$ cat notes.txt

It works. The file is -rw-r--r--, so "other" — which is everyone who is not the owner and not on the owner's project — has r.

Instructor: take the read permission away.

[scc4]$ chmod go-r notes.txt
[scc4]$ ls -l notes.txt
-rw------- 1 kkurkela scv 617 Oct  1 17:46 notes.txt

Nothing about the file itself changed. It is the same bytes in the same place. Two characters of the permission string are different: the r for group and the r for other.

Everyone: read it again.

[scc4]$ cat notes.txt
cat: notes.txt: Permission denied

The instructor will not see this error. The owner still has rw-, so for them the file reads normally. That is the whole point: permissions are not a property of the file alone, they are a question about who is asking.

Instructor: put it back.

[scc4]$ chmod go+r notes.txt
[scc4]$ ls -l notes.txt
-rw-r--r-- 1 kkurkela scv 617 Oct  1 17:46 notes.txt

Everyone: read it one more time. It works again, immediately — no logging out, no waiting.

[scc4]$ cat notes.txt

This is why a collaborator can be staring at the same path you are and still get Permission denied, and why ls -l is the first thing to check when they do.

Further Reading

• RCS research storage documentation