Introduction

I have seen some misinformation regarding Unix file permissions. I will try to set the record straight. Take a look at this example of some output from ls:
On the first line, that "root" says that the directory is owned by the user called "root". And that "bin" is the group of the directory. You will need to understand users and groups and I will assume that you do. My goal is to explain that "drwxrwxr-x" and "-r-xr-xr-x" stuff. That field is a combination of the file type and access permission. Collectively, this information is sometimes called the file mode. And sometimes it is called the permissions. A good place to start is by taking a quick look at how this information is actually stored on disk. The way it is stored affects a number of decisions that have been made over the years.

How the "File Mode" is stored.

On disk, information about a file is stored in structure called an "inode". Each file will have it own inode. One data item in an inode is called the "mode" and it looks like this:
I think that the "mode" really means just the permissions and that the file type was shoved in to save space. But the author of the ls program is treating the mode as a single item. That's why the file type is the first character of the permission string in the "ls -l" output. The file type will usually be one of the seven types that I show above. Some versions of Unix will add a few more. The only other thing to notice from the data layout is that there is no room to add more permission bits.

Representing those permissions in octal

The ls program might display, say, "rwxrwxrwx" for the permissions on a file. It is also very common to use an octal number to express the permissions on a file. And as you see above, this is how they are stored. You may hear someone say that some has 777 permissions. This is the same as "rwxrwxrwx" and much easier to pronounce. So you need to know how to convert them. Three binary digits or bits corresponds to one octal digit:

So the "read bit is worth 4, the write bit is worth 2 and the execute bit is worth 1. You just add these up to get an octal digit. So "-rwxrwxrwx" is 777. And "-rwxr-x---" is 750. If you still can't see how to get from "r-x" to 5, maybe this table will help:

Note we need 4 octal digits to express all of the permission bits. The first three bits are special and are frequently zero. And you almost always learn about the trailing 9 bits first. Some people stop there and never learn those first three bits. But there are 12 permission bits, not just 9. That said, let's now take a look at the trailing 9 bits.

The Basic Permission Bits

We have 3 triples: one triple for the user, one triple for the group, and one triple for other. Sometimes the "user" is called the owner. And sometimes "other" is called "world". I will use "user" and "other" because the chmod command uses the letters u g and o to refer to these triples.

Which Set of Bits Applies to You?

When Unix decides what you can do, it doesn't use all 9 bits. Unix picks the first triple that applies to you. Consider this: Even though joe owns this file, he cannot access it. (Since joe owns the file, he could give himself access. More on that later.) Also root is special. root is granted rwx to all directories and rw to all files. On a file, if any of the 3 x bits are set, root has execute permission. This special permission is often disabled on network mounted filesystems.

What do r w and x really mean for a file?

For a file, "read" and "write" are pretty intuitive. The x for "execute" means that the kernel may attempt to run the file. For that to work, the file must be an executable (output from a compiler) or a shell script with a "#!" first line. For a directory, things are a little more complex. With a directory, "write" permission means that you can create new files in the directory or remove old files. It sometimes surprises people that you can remove a file which you cannot read. The unix rm command will test for that and issue a warning, but you can suppress that warning with -f. And warning or no, if you want to remove an unreadable file from a writable directory, you may. And rmdir will not bother to check at all.

What do r w and x really mean for a directory?

A directory is a file too, and "read" permission means you can read it. But you really cannot do very much without x permission as well. With directories, you usually have both read and execute permission or neither. On a directory, that x is officially called "search permission". You need x to use a directory in a pathname. So if you try "cat /etc/passwd", you will need x on / and /etc. You also need x to cd into a directory. Suppose you have read but not search (x) permission on a directory. What can you do? Not much. You can use "ls" to view the file names. Even "ls -l" will not work. Read access without search permission is not very useful. Still that is better than having only write permission on a directory...that is completely useless. I have not seen any other documentation that states this explicitly, so let me repeat it: write but no execute permission on a directory grants nothing at all.Suppose you have search (x) permission but no read permission on a directory. Now you can open files in the directory if you happen to know the file's name. You can cd into the directory. And that is it. You cannot even create a new file. Adding write permission will allow you to create files. And you can then delete files if you happen to know their name.

Symbolic Links Are Special

The permission settings on a symbolic link are a little special as well. They are completely ignored. Many versions of Unix have no way to change them.

The Setuid and the Setgid Bits

Take a look at this: The passwd file is writable only by root (Remember, root is special. It can write a file that has no write permissions set). The shadow file, which is where passwords are stored, cannot even be read by ordinary users. But joe wants to change his password. He can do that by running /usr/bin/passwd. Notice those r-s permissions. The passwd program has the suid and sgid bits set. This turns the x's into s's. In octal, it would be 6555. The passwd program is owned by root. When joe runs it, it does not run as "joe". Instead, it runs as it owner which is root. So the passwd program can change joe's password for him. The sgid bit works the same way, except it causes the passwd program to run with the group sys instead of joe's group. The suid and sgid do not get their own position in the ls. When the suid bit is set, ls displays a s rather than a x for the owner execute permission. What if the suid bit is set, but the owner execute bit is off? ls will display a capital S in the case. The sgid bit is displayed in a similiar manner, except that it interacts with the group execute permission. (The set uid concept was invented by Dennis Ritchie as he was developing Unix.)

To expand on things a little, while joe is running the suid-to-root passwd program, "joe" is the real uid and "root" is the effective uid. The passwd program can obtain both of these id's if it wants to. This is how the passwd program knows to allow joe to change only joe's password.

The Sticky Bit

The Posix standard says that if the sticky bit is set on a directory, mere write permission on the directory is no longer enough to allow files to be removed. You must additionally own the file or own the directory. root continues to be able to delete from any directory regardless of permissions. Formerly this bit served another purpose. On some OS's it still does. I will elaborate in an appendix below. The sticky bit affects the "other" execute bit in the ls display. Except that it uses t and T rather than s and S. For example: In that /tmp directory above, anyone can create new files. But because of the sticky bit, one user cannot delete another user's files.

Limiting File Permissions with umask

When files are created the program that creates can specify the initial permission setting. You can override that with umask. The umask is a set of prohibiting bits. There is a umask command that allows you to view and change the umask. For example, "umask 022" prohibits group write and other write on newly created files. Or "umask 027" prohibits group write and it prohibits other read, write, or execute. You can do a "umask 0" to let the program do what it wants as it creates programs. But you cannot go any further. You cannot force a program to turn a bit on. The umask setting affects files, directories, named pipes (aka fifos), and special files. It may or may not affect symbolic links. It also affects somes forms of Inter Process Communication but that is beyond the scope of this article. And believe it or not, named sockets are exempt from umask. This exemption is required by Posix.

Changing File Permissions with chmod

Only the owner of a file or root can change the permissions on a file. This operation is not affected at all by the umask setting. If you change permissions on a symbolic link, the link will be followed and you will change the target file. It is possible that only root will have the power to set a file's sticky bit. As an example, "chmod 700 somefile" wil let the owner read, write and execute the file, while disallowing all access to any other users.

Using Symbolic Mode with chmod and umask

Posix introduced a new syntax for the chmod command. The idea was that the new syntax will replace the use of an octal constant with the chmod command. The octal constant is still allowed and I think it's here to stay. But the new symbolic syntax lets you change a few bits without knowing what the others are. For example, let's say that I want to make a file inaccessible to others but I don't what to change the acess for the user or the group. I would need to do:
ls -l file
Look at file and figure out the current settings.
chmod 750 file
I had to determine that the first two digits are currents 7 and 5 before I could do my chmod command. With the new syntax, I can simply do:
chmod o= file
to turn off the final 3 bits. As another example, "chmod u+x file" will allow the user to execute the file. On the other hand, these two commands are equivalent:
chmod 750 file
chmod u=rwx,g=rx,o= file
and I prefer the first syntax.

The symbolic mode can be a comma separated list of specifications. Each specification has three components: <who><operation><bitlist> The X is the same as x unless the file is a nondirectory and currently has no x bits set. The s (set uid or set gid) can only be specified for user or group permissions. The t can only be specified for user permissions. A few examples with help. Above we saw /usr/bin/passwd was 6555 (-r-sr-sr-x in ls). Here are some ways that could be achieved:
chmod 6555 /usr/bin/passwd
chmod u=rxs,g=rxs,o=rx /usr/bin/passwd
chmod ug=rxs,o=rx /usr/bin/passwd
chmod a=rx,ug+s /usr/bin/passwd
And there are many other ways to do this.

For the most part chmod is immune to umask in that whatever bits it wants to set do not get modified by the umask. However, under one condition, the chmod command will examine the current setting of umask to determine which bits it would like to set. This happens when you leave the "who" field blank. Like this:
chmod =w somefile
The difference between "a=w" and "=w" is subtle. Here is an example that may help. Many programs try to create file with 666 (-rw-rw-rw-) permissions. But this gets modified by the umask. Suppose the you want to set a file to 666 but modified by the current umask. We could turn off all bits, then turn on read and write bits that are allowed by the current umask:

chmod a=,=rw somefile

And speaking of umask, you can use symbolic arguments with the umask command as well. However, Posix, in their wisdom, decided that in this event to logic would be reversed. So if use umask with an octal argument you specify the bits to be prohibited. But if you use umask with a symbolic argument, you specify the bits to allow. So these are equivalent:
umask 022
umask u=rwx,go=rx

Summary

At this point you have enough information to mostly understand those 12 permission bits. There are several very special cases that I have ignored or glossed over. In separate posts below I will address these. The information in this first post is pretty much universal. A Posix compliant OS must support this stuff. That covers almost all versions of Unix released in the past 10 years. The following articles discuss features that may not be universal.

I previously have mentioned that when a directory has the sticky bit set, a file can be deleted only by the owner of the file or the owner of the directory. This behavior is specified by Posix and is now rather universal. However that was not the original purpose of the sticky bit. I see that the Posix definition of the ls command actually calls the sticky bit "the restricted deletion flag". The constant in the C header files for this bit is S_ISVTX. The svtx stands for save text and this reveals the original purpose of the bit.

The original idea was that if the sticky bit was set, the text segment of a process would stay in the swap area. This would allow it to be read into memory with a single disk read. Originally, Unix used a filesystem with a block size of 512 and the blocks tended to scatter. So it would take time to collect the text segment and load it into memory. At first there was no paging, only swapping. And to run a program would need to be completely loaded into memory. Now we page-fault a program into memory. Pages are loaded as they are needed. So is the original use of the sticky bit dead? Well, no. It depends on the OS.

With HP-UX, this usage is still alive and is documented on the HP-UX chmod(2) man page. What use is this? From The HP-UX Kernel Tuning and Performance Guide via HP-UX Faq:
"When applications are located remotely, set the "sticky bit" on the applications binaries, using the chmod +t command. This tells the system to page the text to the local disk. Otherwise, it is "retrieved" across the network. Of course, this would only apply when there is actual paging occurring. More recently, there is a kernel parameter, page_text_to_local, which when set to 1, will tell the kernel to page all NFS executable text pages to local swap space."

With Solaris, there is no sign of the original use of the sticky bit, however according to the Solaris chmod(2) man page:
"If a regular file is not executable and has S_ISVTX set, the file is assumed to be a swap file. In this case, the system's page cache will not be used to hold the file's data. If the S_ISVTX bit is set on any other file, the results are unspecified."

So you will need to check the chmod(2) man page for your particular OS to see what effect, if any, the sticky bit has on a file. Also, be aware that where an effect does exist, the OS may restrict setting a sticky bit to root. For example, with HP-UX, a malicious user could set a lot of sticky bits and run the system out of swap space.

HP-UX actually also has some symbolic links that have the sticky bit set. I will address this below.

We briefly mentioned that files have a user and group associated with them. Originally, it was just the user and group of whoever created them. But originally, a user could be in only one group at a time. BSD introduced the concept that a user could be in mutiple groups simutaneously. So in BSD, which group was used? BSD decided to use the group of the directory that contained the newly created file.

Many modern versions of unix try to have it both ways. A newly created file gets the group of the user unless the directory has the setgid bit. In that case, the newly created file gets the group of the directory.

And there is an exception to that! Changing the owner or group of a file has security concerns. For that reason, some versions of unix will, optionally, prohibit a user other than root from changing the owner of a file. Additionally, a user is prohibited from changing the group of a file unless he is a member of the new group. This restriction will override the setgid bit on a directory if needed.

We aren't finished with that Set Gid bit yet... Unix has a concept of file locking. File locking is beyond the scope of this thread. But you need to know that file locking comes in two flavors: advisory and manditory. Which flavor applies to a particular file depending on the permission settings. If the group execute bit is off but the setgid bit is on, any file locks on that file are manditory.

Useless Bit Combination?

Every reference that I have seen says that setgid on / group execute off is a otherwise useless combination. Even Richard Stevens (in Advanced Programming in the Unix Environment) says "Since the set-group-ID bit makes no sense when the group-execute bit is off, the designers of SVR3 chose this way to specify that the locking for a file is to be maditory locking and not advisory locking."

Well consider this case: Fred runs the Human Resources department. Fred and his group often need to lookup the vacation days used for employees. Fred decides to write a program so employees can lookup their own vacation days used. For security, Fred makes this program do a lot of logging. Fred decides that he doesn't want his group to use this program. They have other tools that won't clutter his log. So Fred does:
chown fred:hr vdays
chmod 2701 vdays
Now the vdays program cannot be run by members of hr (except fred). But it can be run by everyone else. And it will assume the gid of hr when it does run. I have written a test program, set it up like this, and have run it on both Solaris and HP-UX. It works.

Effect on ls output

While this bit combination may be useful is some limited cases, for better or worse, it will have two effects. The vdays program does work, but if a lock is attempted on the file, it will be manditory. As a practical matter, this would impact only an occasional program like a debugger. But ls may treat this bit combination differently. I have seen both of these...

Symbolic links have evolved over the years. At first, a symbolic link was followed only when opening a file. So:
touch datafile
ln -s datafile slink
chmod 700 slink
did not protect the file called "datafile". This was a security problem. Today that "chmod 700 slink" will change the permissions on datafile. "chown fred datafile" will also change datafile and leave slink alone.

Symbolic Links Must have an Owner

There is a way to change the owner of a symbolic link. It is "chown -h slink". This will change slink and leave datafile alone. The original reason why symbolic links need an owner is a feature called "quotas". Quotas allows the system administration to limit the disk space used by a user. A symbolic link consumes a small amount of disk resources so it must be charged to the appropiate user. We now have a second reason: sticky directories. We need to know who can remove a symbolic link from a sticky directory.

In addition to an owner, Posix requires that symbolic link have a size. That is all that you can depend on. Symbolic links may not even have any permission bits.

Permission Bits Are Required by Posix to be Ignored

On Solaris and Linux, newly created symbolic links are 777 and this is not affected by umask. I could not not find any way to turn the bits off. On HP-UX, symbolic links are also created with 777, but umask does affect this. So:
umask 777
ln -s datafile slink
creates a symbolic link with all of the bits turns off. This had no effect at all on what I could do with datafile. And symbolic link (with a mode of 0) to directories also worked fine.

BSD Has a Way to Change the Permission Bits on a Symbolic Link

The various BSD distros all have a "chmod -h" which is like the required "chown -h". Using this command I tested symbolic links on FreeBSD and found that they ignore permission bits as well. The "chmod -h" command is implemented using a lchmod() system call. (BSD even has a lutimes() system call and a "touch -h" command to invoke it.) So far, so good. Nothing here is violating the Posix standard.

NetBSD May be Violating the Posix Standard

According to the NetBSD symlink man page: "The readlink(2) system call requires read permissions on the symbolic link." readlink() is the system call used by ls to display the target of a symbolic link. So with something like this: that datafile was obtained with readlink(). I don't have access to a NetBSD system for testing so I could not verify this.

HP-UX Transition Links

When HP rewrote HP-UX to conform to System V Release 4, the location of lots of files changed. As one example, my favorite shell moved from /bin/ksh to /usr/bin/ksh. Some people still use /bin/ksh and that will work (for now) because of a symbolic link:
How is HP turning on that sticky bit? The lchmod() system call has been borrowed from BSD. This is an undocumented feature of HP-UX. HP's idea is that they can be sure that a sticky symlink was not created by any users because users do not know about lchmod. Using lchmod() to turn on the sticky bit does not change the properties of the symlink in any way. The only difference is that HP's tl tools will be willing to operate on the symlink.

But here is what the HP-UX faq has to say: "Transition links are a bit faster, because the linked-to filename is stored in the inode itself, instead of using an allocation unit to store the link." While that statement is not exactly false, it is terribly misleading. HP-UX simply supports fast symlinks. If the referenced file name is short enough, it is stored directly in the inode. The transition links happen to be short enough. I have seen some people use lchmod() to turn on the sticky bit of a symlink in an effort to speed things up. It does not work like that.

Transition links are starting to disappear. People who have not switched from /bin/ksh to /usr/bin/ksh may one day get a rude surprise. Below are some links to HP's website.

Maintaining Transition Links
Transition Links Commands (Deprecated)
Transition Links (Deprecated)