Controlling AFS

Working in an AFS environment

Further information

AFS Overview

AFS is an acronym for the Andrew File System, developed at Carnegie-Mellon University, Pittsburgh, under a sponsorship from IBM. For years, AFS has been maintained & sold by the Transarc Corporation, now part of IBM. Today, AFS has been "opened" by IBM, and is mainly maintained & distributed by the OpenAFS consortium.

The AFS global name space

Files and directories stored in the AFS file system are stored in logical entities called volumes. A volume physically resides in parts of a file server machine's physical storage (i.e.disk's) called partitions. The information about which volume is located where and on which file server is managed by volume location server processes running on servers inside the AFS cell (see below).

The mapping information of files/directories to volumes ("mount points") is contained in the file system itself and therefore the same for any client traversing the AFS file space. Through adequate construction of mount points it is possible to reference each file in AFS by a path name which will be the same for every AFS client machine in the world.

In order to keep path names unique, an early component of the path name is the cell name. An AFS cell is a collection of AFS servers and clients. The cell is an administratively independent domain, and among others the boundary for the validity of AFS userids (see below). A typical pathnames of an AFS file is


Access to AFS files is location transparent. System processes resolve each file's physical location automatically based on information contained within the file system. The user does normally not even know how many servers are in a cell. Files can sometime exist in multiple copies on several servers for load balancing and availability purposes, with the AFS cache manager choosing a copy based on availability and topological distance in the network.

Authentication and access control

AFS uses Access Control Lists stored with each directory, which allow fine specifications.

Users are identified to AFS using Kerberos authentication. AFS user identifications are global to the whole cell, and not necessarily related to the local authentication on the user's workstation. Authenticated users hold Kerberos tickets for the service 'afs', in AFS called a token.

Simply presenting a valid token is already proof of authenticity. In order to limit the damage in case of accidental disclosure tickets are therefore protected by an expiry date - after that date the ticket or token is no longer accepted. The usual lifetime for AFS is 25 hours, which forces the user to re-authenticate every day.

Based on the user's authentication, connections to AFS file servers are automatically established when needed, using complex Kerberos mutual authentication. Although data is exchanged in clear, control information between a user's workstation and a fileserver is encrypted using a key shared only between the workstation and the fileserver, making it virtually impossible for another node on the network to masquerade as the user, or to insert himself into an ongoing connection and act on behalf of the user.

Controlling AFS

AFS commands

The most important command are

to obtain an AFS tokens
interacts with the host's AFS cache manager; this command is used to    manage Access Control Lists, obtain information about AFS directories (e.g.    mount points, AFS volume where the directory resides, quota), and    control/configure various variables in the cache manager (e.g. cache size,    server preferences).    
lists all afs tokens the user currently holds
destroys AFS tokens

Example: in order to find out on which AFS file server a file resides issue the command:

fs whereis filename

Example: an alternative and more verbose method is:

fs examine filename
vos examine 'AFS volume', with the 'AFS volume' name returned by the previous command.

Online documentation

All commands provide online help about command arguments, displayed when using the -help switch.

fs help
fs liste -help

AFS Access Control Lists

Access Control Lists (ACLs) are used by AFS to control access to files residing in the AFS file system. ACLs are defined for a directory and apply to all files immediately residing in that directory. An ACL is a list of userid-access right pairs. A userid is either an individual user or a named collection of users called a protection group, in which case the rights apply to all users in the group.

AFS defines the following access rights:

reading of files    
writing to files    
locking of files (flock() system call)    
the right to search for files in the directory, i.e. list which files it   contains    
creation of new files or directories    
deletion of files    
the right to change the ACL 

Mnemonic rights formed from those above are

entry deleted from access list. This does not mean that the user has no rights, since other ACL entries may still apply
r+w+k+l+i+d, i.e. everything except 'a'

Access rights for a directory can be examined and defined using the fs listacl (or just fs la) and fs setacl (or fs sa) commands.


    > fs la /afs/fysik.su.se/home/r/a/rahman/public
    Access list for /afs/su.se/home/r/a/rahman/public is
    Normal rights:
      system:anyuser rl
      rahman rlidwka
    > cd /afs/fysik.su.se/home/r/a/rahman
    > fs la ./Mail
    Access list for ./Mail is
    Normal rights
      rahman rlidwka

    This means that anybody may read files in /afs/fysik.su.se/home/r/rahman/public but that only rahman can write to and create files there, whereas /afs/fysik.su.se/home/r/rahman/Mail is completely private to rahman.
    > fs sa /afs/fysik.su.se/home/r/a/rahman/public rub write
    > fs la /afs/fysik.su.se/home/r/a/rahman/public
    Access list for /afs/su.se/home/r/a/rahman/public is
    Normal rights:
      system:anyuser rl
      rahman rlidwka
      rub rlidwk

    Here the user rub is granted write rights for the directory /afs/fysik.su.se/home/r/rahman/public, which enables him to create, alter and erase files, but not to alter the ACL.
    > fs sa /afs/fysik.su.se/home/r/a/rahman/public rub none
    > fs la /afs/fysik.su.se/home/r/a/rahman/public
    Access list for /afs/fysik.su.se/home/r/a/rahman/public is
    Normal rights:
      system:anyuser rl
      rahman rlidwka

    Here the user rub looses any rights for the directory /afs/fysik.su.se/home/r/rahman/public. This is the right way to remove someone from ACLs.

    Protection groups

    Protection groups are defined by the system or can be can be defined by users themselves using the pts creategroup command. Special predefined protection groups are

    just any AFS user in the world    
    any AFS user with a valid Kerberos token for the local cell    
    all machines at Fysikum (based on IP addresses)    
    all machines of the group 'gg'   
    AFS space administrators for group 'gg'    
    all registered members of group gg 

    Users are encouraged to use protection groups whenever there is a need to control access rights for even a small number of users. Any user can create a protection group and then place the group on an ACLs as though it were an individual user. Using groups simplifies maintenance considerably since whenever there is a change of membership of the group, the change needs only to be made in the definition of the group and not on each directory concerned.

    Using protection groups rather than individual users is also a security feature.

    Protection groups are managed with the pts command, for example

    pts creategroup -name gaston:myfriends
    fs sa ~gaston/photos  gaston:myfriends rl

    issued by user gaston would create a protection group called gaston:myfriends, owned by gaston and give access to directory "photos".

    Any user who is a member of the group xyz can create a protection group with a name of the form xyz:abcd. For example

    pts creategroup -name gg:production -owner gg

    would create a protection group called gg:production, owned (and thus modifiable) by members of the protection group gg. Remember that gg is the protection group containing the AFS space admininistrators of the computing group GG.

    The membership of a protection group is modified with the commands pts adduser and pts removeuser.

    pts adduser -user rahman gaston gabri  -group snova:members

    would add the userids rahman, gaston and gabri to the protection group called snova:members.

    pts removeuser -user gaston -group snova:members

    would remove userid gaston.

    The pts membership command can be used both to find the members of a group or to which groups a user belongs. Note that nesting of protection groups is not allowed. A protection group can not be a member of another protection group.

    pts membership -name fysik:mol

    would list the members of protection group fysik:mol.

    pts membership -name gaston

    would list the protection groups of which gaston is a member.

    AFS volumes

    Volume quotas

    AFS volumes are containers for files and directories. A volume is a certain amount of space allocated on a physical disk on some file server. A volume shrinks and grows to accommodate the data it contains, but a volume has a quota, i.e. a maximum size assigned beyond which it will not grow. Quotas are assigned by system administrators. The current space usage and the quota can be examined using the fs listquota command, which takes the directory pathname as an argument:

    > fs listquota /afs/fysik.su.se/home/r/rub
    Volume Name            Quota    Used    % Used   Partition
    home.r.rub             5000      12        0%         82%

    The command also lists the amount of space available on the file server physical disk (partition). Indeed an AFS user can run out of space in both fashions: his volume quota may be exceeded, or there might not be enough space available on the disk. The latter can only happen in case the disk has been over-committed, i.e. the sum of all quotas for volumes residing on a disk exceeds the amount of space available on the disk. In both cases there is not much that he can do about it except starting to erase files or enter negotiations with the system administrator.

    Mount points

    Given that volumes cannot exceed the physical partition size, the world would be confined to one single disk if mount points did not exist: a mount point is a directory entry leading to another AFS volume. The mount point contains the name and cell of the AFS volume it points to. The volume can reside on a different disk on the same fileserver, on another fileserver or even on a fileserver in an entirely different cell. Whenever the mount point is crossed, i.e. is referenced as part of a filename (or a cd command), the new volume is automatically opened by the AFS cache manager, who contacts the volume location server in the volume\'s cell to discover the network address of the file server housing the volume.

    Mount points can be examined using fs lsmount. However, in order to just find out in which volume a file resides or on which fileserver, the commands fs examine, fs whereis and vos examine are more convenient:

    > cd /afs/fysik.su.se/home/r
    > fs lsmount rahman
    'rahman' is a mount point for volume '#home.r.rahman'
    > fs examine /afs/fysik.su.se/home/r/rahman
    Volume status for vid = 536870978 named home.r.rahman
    Current disk quota is 2000000
    Current blocks used are 1807774
    The partition has 15142888 blocks available out of 68811300
    > fs whereis /afs/fysik.su.se/home/r/rahman
    File /afs/fysik.se/home/rahman is on host afs48.fysik.su.se
    > vos examine home.r.rahman

    home.r.rahman                          536870978 RW    1807774 K  On-line
        afs48.fysik.su.se /vicepa 
        RWrite  536870978 ROnly          0 Backup  536870980 
        MaxQuota    2000000 K 
        Creation    Wed May 20 13:24:17 1992
        Last Update Fri Mar 28 08:27:47 2003
        1598 accesses in the past day (i.e., vnode references)

        RWrite: 536870978     Backup: 536870980 
        number of sites -> 1
           server afs48.fysik.su.se partition /vicepa RW Site

    The fs lsmount command not only returned the name of the volume the mount point rahman pointed to, a hash sign (#) preceding volume name indicates that directory is a regular mount point and a percent sign (%) preceding volume name indicates that directory is a ReadWrite mount point.

    AFS accounts and tokens

    In order to act on AFS files in a way that requires privileges, such like writing into file or reading files which are not readable by just anybody, a user has to hold an AFS token. An AFS token can be obtained automatically during login if the system has been set up accordingly, based on the user\'s AFS password. In order to find out which tokens have been created the tokens command can be issued.

    > tokens
    Tokens held by the Cache Manager

    User's (AFS ID 1704) tokens for afs@fysik.su.se [Expires Feb 27 16:51]
       --End of list--

    Obviously this user holds a token as AFS user 1704. The token management uses numeric user ids just like the Unix system, although they might be different.

    A token expires automatically after a certain time, or the user may decide that he wishes to change identity (assuming he owns several accounts): the aklog command can then be used to obtain a token.

    > aklog

    The command will not login/logout him of the Unix system, it will only change his current AFS token. A token can be discarded with the unlog command.

    At any given time a user can only hold one(!) AFS token per cell, but is is posible to be authenticated to many cells, say, 'cern.ch' and 'su.se' at the same time.

    Ticket granting tickets

    The command aklog requires the the user already is in possession of a valid Kerberos ticket granting ticket. This can be obtained with the kinit command:

    > kinit kalle@SU.SE
    Password for kalle@SU.SE:

Working in an AFS environment

The AFS file system differs from the standard Unix file system in several aspects, with more or less impact on the average end user.

The most common problem users face is missing authentication. AFS tokens are required for access to data that is not publicly accessible. Tokens are stored in the operating system kernel and are normally linked to the user's process tree, however the link between the user's process and stored token(s) can be modified by commands like aklog and unlog, and, in some cases, even by an implicit (and therefore difficult to detect) change of Unix UID.

Most of these problem can be avoided by regular log-out (e.g. before going home).

In case of permission problems, the tokens command should be used to find out what tokens have been acquired and which are still valid. the pts membership command tells to which groups a user belongs, and the fs listacl command should be used to find out which permissions are required.

What to do when the token has expired

What can a user do when he cannot access a file because the token has expired? The most appropriate is to issue the aklog command, which simply acquires a new token. All windows in the current session, i.e. normally all windows on an X screen except those in remote sessions will use the new token.

When this happens in an editor session - don't panic! If another window within the same session is open where you can issue the aklog command, this should help. Alternatively, many editors allow opening up a temporary shell from within the editor; a aklog issued in such a subshell will access the same PAG and is therefore all that is required.

Unix access mode bits

File access is determined by examining the Access Control List for the file's directory. If the user accessing the file passes this test, the Unix access bits for the owner are examined to determine the access granted. Group access bits are ignored.

Setuid/Setgid programs

Setuid/setgid programs are supported in AFS, but the chown function to set the owner/group of a file requires special privileges which users will obviously not be granted. If they were, users could create setuid root programs on their own workstation and become root on any other station within the local cell on which they have an account.

If setuid/setgid programs are required, they should reside on the local file system of the workstation where they are intended to be run. If required, symbolic links can be set up within the AFS name space for ease of use.

Cross-directory hard links

Hard links inside AFS can only be created in the directory where the target resides. It is not possible to create a hard link from an AFS file into another AFS directory (nor into another file system).

Execute-only permission on files

In AFS it is not possible to execute programs that do not also have 'read' access, i.e. files with only --x attributes.

The reason for this is obvious: the AFS fileserver cannot trust the requesting machine's operating system (since it does not have an identity, only the user has), and a request of the type 'please pass me that file, and I shall execute it but I promise not to look at it' does not make sense.

Architecture-dependent directories

AFS has been designed to be usable on machines of different system flavors and even CPUs with different instruction sets. Consequently, users working on machines of different architectures are faced with the need of having to compile programs several times, and often to store them in appropriately named places so that surrounding applications which are common to all architectures, e.g. shell scripts, can easily locate them.

To this effect AFS introduces the special name @sys. If used as the last characters of any but the last element of a path name (in other words, it is followed by a slash!), it will automatically be replaced by the current AFS architecture string.

Example: on a PC Linux box running RedHat7.3 the path $HOME/.@sys/bin is equivalent to $HOME/.i386_redhat73/bin. The same path name $HOME/.@sys/bin on a Mac OS X system would automatically translate into $HOME/.ppc_darwin_60/bin. This can be used to access programs for different architectures under the same name regardless of the architecture, e.g. by setting up a directory per architecture and appropriate symbolic links:

> cd
> mkdir -p .alpha_dux40/bin
> mkdir -p .i386_redhat90/bin
> ln -s .@sys/bin bin

Now the following works on both Solaris and Linux, with nobody "stepping on the other's feet":

on Solaris:
> cc -o bin/helloworld helloworld.c
on Linux:
> cc -o bin/helloworld helloworld.c
on both:
> bin/helloworld

The architecture string is compiled into the AFS kernel and varies according to the processor type and operating system characteristics.

The architecture string for the machine you are working on can always be queried with the fs sysname command.

PAG - Process Authentication Group

AFS tokens are valid for all processes in a Process Authentication Group (PAG), or, if no PAG has been established, for all processes running under the same Unix UID as the last issuer of klog outside a process authentication group.

Normally a PAG is established for every session initiated by login. Processes sprung off the PAG-creating process (i.e. its children) inherit the same PAG.

Any child can modify the AFS tokens within a PAG, and the modification applies to the whole 'family'. Using aklog on a graphics display with twenty-three windows open in six different workspaces can therefore have surprising effects, the reason for which is not immediately obvious.

However, the effect can be even more drastic and difficult to debug when tokens are associated with Unix UIDs instead of a PAG: in addition to any process running with the same UID being able to change the token (and not only processes in the same 'session'), a setuid root command (such as lpr on some systems) will not inherit the user's tokens (unless he runs as root) and can therefore not access his files.

Temporarily assuming the identity of another user

The aklog can be used to acquire a token for a different userid. However, in a windowed system with several shells and processes running using the same token, this approach can be have significant side-effects.

Further information

As of writing, the AFS documentation from the OpenAFS can be found here.