Navigation:

Search



Related Articles

Our Friends

Articles Network Filesystems
 

Network Filesystems

Describes the options for accessing files remotely, via a Linux network filesystem.

This was written by James Crasta and given on Tue Jan 07 2003.

Table of Contents


1. Overview of Network Filesystems

Network filesystems are used to allow a machine to access another machine's files remotely. There are a plethora of ways to do this in Linux. The most common way today is NFS, however many people are shifting to different methods because of either security or administration reasons.

2. NFS
2.1. Background

NFS was originally developed by Sun for the purpose of mounting a disk partition on a remote machine as if it was on a local hard drive. this allowed for fast seamless sharing of files across a network. NFS is an RPC service, and works through the RPC portmapper daemon. Most unix-like systems ship with an NFS daemon included, or a NFS daemon can be obtained, which is why it is the de-facto standard for network file sharing.

The problem with NFS is that it is inherently insecure. The current NFS protocol does not support any authentication, and any validation is only done by IP, which can be easily spoofed. There is no user-level restriction, so if anyone can exploit a client machine on a NFS network or in some cases just plug another machine into the network, they can gain access to the NFS server's shares.

Linux supports NFS protocols v. 2 and 3. the implementation of NFS in the linux kernel is partly user-space, and partly in the kernel space, in order to keep things running fast.

2.2. Server Setup

To set up a server, one must have the portmap and nfs-utils installed (see resources for links). To configure your exported filesystems, edit the file /etc/exports. for example, if your fileserver BARNEY wanted to share folders /mnt/mp3 and /mnt/work with system JOHN, and also share /mnt/work with systems MARY and BETTY, with betty also having read-only access to /mnt/mp3, your exports file would look like this:

/etc/exports:

# Shares on BARNEY
/mnt/mp3 JOHN(async,rw) BETTY(async,ro)
/mnt/work JOHN(async,rw) MARY(async,rw) BETTY(async,rw)

As you can see, you merely specify the directory you want to share, and then follow it by the hosts you want to be able to access it (IP's are fine too) and the permissions for that user. Now you are ready to share your files. start portmap, and then start the nfs service (rc scripts are included with most distributions to do this)

2.3. Client setup

To be a NFS client, you don't need to edit any config files. all you need to do is start portmap, and then provided you have the nfs tools installed, simply mount the partitions. Let's say JOHN from the previous example wanted to mount the mp3 and work folders now. he simply runs at the command line:

# mount -t nfs BARNEY:/mnt/mp3 /mnt/mp3
# mount -t nfs BARNEY:/mnt/work /mnt/work

It is very simple to mount NFS partitions. because no further authentication is needed, on a fixed network with fixed hosts, one can simply set a bootup script to mount these on startup.

2.4. Secure NFS over SSH

Okay, so that's great, now what happens when John is on a business trip in Paris and wants to mount his network share to grab some files he forgot for a presentation he's doing? It's not very smart to set up the NFS server to allow connections from outside IP's, and even if it was, it would be even stupider to actually access it, since all the data goes in the clear, and can be inspected at any point by anyone with half a brain. Well, John uses SSH to get into a secure shell, so why not use it to forward nfs? Well, NFS uses UDP, and SSH can only forward TCP, it does not know what to do with UDP datagrams. Enter SNFS and sec_rpc. How does it work? you ask. sec_rpc basically translates the UDP datagrams into something that SSH can forward, and then translates them back on the other side. so you're still using NFS, but through a tunnel.

Here's how you start out. on the server, you create an /etc/exports file like so:

/etc/exports:

# SSH exports file
/mnt/mp3 localhost(async,rw)
/mnt/work localhost(async,rw)

Now that you've exported the filesystems to the local host, you need to install sec_rpc on both the client and server machines. you do it the standard autoconf way, ./configure; make; make install. so now on BARNEY you need to be running nfsd like normal.

On John's pc is where all the complex stuff is happening. John runs at his PC:

# snfshost REMOTE:MOUNTPROG

Here MOUNTPROG is a six-digit number chosen between 200000 and 249999 such that both MOUNTPROG and NFSPROG=MOUNTPROG+50000 are unassigned RPC program numbers (e.g. MOUNTPROG=201000, NFSPROG=251000). REMOTE is the remote host name.

That will create the config file needed for the RPC numbers. now you add a line to john's fstab:

/etc/fstab:

LOCAL.DOMAIN:/DIR /REMOTE/DIR nfs user, noauto, hard, intr, rsize=8192, wsize=8192, mountprog=MOUNTPROG, nfsprog=NFSPROG 0 0

(sorry if that wraps weird, it should go on one line) in that case above, MOUNTPROG and NFSPROG are the numbers you figured out before. LOCAL.DOMAIN is john's fully qualified domain or localhost.

Now that John is thoroughly confused, he creates the local mount directory, starts portmap, and now has some more weird commands to throw at the host:

# smkdirall
# rpc_psrv -r -d /usr/local/etc/snfs/REMOTE
(where REMOTE is the remote host name)

Now john types in the root password for BARNEY, and he should get the message: "ID String correctly read: RPC Proxy Client" . Now he is ready to mount:

# mount /REMOTE/DIR
# df

woo, you have a mount or something. And now you are thorougly confused. As you may have noticed, a major disadvantage of SNFS is that you need to know the host's root password. also, you need to have remote root SSH enabled. Not to mention, this is a messy setup. but it works (sorta).

It is possible to set it up so that the remote mount runs as non-root, with the correct setuid binaries, but that is still messy. The people at SFS agree with you. the next section will show you how to do it using SFS, a more elegant solution.

3. SFS
3.1. Background

SFS was created because the insecurities of NFS made it easily spoofed and not very insecure. not to mention, if a client was connecting with a dynamic IP, the NFS would have to change its exports as per each time a client's ip changed, or (very insecure) export it to an entire IP range.

SFS version uses shared-key authentication at 1024 bits default (higher bit keys are allowed)

3.2. Server set-up

To set up an SFS server, you create a user sfs and a group sfs, and then get the sfs-0.6 source from www.fs.net and compile it. then you set up your /etc/exports to export filesystems to localhost similar to like you would SNFS:

/etc/exports:

/var/sfs/root localhost(async,rw)
/mnt/mp3 localhost(async,rw)
/mnt/work localhost(async,rw)

Then you create the file /etc/sfs/sfsrwsd_config as follows:

/etc/sfs/sfsrwsd_config:

Export /var/sfs/root /
Export /mnt/mp3 /mp3
Export /mnt/work /work

note sfs requires a 'root' for the server. the exports will be under this root. you now create /var/sfs/root , /var/sfs/root/mp3 , and /var/sfs/root/work. then chown these to the user sfs group sfs. you generate a host key for the server like so:

# sfskey gen -P /etc/sfs/sfs_host_key

After doing this, start your nfsd, and sfssd.

3.3. Client setup

Now John wants to connect to sfs. Here's what he does: on the server BARNEY (not his local client box) john creates a key from the shell like this: sfskey register . he can do this from an ssh shell if he wants to. it's a one-time process. Now what john does from paris is, he starts portmap and sfscd from his client box. Now as non-root, he runs:

# sfsagent john@barney.office.domain.com

sfs will not read your /etc/hosts file, it will totally ignore it (to prevent spoofing, only a globally-verifiable DNS name will be accepted) now john logs in with the passcode he put on his key, and now he is into barney. he now proceeds to access his share:

# cd /sfs/barney.office.domain.com/
# ls
    work/
    mp3/

All his files will be read over a 1024-bit secure encrypted channel. SFS is probably the best way for sharing files over an insecure network, because it uses the tried-and-true nfs protocol, and once a key is made, john can log in from anywhere.

4. OpenAFS
4.1. Introduction

AFS was pioneered at Carnegie Mellon University, supported and developed by TransArc corporation (now owned by IBM). AFS, or the Andrew File System, is a distributed filesystem. IBM in 2001 branched out AFS and created OpenAFS, an open source version of AFS. OpenAFS is unique among this bunch in that it works on windows systems as well as Linux.

Because openAFS is a distributed filesystem, it allows for high availability. a cluster of servers can mirror the same AFS cell. Any client will not know which server they are connected to, and when they write to a file, the changes are propagated to the other servers in the cell. if any server goes down, the cell can continue to operate.

4.2. Server setup

Gentoo has a good guide on how to set up OpenAFS. I will put up my own guide to OpenAFS setup soon.

4.3. Client setup

Check for the next version of this howto to learn about OpenAFS client setup.

5. Coda

Coda is another distributed filesystem from carnegie-mellon based on AFS. Coda was open source from the beginning however, and it has been available in the Linux Kernel since 2.2. Coda has one important feature which NFS and OpenAFS do not, and that is disconnected operation. disconnected operation allows a client to disconnect from the network, and continue to access and modify files. when the client reconnects to the network, that user's changes are propagated back to the coda cell. This is excellent for laptops, because then a user can take the laptop away, modify his own home directory files, and then put it back on the network and have all the files be resynched with the global fileserver. Coda does this by locally caching files. Because of aggressive local caching, Coda is also faster than OpenAFS.

One downside of all these features is that, since Coda needs to keep so much metadata on the filesystems, it needs its own raw partition for data storage.

Check back for the next version of this Howto for more info on setting up coda.

6. Intermezzo

Intermezzo is another distributed file system, loosely based on coda. Like coda, it allows disconnected operation. Intermezzo is meant to be lightweight. It also needs a dedicated raw partition, which can be any of ext2/3, xfs, or reiserfs. The intermezzo protocol is based on HTTP, and it can use apache or any cgi-supporting http daemon for synchronization.

Intermezzo is still in its early stages. However, linus made sure it was included in the 2.4 kernel starting with 2.4.14, just before he went off to work on the 2.5 kernel. Intermezzo is still in early beta, but it shows a lot of promise.

In my tests with Intermezzo, i found it is a little flaky in its setup, and the setup process is not very well documented. Whatever documentation exists is inconsistent with the most recent version.

7. Summmation

There are many alternatives to NFS. NFS is a very good protocol, and it has been tried and tested, but it is too insecure to be used over wide area networks, unless encapsulated inside a VPN tunnel, or another more secure protocol such as SNFS or SFS is used. Also, NFS is limited in that a volume can only be exported by one server, so that you cannot distribute the load across multiple servers or implement failover. Coda, Intermezzo, and OpenAFS aim to leverage that by distributing the filesystem. As of now, Coda looks to be the best choice for a distributed FS, although IBM is working hard to improve OpenAFS. Intermezzo is a little too unstable to be considered usable, and not enough documentation exists.

Which one you choose is dependent on how you want to do things. If you have a home directory you want to access when on the road and not connected, then a caching system like Coda is probably for you. However, if you simply want to access files from two connected systems over the net (over Resnet or LAWN for example), you probably want to use SFS as SNFS is a little tricky to setup, and SFS allows connection from anywhere with the same public key using strong encryption.