What is ZFS?

Tagged:  

ZFS, (originally the "Zettabyte File System"), is a new filesystem and volume management software by Sun Microsystems and available as open-source since November 2005.

Here are some analogies to compare ZFS vs. traditional filesystem and storage management, to get an idea of how much of a leap ZFS is from previous technologies:

  • Broadband Internet vs. dial-up
  • Blogging vs. text-editors+FTP
  • Email vs. FAX
  • CSS vs. table-based layout
  • Digital cameras vs. film cameras
  • ZFS vs. older filesystems and storage management
  • ...you get the idea

This list refers to the consumer market. In the enterprise market, it's not especially difficult to pay tens or hundreds of thousands of dollars to get some high-end software or gear. For example, before consumer broadband was available, companies might pay $1500/mo for a 1.5 Mbs T-1 connection, but it was unheard of to have this at home. The true market shifts don't occur until these capabilities are available to the general public.

Each of these game-changers, when introduced, removed at least one major bottleneck in a particular industry. Whether it was for the better or not is up to you to decide. For example, "Email vs. FAX". Spam-faxes were a common nuisance before email became the messaging platform of choice. But sending FAX messages are relatively slow, even if you demon-dial a slew of FAX numbers. Email spammers on the other hand, could scale to millions and billions of messages, and (unfortunately) use the resources of others (unwitting open relays, compromised home PCs, etc.) to do their bidding. Spam has become an entire cottage industry both for those fighting it and those producing it.

Likewise, ZFS removes many traditional bottlenecks in computer filesystems:

  • Scalable: ZFS is a 128-bit filesystem, meaning it can store up to 2128 data blocks in a single storage pool. The original "zettabyte" moniker was really a misnomer. A zettabyte is only 70-bits or 270. They just called it "Zettabyte File System" since it sounded cool with the Z-name. 128-bits is not infinite, but it's more than you could likely ever use. For all practical purposes ZFS can hold as large a file, as many files, as many separate file systems, on as big a volume as you could ever cobble the drives together to support. Granted, current filesystem that "only" go up to 64-bits, can still store a lot. A terabyte is 40-bits, 240, and a thousand terabytes, a petabyte is 50-bits.
  • Robust: Although techies can debate the particulars, in ZFS uses a method of writing new data to the disk such that any data blocks that contain active data are never overwritten in place; instead, a new block is allocated, and modified data is written to it. This way if anything happens to interrupt a write-operation (power outage, etc.) there is essentially always a way to rollback the transactions (read or write operations). No more "fsck", no more "Disk Utility", etc.
  • Ease of Use: Although this is mostly of interest to system administrators, the ease with which ZFS let's you manage filesystems and storage volumes that it is at least an order of magnitude more efficient in terms of labor. This labor bottleneck is one thing ZFS is extremely good at tearing down.
  • Snapshots: Users of VMware may be familiar with the concept of snapshots - take a snapshot in time of your filesystem and roll back to any previous version. ZFS makes snapshots of entire disk volumes as simple as copying a file, and just as quick (literally a few seconds or less).

Today, without ZFS, you could spend tens of thousands on fancy storage arrays and controller cards and dedicated servers to get the same functionality. But why bother when you can just use ZFS. We already do this in our work environment. Instead of dedicated storage hardware, we can use the Sun X4500 disk server, which is just a server filled to the brim with drives. ZFS takes care of the rest.

Just as broadband Internet made video streaming possible, ZFS makes lots of storage capabilities possible. Once Mac OS fully supports ZFS, you can expect to see some amazing new ways to manage your data storage. Time Machine, which does not currently use ZFS, could be easily done with ZFS and much more. With snapshots and cloning (take a snapshot and make a completely separate writeable copy), you could rollback your entire computer, and not just individual files, to a specific point in time.

Anyway, this is just a quickie description. You can read lots more about ZFS at Wikpedia or at OpenSolaris.org.