One Billion Files:
Scalability Limits in Linux FS
Ric Wheeler, Red Hat
- fsck exhausts system memory at less than a billion files
- ext4 rules, but Oracle's btrfs (MeeGo) is hot and xfs (SGI) and
zfs (BSD/Solaris) have supporters.
- Features: xfs, enterprise performance and scalability; zfs, native
support for encryption; btrfs, snapshotting and reversion.
- Observed dramatic implementation-specific effects at large scale:
- ext4 is slow where stat() must be run on each file
(e.g. "ls").
- Some FS do better with larger files, some with many files.
- Hype about btrfs largely justified.