Notes on F2FS

F2FS is quite a niche file system on Linux. It is aimed at flash storage with low endurance.

F2FS is a mostly log structured file system. This means that it writes to storage in a linear fashion (not random).

It does some random writes at the beginning of the partition to the checkpoint and metadata areas. It also does random writes when space runs low.

Features

• A full POSIX file system.

• File attribute support (cmF; see chattr(1)).

• Extended attributes support (acl and SELinux support).

• Fully Copy-on-Write.

• File inlining up to 3692B (-36B extra_attr).

• Auto trim

• Lazytime default

• Zoned storage support

• Can span multiple block devices

• Metadata checksums (if enabled).

• Compression (if enabled).

• FSCrypt support (if enabled)

• Classifies data and metadata by temperature (rate of change) to reduce write amplification

It does not support snapshots or reflinks despite being fully CoW. It does not checksum data.

Design

F2FS is quite simple and is structured much like pre UFS file systems. Just as a quick reminder: Such old FS start with the superblock followed by an inode array and a free space bitmap. The rest is reserved for data blocks. They use limited size block pointer trees to address file content. Directories are files containing simple lists of inode and file name pairs.

F2FS differs in the following aspects:
It features a checkpoint area which is used for journaling and the implementation of transactions. It addresses inodes indirectly. Inodes are whole blocks in the data area. This space is shared with block pointers which are also addressed indirectly. In addition to free blocks f2fs tracks temperature for segments. It also stores back pointers which are need for garbage collection. It structures the data area into segments which are written to linearly.

Block addresses are 32 bit. Inode allocation is static (use extended node bitmap for more on creation). Using whole blocks for inodes means the are very large compared to other file systems (XFSv5 512B, EXT4 256B, EXT3 128B). The large space is used for inlining of file content or direct data pointers.

It uses a copy-on-write scheme. Modifications are written to unallocated space. Metadata is then updated to point to the new location. Top level changes are recorded to the checkpoint area of which there are two. They are used in a rotating fashion so one is always valid.

Unlike most modern file systems it does not use extents and instead addresses each block of a file. An inode can actually store a single extent as a performance optimization.

It is the only file system (that I know of) that stores the file name in the inode. I don’t know how this interacts with hard links. This is used to classify initial temperature.

Structure

F2FS uses three levels to section up a disk. Most importantly the data area.

1. Zones
2. Sections
3. Segments

Zones are used for log allocation. Sections are used as gc units. Segments are 512 blocks. A block is 4kiB (can be made 16k). So a segment is 2MiB (or 8MiB).

Default max size is 16TiB for a file system (or 64TiB).

4kiB * 2**32 = 2*2*2**10 * 2**32 = 2**12 * 2**32 = 2**44 = 2*2*2*2*2**40 = 16TiB

Max file size is slightly below 4TiB.

It uses six log heads. Three for data and three for metadata. Each has hot, warm and cold separation.

A write is ranked according to the file name extension of the associated file. You can specify a list of hot and cold file extensions that are stored in the superblock.

Metadata

Static

F2FS has five main metadata structures.

• Superblock (SB)
• Checkpoint (CP)
• Segment information table (SIT)
• Node address table (NAT)
• Segment summary area (SSA)

SB, CP, SIT and NAT exist twice. SB and CP are used in a rotating fashion to grantee consistency. SIT and NAT entries are rotated block wise by a checkpoint bitmap. SSA is journaled in the CP. SIT and NAT are journaled if changes are small for better performance.

The SIT contains validity bitmap and temperature (74B pS). The NAT is used for indirect Node addresses (9B pN; 20% of blocks can be nodes with extended node bitmap). The SSA contains back pointers (one block pS).

Total usage of static metadata is

2MiB + 2*(X + (74B/2MiB * 1/8) + (1/455 * 0.2 * 1/8)) + (2 * 74B/2MiB) + (2 * (1/455) * 0.2) + (4kiB/2MiB) = ~0.3%

Dynamic

Node types

INodes Block pointers (direct, indirect and double indirect)

Use extended node bitmap to properly size the metadata sections on mkfs.

mkfs.f2fs -l <LABEL> -i -O extra_attr,inode_checksum,inode_crtime,sb_checksum,compression <DEV>

Use flexible_inline_xattr if you plan to use SELinux. So xattrs are inlined into the inodes. Use encrypt if you plan to use fscrypt.

extra_attr slightly reduces maximum file size.

Directories

F2FS structures its directories as multi-level hash tables. Each hash bucket is two pointers to blocks. Like a common hash map it grows by doubling its size. It just keeps the old ones around.

At a defined threshold it does not double the hash buckets but increases the bucket size to four.

Directory data blocks contain a fixed amount of slots for directory entries up to eight characters each. If the file name is longer more adjacent slots will be used.

Compression

Available algorithms are lzo, lz4, zstd and lzo-rle. You can choose a level for lz4 and zstd. I recommend zstd with a medium level.

With the mount option compress_log_size=%u you can increase the compression chunk size. Default for a value of 2 is 16kiB. Each increment will double the chunk size. Raising this value significantly increases the effectiveness of compression.

In f2fs compression does not free up space. It just reduces writes.

Example mount options:

compress_algorithm=zstd:5,compress_log_size=4,compress_chksum

You can manually enable compression for a file or a whole directory (best).

chattr +c dir && touch dir/file

When setting up a root file system you should set this attribute on /bin, /usr/bin, /usr/local/bin, /sbin, /usr/sbin, /lib, /usr/lib, /usr/local/lib etc. All directories which can contain many files that compress well but not files that are written to after creation. So for log files the compression flag is useless. Writes to any part of a file will cause that chunk to be decompressed.

You can actually free spaced saved by compression if you make a file immutable. Truncate to 0 is allowed so not fully immutable.

f2fs_io release_cblocks <FILE>

Here is an example to show the effect of compression. Take note of the extends an their starting position. They are in fact not 16 blocks in size.

pi@raspberrypi:~ $ filefrag -v /bin/ls
Filesystem type is: f2f52010
File size of /bin/ls is 199576 (49 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..      15:    1132764..   1132779:     16:             encoded,merged
   1:       16..      31:    1132772..   1132787:     16:    1132780: encoded,merged
   2:       32..      47:    1132781..   1132796:     16:    1132788: encoded,merged
   3:       48..      48:    1132784..   1132784:      1:    1132797: last,merged,eof
/bin/ls: 4 extents found

GC

There are two GC algorithms: foreground and background gc.

The background runs every two minutes when load is low. It uses segment temperature to pick sections with blocks that are unlikely to be rewritten. With the mount option atgc you tune the background algorithm to heavily favor very old sections (older than a week). The goal is to reduce write amplification. This is particularly good for long running systems with not much activity.

The foreground gc is triggered when space runs low and picks the emptiest section. It is designed to quickly free as much space as possible.

With merge_gc you can serve foreground requests by the background gc.

Manual running the gc

f2fs_io gc_urgent <DEV> run <SEC>

In low space conditions (95% full) data is written to non-full chunks (random writes). Metadata is always written sequentially.

With this command you can see some stats:

grep . /sys/fs/f2fs/mmcblk0p2/{gc_foreground_calls,gc_background_calls,\
       gc_reclaimed_segments,moved_blocks_background,free_segments,\
       dirty_segments}

5% of drive space is reserved for the gc by default.

On my raspi

The following options are shown on mount:

Links

• https://docs.kernel.org/filesystems/f2fs.html
• https://github.com/torvalds/linux/blob/master/include/linux/f2fs_fs.h
• https://github.com/oldoldman/f2fs-design