NOTE 004intermediate14 min read

Why df and du disagree

Reconcile filesystem allocation with files a directory walk can reach, then diagnose gaps without reaching for a cleanup command.

FIG. 04 / INTERACTIVE MODEL

Allocation and reachability are different ledgers

Switch scenarios to see why filesystem-wide df and directory-walking du can count different things.

Filesystem ledger10 / 16 blocksdf
Reachable tree8 / 16 blocksdu
Logical length8 / 16 blocksdu --apparent-size
  1. 00V
  2. 01V
  3. 02V
  4. 03V
  5. 04V
  6. 05V
  7. 06V
  8. 07V
  9. 08M
  10. 09M
  11. 10·
  12. 11·
  13. 12·
  14. 13·
  15. 14·
  16. 15·
  • V visible file data
  • O deleted-open data
  • H hidden by mount
  • M metadata / reserved
  • · free or sparse hole

Most allocated data is reachable by walking the directory tree; metadata still makes the totals differ slightly.

Text equivalent

df asks a filesystem for allocation totals. duwalks reachable names and sums reported blocks. Deleted-open files and data hidden beneath a mount remain allocated without being reachable from that walk. Sparse files reverse the intuition: their logical length can exceed their allocated blocks.

Figure note

Counts are a teaching model, not an ext4 layout. Metadata, reservations, compression, reflinks, snapshots, and copy-on-write behavior are filesystem-specific.

df says a filesystem has used 92 GiB. du finds 63 GiB under the directory you expected to explain it. Neither number needs to be wrong: the tools answer different questions.

df: How many blocks does this mounted filesystem account as used?
du: How many blocks belong to entries reachable by this directory walk?

Model boundary: The exact accounting is filesystem- and tool-specific. Allocation units, compression, reflinks, snapshots, quotas, reserved space, delayed allocation, and network storage can complicate both totals. The map illustrates why two valid scopes diverge; it is not an ext4 block-layout map.

Measure scopes before sizes

GNU df obtains space totals for mounted filesystems 1. GNU du walks named directory trees and sums usage associated with reachable entries 2. So compare them only after aligning:

  • the filesystem and mount namespace,
  • the starting path,
  • whether the walk crosses filesystem boundaries,
  • allocation versus apparent size,
  • units and rounding,
  • permissions and unreadable paths.

Begin with read-only observations:

findmnt -T .
df -hT .
df -i .
du -x -h --max-depth=1 . 2>&1 | sort -h

findmnt, -T, du --max-depth, and human-readable formatting are common Linux utilities, not a portable POSIX interface. Permission errors in du mean its walk is incomplete; do not discard the diagnostic lines. BusyBox 1.37 uses du -d 1 for a bounded depth and du -b for apparent size, and may not include findmnt; inspect --help before translating the diagnostic.

Allocation is not reachability

A directory maps names to inodes. An inode carries file metadata and points toward file data; it does not store the filename itself 5. du starts with names and follows that reachable graph. Filesystem accounting also includes allocated state that a walk may not see.

The ordinary gap includes directory and filesystem metadata, allocation-unit rounding, and tool conventions. A large or sudden gap suggests a sharper case.

Case 1: deleted, still open

Removing a pathname unlinks a directory entry. If a process still has the file open, the file remains allocated until the last reference is closed; only then can its space be reclaimed 3.

The result:

  • df still counts the allocated blocks.
  • A new du pathname walk cannot discover the removed name.
  • Restarting or signaling the owning process may release the file, but doing so is an operational decision—not a generic cleanup step.

Inspect without changing process state:

lsof +L1

lsof is optional and may need elevated visibility to show other users’ processes. On Linux, /proc/*/fd symlinks can provide the same kind of evidence, subject to permissions and races. Record the PID, file descriptor, filesystem, and owning service before deciding what to do.

Case 2: a mount hides earlier entries

Mounting a filesystem on a non-empty directory makes the previous contents invisible through that pathname until unmounted 4. The hidden data still consumes blocks on the underlying filesystem. Meanwhile, du /mountpoint walks the mounted filesystem now visible there.

findmnt -R /suspected/path
findmnt -T /suspected/path

Mount namespaces matter. A host process and a container can see different mount trees at the same textual path. Inspect from the namespace where the discrepancy was observed.

Caution: Do not unmount a live filesystem merely to “look underneath.” It can interrupt users and services. Reproduce the layout in a disposable namespace or schedule an operational procedure with ownership and rollback.

Case 3: reserved blocks and metadata

Filesystems allocate space for their own structures: inode tables, journals, allocation maps, trees, and other metadata. A pathname walk has no reason to attribute all of that to visible file content.

Some filesystems also reserve capacity. On ext2/3/4, for example, a configurable percentage of blocks can be reserved for privileged processes, affecting the available value shown to ordinary users; tune2fs documents that filesystem-specific control 6. Do not generalize ext settings to XFS, Btrfs, ZFS, tmpfs, or remote filesystems.

Read the filesystem type first:

findmnt -no FSTYPE,SOURCE,TARGET -T .
stat -f .

Then use that filesystem’s own read-only inspection tools and documentation.

Case 4: sparse and shared extents

A sparse file has a logical length containing holes. Reads from a hole produce zero bytes, but no data blocks need be allocated for that region. That separates apparent size from disk usage.

Caution: This bounded lab creates a sparse file in a fresh temporary directory. Its apparent length is 1 GiB, but normally only metadata consumes disk blocks. Run it only where a temporary entry is acceptable.

lab=$(mktemp -d)
printf 'lab=%s\n' "$lab"
truncate -s 1G "$lab/sparse.img"
stat -c 'logical=%s bytes  allocated=%b blocks of %B bytes' "$lab/sparse.img"
du -h "$lab/sparse.img"
du -h --apparent-size "$lab/sparse.img"

The two du values should differ substantially. Filesystems can also share physical extents through reflinks or snapshots, making “which file owns these blocks?” a many-to-one question rather than a simple sum.

Clean up only the printed lab file and directories:

rm -- "$lab/sparse.img"
rmdir -- "$lab"

Case 5: blocks are free, inodes are not

Many filesystems have separate limits for data blocks and inodes. A workload with millions of tiny files can exhaust available inodes while bytes remain. Conversely, a few large files can exhaust blocks with many inodes free.

df -h .
df -i .

If inode use is near 100%, searching only for the largest byte consumers answers the wrong question. Count entries in known application-owned trees with bounded, read-only tooling and retain permission errors. Avoid an unscoped walk from / on a production host.

Container layers are an extension, not an exception

OverlayFS presents a merged view built from lower and upper directory trees. A container-visible deletion can become a whiteout in the upper layer while lower data still exists. Copy-up can allocate upper-layer data when a lower file is modified 7.

That introduces at least three scopes:

  1. du inside the container’s merged mount,
  2. allocation in its writable upper layer,
  3. host storage used by images, snapshots, logs, and runtime metadata.

Container runtime “disk usage” commands can add their own ownership model. Compare the same namespace and backing filesystem before reconciling totals. Never delete files directly from a runtime’s storage directory while it is managing them.

Read-only diagnostic sequence

1. SAME VIEW?
   Record host/container, mount namespace, path, filesystem type, and units.

2. BLOCKS OR INODES?
   Compare df -h with df -i on that exact filesystem.

3. COMPLETE WALK?
   Run a filesystem-bounded du; retain permission and I/O errors.

4. DELETED-OPEN?
   Inspect lsof +L1 or authorized /proc file descriptors; map PID to service.

5. HIDDEN BY A MOUNT?
   Inspect findmnt from the same namespace. Do not unmount live storage casually.

6. FILESYSTEM FEATURES?
   Check metadata, reservations, snapshots, reflinks, quotas, and compression
   using the identified filesystem's documentation and read-only tools.

7. CONTAINER OR SNAPSHOT LAYERS?
   Compare merged, writable-layer, image, snapshot, and host backing-store scopes.

8. ONLY THEN PLAN RECLAMATION
   Name the owner, expected effect, service risk, rollback, and verification.

The diagnostic principle is simple: reconcile scope before acting on size. A gap is evidence about reachability and accounting, not permission to delete something.

Reference ledger

Primary sources

  1. GNU Coreutils manual — df invocation
  2. GNU Coreutils manual — du invocation
  3. Linux man-pages — unlink(2)
  4. Linux man-pages — mount(8)
  5. Linux man-pages — inode(7)
  6. Linux man-pages — tune2fs(8)
  7. Linux kernel documentation — OverlayFS