Getting a disk usage alert on a production server is one of those moments that demands immediate action. If the disk fills completely, the website goes down — no logs can be written, no uploads accepted, databases can’t write. So when I see that alert, I follow a systematic chain of commands to find the culprit fast.

This post walks through my exact process.

---

The Alert

Disk usage at 95%. The site is still up but barely. I have maybe a few hundred megabytes of breathing room before things start breaking.

First thing — don’t panic and don’t start deleting files randomly. Find the cause first.

---

Step 1: Confirm the Situation with df -h

df -h

Output:

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        40G   38G  2.0G  95% /
tmpfs           1.9G     0  1.9G   0% /dev/shm

df shows me disk usage at the filesystem level. The root partition /dev/sda1 is 95% full. Everything else is fine. So I know exactly where to dig — the root filesystem.

---

Step 2: Find the Heaviest Directory with du

df tells me the filesystem is full. du tells me where the space actually went.

du -h --max-depth=1 /

Output:

128M    /home
18G     /var
12G     /usr
1.2G    /opt
38G     /

/var at 18GB is the obvious suspect. Everything else is a reasonable size. I drill down.

du -h --max-depth=1 /var

Output:

16G     /var/log
1.2G    /var/lib
512M    /var/cache
18G     /var

/var/log is consuming 16GB. Logs have grown completely out of control. One more drill down.

du -h --max-depth=1 /var/log

Output:

12M     /var/log/nginx
128M    /var/log/mysql
15G     /var/log/php8.3-fpm.log
512M    /var/log/journal

There it is. php8.3-fpm.log is 15GB. A single log file has consumed most of the disk.

---

Step 3: Understand What’s Inside the Log

Before deleting or truncating anything, I want to know what caused this. A 15GB log file means something was throwing errors at a massive rate — possibly thousands per second.

tail -n 50 /var/log/php8.3-fpm.log

I use tail not cat. Never use cat on a multi-gigabyte file — it will flood your terminal and tell you nothing useful. tail shows the most recent lines which is exactly what I want.

I see PHP errors repeating thousands of times — in this case the php-fpm worker pool was exhausted and each failed request was being logged. This is the root cause. Fix that and the logging stops.

---

Step 4: Free the Disk Space Safely

Now I need to clear the log file. The key word is truncate, not delete.

Why not delete? If a process (php-fpm in this case) still has the file open, deleting it removes the directory entry but the process keeps holding the file handle. The space never gets freed. The file becomes a ghost — invisible to du but still consuming disk space.

Truncating sets the file size to zero while keeping the file in place, so the running process continues writing to it without any disruption.

sudo truncate -s 0 /var/log/php8.3-fpm.log

One important thing — this common command does NOT work:

sudo echo "" > /var/log/php8.3-fpm.log  # WRONG

The redirect > runs as your user, not root, so you get a permission denied error even with sudo. The correct alternative:

echo "" | sudo tee /var/log/php8.3-fpm.log

tee runs as root and handles the redirect correctly.

After truncating I run df -h again to confirm space is recovered:

/dev/sda1        40G   23G   17G  58% /

Crisis over. But this is only the immediate fix.

---

Step 5: Fix the Root Cause — Configure Logrotate

Clearing the file stops the bleeding. But if I don’t set up log rotation, this file will grow back to 15GB within days.

I check if logrotate is already configured for php-fpm:

ls /etc/logrotate.d/

Output:

apache2  apt  dpkg  mysql-server  nginx  rsyslog  ufw

No php-fpm entry. That’s exactly why this happened — nobody configured log rotation for it so it grew unchecked.

I create /etc/logrotate.d/php8.3-fpm:

/var/log/php8.3-fpm.log {
    daily
    rotate 7
    compress
    missingok
    notifempty
    create 0640 www-data adm
}

What each directive does:

• daily — rotate the log every day
• rotate 7 — keep 7 days of old logs, delete anything older
• compress — compress old logs with gzip to save space
• missingok — don’t error if the log file is missing
• notifempty — skip rotation if the file is empty
• create 0640 www-data adm — create a fresh empty log file after rotation with correct permissions

Then I test the configuration:

sudo logrotate --debug /etc/logrotate.d/php8.3-fpm

--debug runs a dry run without actually rotating anything. It tells me if the config has any syntax errors before I rely on it in production.

---

The Full Chain

Looking back at this investigation, the chain of commands was:

Alert → df -h → du -h / → du -h /var → du -h /var/log → tail log → truncate → logrotate

Each step narrows the search. df gives the filesystem, du gives the directory, tail gives the content, truncate fixes the immediate problem, logrotate prevents recurrence.

This is the systematic approach I follow every time. No guessing, no random deletion — just a clean drill-down from the top level to the exact file causing the problem.

---

Running Ubuntu 24.04 on a Hetzner CPX22 VPS. Stack: nginx + php-fpm 8.3 + MySQL + WordPress.