Case: gitlab-runner ran out of disk space

Investigate which directory consumed all the space:

• df -h
• cd / && du * -sh

You’ll see something like this

0 bin
34M boot
0 dev
31M etc
598M home
0 lib
0 lib64
0 local
0 media
0 mnt
116K opt
0 proc
32K root
464K run
0 sbin
0 srv
0 sys
0 tmp
1.5G usr
385M var

In my case unused docker images consumed most of the free space

Mitigation

• run docker system prune -af
• add a cron job running the command above daily

crontab -e

0 0 * * * /root/docker-clean.sh (runs at 00:00 daily)

• check cron logs tail -f /var/log/cron