Visual disk-usage analyser for docker images

Last update: Sep 01, 2022

Related tags

Overview

whaler

What? A command-line tool for visually investigating the disk usage of docker images

Why? Large images are slow to move and expensive to store. They cost developer productivity by lengthening devops tasks and often contain unnecessary data

Who is this for? Primarily for engineers working with images containing Python packages.

User Stories

This tool should allow you to answer questions such as:

Which file types are occupying the most disk space?
Which are my largest Python packages?
What are my unknown causes of high disk usage?

Quick start

pip install whaler

Run against a local directory

➜ whaler .venv
Running bash -c cd .venv && du -a -k
Done. Serving output at http://localhost:8000 (ctrl+c to exit)
Running python3 -m http.server 8000 --directory=_whaler/html

Run against a docker image

The tool will pull the image first if it is not present.

whaler --image='hl:latest' /
Running docker run --rm --entrypoint=du --workdir=/ hl:latest -a -k
Ignoring what seems to be non-fatal error(s):
du: cannot access './proc/1/task/1/fd/4': No such file or directory
du: cannot access './proc/1/task/1/fdinfo/4': No such file or directory
du: cannot access './proc/1/fd/3': No such file or directory
du: cannot access './proc/1/fdinfo/3': No such file or directory


Done. Serving output at http://localhost:8000 (ctrl+c to exit)
Running python3 -m http.server 8000 --directory=_whaler/html

Play with a hosted demo

Limitations

Platform: whaler uses du to gather disk usage data. It must be present in your docker image
Scale: I have tested the web UI with up to 500,000 file system nodes with du output of up to ~100MB.

Alternatives/Complements to this tool:

Whaler can tell you what is taking up space in the final layer of your Docker image, but you may have intermediate layers which are contributing to the image size. For diving through the layers, use dive
- Related: read up on multi-stage builds to understand how to mitigate the problem of intermediate layers bloating your image.
For investigating disk usage in non-docker directories, Disk Inventory X is a great tool on OS X which I have based whaler on.
GoogleContainerTools/distroless for base images not containing standard linux distro contents

Developing

See .github/workflows/test.yml for the development platform and setup.

For UI, see whaler-ui

You might also like...

Lima is an alternative to using Docker Desktop on your Mac.

lima-xbar-plugin Table of Contents Description Installation Dependencies Lima is an alternative to using Docker Desktop on your Mac. Description This

68 Dec 22, 2022

Build Netbox as a Docker container

netbox-docker The Github repository houses the components needed to build Netbox as a Docker container. Images are built using this code and are relea

1 Dec 18, 2021

Some automation scripts to setup a deployable development database server (with docker).

Postgres-Docker Database Initializer This is a simple automation script that will create a Docker Postgres database with a custom username, password,

1 Nov 11, 2021

A job launching library for docker, EC2, GCP, etc.

doodad A library for packaging dependencies and launching scripts (with a focus on python) on different platforms using Docker. Currently supported pl

55 Aug 27, 2022

A repository containing a short tutorial for Docker (with Python).

Docker Tutorial for IFT 6758 Lab In this repository, we examine the advtanges of virtualization, what Docker is and how we can deploy simple programs

0 Dec 14, 2021

Bitnami Docker Image for Python using snapshots for the system packages repositories

Python Snapshot packaged by Bitnami What is Python Snapshot? Python is a programming language that lets you work quickly and integrate systems more ef

1 Jan 13, 2022

Hatch plugin for Docker containers

hatch-containers CI/CD Package Meta This provides a plugin for Hatch that allows

11 Dec 30, 2022

Create pinned requirements.txt inside a Docker image using pip-tools

Pin your Python dependencies! pin-requirements.py is a script that lets you pin your Python dependencies inside a Docker container. Pinning your depen

4 Aug 18, 2022

This repository contains useful docker-swarm-tools.

docker-swarm-tools This repository contains useful docker-swarm-tools. swarm-guardian This Docker image is intended to be used in a multihost docker e

4 Jan 12, 2022

Comments

Bounding box on selected node barely visible

This is due to the canvas not being fully re-drawn upon selection.

The solution is probably to ensure the canvas can be re-drawn quickly and creating a fresh render every time
bug

opened by alex-treebeard 0

Visual disk-usage analyser for docker images

Related tags

Overview

whaler

User Stories

Quick start

Run against a local directory

Run against a docker image

Limitations

Alternatives/Complements to this tool:

See also

Developing

You might also like...

Lima is an alternative to using Docker Desktop on your Mac.

Build Netbox as a Docker container

Some automation scripts to setup a deployable development database server (with docker).

A job launching library for docker, EC2, GCP, etc.

A repository containing a short tutorial for Docker (with Python).

Bitnami Docker Image for Python using snapshots for the system packages repositories

Hatch plugin for Docker containers

Create pinned requirements.txt inside a Docker image using pip-tools

This repository contains useful docker-swarm-tools.

Comments

Bounding box on selected node barely visible

Releases(0.1.2)

0.1.2(Dec 31, 2020)

0.1.1(Dec 31, 2020)

0.1(Dec 31, 2020)

Owner

Treebeard Technologies

Utilitaire de contrôle de Kubernetes

Kubediff: a tool for Kubernetes to show differences between running state and version controlled configuration.

Bash-based Python-venv convenience wrapper

🎡 Build Python wheels for all the platforms on CI with minimal configuration.

A Kubernetes operator that creates UptimeRobot monitors for your ingresses

A cpp project template that uses CMake to build and Google Test / Github Actions to provide a CI

Bitnami Docker Image for Python using snapshots for the system packages repositories

KivyPassword - A password generator using both Kivy framework and SQL in order to create a local database for users to generate strong passwords and store them

Rancher Kubernetes API compatible with RKE, RKE2 and maybe others?

A honey token manager and alert system for AWS.

Python utility function to communicate with a subprocess using iterables: for when data is too big to fit in memory and has to be streamed

Cross-platform lib for process and system monitoring in Python

Deploying a production-ready Django project using Nginx and Gunicorn

Run your clouds in RAID.

Manage your SSH like a boss.

Oncall is a calendar tool designed for scheduling and managing on-call shifts. It can be used as source of dynamic ownership info for paging systems like http://iris.claims.

Pulumi - Developer-First Infrastructure as Code. Your Cloud, Your Language, Your Way 🚀

This project shows how to serve an TF based image classification model as a web service with TFServing, Docker, and Kubernetes(GKE).

Wubes is like Qubes but for Windows.

DataOps framework for Machine Learning projects.