# checkpointctl - a tool for in-depth analysis of container checkpoints [](https://github.com/checkpoint-restore/checkpointctl/actions/workflows/tests.yml) [](https://codecov.io/gh/checkpoint-restore/checkpointctl) Container engines like *Podman* and *CRI-O* have the ability to checkpoint a container. All data related to a checkpoint is collected in a checkpoint archive. With the help of this tool, `checkpointctl`, it is possible to display information about these checkpoint archives. Details on how to create checkpoints with the help of [CRIU][criu] can be found at: * [Forensic container checkpointing in Kubernetes][forensic] * [Podman checkpoint][podman] ## Usage ### `show` sub-command To display an overview of a checkpoint archive you can just use `checkpointctl show`: ```console $ checkpointctl show /tmp/dump.tar Displaying container checkpoint data from /tmp/dump.tar CONTAINER IMAGE ID RUNTIME CREATED ENGINE CHKPT SIZE ROOT FS DIFF SIZE --------- ----- -- ------- ------- ------ ---------- ----------------- looper docker.io/library/busybox:latest 8b5c2ca15082 crun 2021-09-28T10:03:56Z Podman 130.8 KiB 204 B ``` For a checkpoint archive created by Kubernetes with *CRI-O* the output would look like this: ```console $ checkpointctl show /var/lib/kubelet/checkpoints/checkpoint-checkpoint-test_default-checkpoint-test-2026-03-13T21:43:06Z.tar Displaying container checkpoint data from /var/lib/kubelet/checkpoints/checkpoint-checkpoint-test_default-checkpoint-test-2026-03-13T21:43:06Z.tar CONTAINER IMAGE ID RUNTIME CREATED ENGINE CHKPT SIZE ROOT FS DIFF SIZE --------- ----- -- ------- ------- ------ ---------- ----------------- checkpoint-test docker.io/library/busybox:latest 68bb2fa6a176 crun 2026-03-13T21:42:59.05135916Z CRI-O 314.6 KiB 3.5 KiB ``` ### `inspect` sub-command To retrieve low-level information about a container checkpoint, use the `checkpointctl inspect` command: ```console Displaying container checkpoint tree view from /tmp/ubuntu_looper.tar.gz awesome_booth ├── Image: docker.io/library/ubuntu:latest ├── ID: 2076a99c49c959375c6ae406818a2c841744a019d30221f134db657232f133f5 ├── Runtime: crun ├── Created: 2026-03-16T08:00:17Z ├── Engine: Podman ├── Network Interfaces │ └── podman (eth0) │ ├── IP: 10.88.0.4/16 │ ├── MAC: 2e:e1:db:6b:68:93 │ └── Gateway: 10.88.0.1 ├── Checkpoint size: 1018.3 KiB │ └── Memory pages size: 952.0 KiB ├── Root FS diff size: 2.0 KiB └── Process tree └── [1] bash └── [4] su └── [5] bash └── [6] sleep ``` For a complete list of flags supported, use `checkpointctl inspect --help`. ### `diff` sub-command To compare two container checkpoints and analyze changes between them, use the checkpointctl `diff` command. ```console $ checkpointctl diff --show-unchanged --ps-tree-cmd --sockets ./cp1.tar cp2.tar ╔════════════════════════════════════════════════════════════════╗ ║ Checkpoint Diff ║ ╠════════════════════════════════════════════════════════════════╣ ║ Container: optimistic_mclean ║ ║ Image: docker.io/library/python:alpine ║ ║ ID: 6ebe04dda2005962a8fb4e971caedfda5e03e22c882e2334... ║ ╚════════════════════════════════════════════════════════════════╝ Checkpoint A: Created: 2026-04-18T10:15:41+01:00 Size: 15186908 bytes Checkpoint B: Created: 2026-04-18T10:15:41+01:00 Size: 15212641 bytes ┌─ Memory Changes ─────────────────────────────────────────────┐ │ ↑ Increased by 0.02 MB └──────────────────────────────────────────────────────────────┘ ┌─ Process Tree ───────────────────────────────────────────────┐ │ Process tree │ └── [= PID 1] sleep infinity │ └── [= PID 2] python3 -m http.server 8080 └──────────────────────────────────────────────────────────────┘ ┌─ Socket Changes ──────────────────────────────────────────────────────┐ │ PID PROTO STATE LOCAL PEER │ + 2 TCP ESTABLISHED 10.88.0.3:8080 10.88.0.1:39838 │ = 2 TCP LISTEN 0.0.0.0:8080 - └───────────────────────────────────────────────────────────────────────┘ Summary: Checkpoint comparison for container optimistic_mclean Sockets: +1 -0 Memory: +0.02 MB ``` ### `memparse` sub-command To perform memory analysis of container checkpoints, you can use the `checkpointctl memparse` command. ```console $ checkpointctl memparse /tmp/jira.tar.gz --pid=1 | less Displaying memory pages content for Process ID 1 from checkpoint: /tmp/jira.tar.gz ADDRESS HEXADECIMAL ASCII ------------------------------------------------------------------------------------- 00005633bb080000 f3 0f 1e fa 48 83 ec 08 48 8b 05 d1 4f 00 00 48 |....H...H...O..H| 00005633bb080010 85 c0 74 02 ff d0 48 83 c4 08 c3 00 00 00 00 00 |..t...H.........| 00005633bb080020 ff 35 b2 4e 00 00 f2 ff 25 b3 4e 00 00 0f 1f 00 |.5.N....%.N.....| 00005633bb080030 f3 0f 1e fa 68 00 00 00 00 f2 e9 e1 ff ff ff 90 |....h...........| * 00005633bb0800a0 f3 0f 1e fa 68 07 00 00 00 f2 e9 71 ff ff ff 90 |....h......q....| 00005633bb0800b0 f3 0f 1e fa 68 08 00 00 00 f2 e9 61 ff ff ff 90 |....h......a....| 00005633bb0800c0 f3 0f 1e fa 68 09 00 00 00 f2 e9 51 ff ff ff 90 |....h......Q....| 00005633bb0800d0 f3 0f 1e fa 68 0a 00 00 00 f2 e9 41 ff ff ff 90 |....h......A....| 00005633bb0800e0 f3 0f 1e fa 68 0b 00 00 00 f2 e9 31 ff ff ff 90 |....h......1....| ``` Here's an example of memory analysis of a PostgreSQL container. In this case, we start a PostgreSQL container with a password set to 'mysecret'. Then, we create a checkpoint of the container and use the `memparse` to find the stored password. ```console $ sudo podman run --name postgres -e POSTGRES_PASSWORD=mysecret -d postgres $ sudo podman container checkpoint -l --export=/tmp/postgres.tar.gz $ sudo checkpointctl memparse --pid 1 /tmp/postgres.tar.gz | grep -B 1 -A 1 mysecret 000055dd725c1e60 50 4f 53 54 47 52 45 53 5f 50 41 53 53 57 4f 52 |POSTGRES_PASSWOR| 000055dd725c1e70 44 3d 6d 79 73 65 63 72 65 74 00 00 00 00 00 00 |D=mysecret......| 000055dd725c1e80 00 00 00 00 00 00 00 00 31 00 00 00 00 00 00 00 |........1.......| ``` Here's another scenario, of memory analysis for a web application container. We start a vulnerable web application container, perform an arbitrary code execution attack, create a checkpoint for forensic analysis while leaving the container running, and finally analyze the checkpoint memory to identify the injected code. ```bash # Start vulnerable web application $ sudo podman run --name dsvw -p 1234:8000 -d quay.io/rst0git/dsvw # Perform arbitrary code execution attack: $(echo secret) $ curl "http://localhost:1234/?domain=www.google.com%3B%20echo%20secret" nslookup: can't resolve '(null)': Name does not resolve Name: www.google.com Address 1: 142.250.187.228 lhr25s34-in-f4.1e100.net Address 2: 2a00:1450:4009:820::2004 lhr25s34-in-x04.1e100.net secret # Create a checkpoint for forensic analysis and leave the container running $ sudo podman container checkpoint --leave-running -l -e /tmp/dsvw.tar # Analyse checkpoint memory to identify the attacker's injected code $ sudo checkpointctl memparse --pid 1 /tmp/dsvw.tar | grep 'echo secret' 00007faac5711f60 6f 6d 3b 20 65 63 68 6f 20 73 65 63 72 65 74 00 |om; echo secret.| ``` For larger processes, it's recommended to write the contents of process memory pages to a file rather than standard output. To get an overview of process memory sizes within the checkpoint, run `checkpointctl memparse` without arguments. ```console $ sudo checkpointctl memparse /tmp/jira.tar.gz Displaying processes memory sizes from /tmp/jira.tar.gz PID PROCESS NAME MEMORY SIZE SHARED MEMORY SIZE --- ------------ ----------- ------------------ 1 tini 100.0 KiB 0 B 2 java 553.5 MiB 0 B ``` In this example, given the large size of the java process, it is better to write its output to a file. ```console $ sudo checkpointctl memparse --pid=2 /tmp/jira.tar.gz --output=/tmp/java-memory-pages.txt Writing memory pages content for process ID 2 from checkpoint: /tmp/jira.tar.gz to file: /tmp/java-memory-pages.txt... ``` Please note that writing large memory pages to a file can take several minutes. ### `build` sub-command Restoring a container from a checkpoint in Kubernetes requires converting the checkpoint archive into an OCI image. The `build` command simplifies this process by extracting the necessary metadata from the checkpoint and creating an OCI-compatible image with appropriate annotations. Example: ```console checkpointctl build ./checkpoint.tar quay.io/foo/bar:latest buildah push quay.io/foo/bar:latest ``` In this example, the `checkpointctl build` command converts the `checkpoint.tar` archive into an OCI-compatible image and tags it as `quay.io/foo/bar:latest`. The following `buildah push` command then uploads the newly created OCI image to the container registry, making it available for deployment. ### `plugin` sub-command The `plugin` sub-command manages external plugins that extend checkpointctl with additional subcommands. Plugins are standalone executables in PATH with names matching the pattern `checkpointctl-`. ```console $ checkpointctl plugin list Available plugins: hello My custom plugin description /usr/local/bin/checkpointctl-hello ``` This plugin architecture allows extending checkpointctl functionality without increasing the binary size of the core tool. Plugins with larger dependencies can be distributed separately, and can be implemented in any programming language. #### Creating a plugin To create a plugin, create an executable named `checkpointctl-`: ```bash #!/bin/sh # Support --plugin-description for help text (must exit with code 42) if [ "$1" = "--plugin-description" ]; then echo "Say hello" exit 42 fi echo "Hello, world!" ``` Save it as `checkpointctl-hello` in a directory in your PATH and make it executable. It will then be available as `checkpointctl hello`. Note: The exit code 42 is required to indicate that the plugin supports the `--plugin-description` flag. Any other exit code will result in a default description being used. ## Installing from source code 1. Clone the repository. ```console git clone https://github.com/checkpoint-restore/checkpointctl.git ``` 2. Install dependencies. On Fedora, CentOS and related distributions: ```console sudo yum install -y make asciidoctor ``` On Debian, Ubuntu, and related distributions: ```console sudo apt-get install -y make asciidoctor ``` Linux distributions often provide an old version of the golang compiler. We recommend to install Go from the [official tarballs](https://go.dev/doc/install). 3. Build. ```console make ``` 4. Once checkpointctl has been compiled, it can be installed on a system by simply typing ```console sudo make install ``` This command accepts the following variables: * **DESTDIR**, to specify global root where all components will be placed under (empty by default); * **PREFIX**, to specify additional prefix for path of every component installed (`/usr/local` by default); * **BINDIR**, to specify where to install the checkpointctl tool (`$(PREFIX)/bin` by default); Thus, to install everything under `/some/new/place`, use the following command: ```console make DESTDIR=/some/new/place install ``` ## Enable autocompletion You now need to ensure that the autocompletion script gets sourced in all your shell sessions. There are two ways in which you can do this: ### User ```console echo 'source <(checkpointctl completion bash)' >>~/.bashrc ``` ### System ```console checkpointctl completion bash | sudo tee /etc/bash_completion.d/checkpointctl > /dev/null sudo chmod a+r /etc/bash_completion.d/checkpointctl ``` Both approaches are equivalent. After reloading your shell, autocompletion should be working. To enable bash autocompletion in current shell session, source the `~/.bashrc` file: ```console source ~/.bashrc ``` ## Uninstalling The following command can be used to clean up a previously installed checkpointctl instance. ```console make uninstall ``` Note that if some variable (e.g., **DESTDIR**, **BINDIR**) has been used during installation, the same *must* be passed with uninstall action. ## How to contribute See [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to contribute to this project. ## License and copyright Unless mentioned otherwise in a specific file's header, all code in this project is released under the Apache 2.0 license. The author of a change remains the copyright holder of their code (no copyright assignment). The list of authors and contributors can be retrieved from the git commit history and in some cases, the file headers. [forensic]: https://kubernetes.io/blog/2022/12/05/forensic-container-checkpointing-alpha/ [podman]: https://podman.io/docs/checkpoint [criu]: https://criu.org/