pv: Progress Bar for md5sum et al.


Tools that know they will take a long time often come with a built-in progress indicator, but there are other utilities on Linux that often leave the user frustratedly tapping their fingers, wondering how much longer they will have to wait.

Luckily, there is a nifty little tool called pv that will donate a progress bar to any program that can read from standard input or a pipe. pv probably stands for pipe viewer.

1. Simple example: figure out how long an md5sum will take:

pv eternal.avi |md5sum

will display something like

96.5MB 0:00:05 [25.3MB/s] [=======>                                    ]  9% ETA 0:00:48

Notes:

  • pv reads from file and prints to stdout.
  • md5sum reads from stdin.
  • pv outputs the progress bar to stderr so as not to interfere with the piped data. See the man page for ways to customize pv’s output.
  • since the bottleneck of such an operation is the media you’re reading from, not the CPU, there will be no noticeable overhead.

2. Complex example: add a progress bar to tar/bzip2 compression/decompression:

tar cf - mydir | pv -n -s $(du -sb mydir | awk ‘{print $1}’) | bzip2 >mydir.tar.bz2

Notes:

  • this example is adapted from the pv man page.
  • the -n switch makes pv output only percentage values.
  • no file is passed to pv, so it reads from stdin (piped to the output of tar).
  • on a system with good cache and enough memory, doing the extra du -s mydir shouldn’t hurt much, since tar will go through the entire directory anyway.

Now let’s decompress it:

pv mydir.tar.bz2 |tar xjf -

By now you realize how awesome this is.

3. Fun example: measure /dev/null throughput:

pv /dev/zero >/dev/null

is close to 3.3GB/s on my 3-year-old system.

Notes:

  • this is not a benchmark ™.
  • pv can’t know the size of its input in this case (infinity), so it obviously can’t display an ETA.

pv is a brilliant example of the UNIX philosophy: simple puzzle pieces combining to create useful results. A couple of last-word remarks:

  • There is apparently a very similar tool called cpipe.
  • There are, unfortunately, programs for which you will not be able to use pv. One example is dpkg, which apparently tries seeking in its input, thus failing to work with pipes.
  • Thanks to my boss for pointing me to this awesome tool.
Posted in Linux, Shell, Technology   Tagged: bzip, bzip2, compression, decompression, md5, md5 sum, md5sum, pipe, progress, progress bar, pv, tar, throughput, tool, trick, unzip, zip   

Sursa
2009-02-05 00:00:00



Comenteaza





Ultimele 25 posturi adăugate

03:27:00ORAȘUL GHEORGHE GHEORGHIU-DEJ PE... DON / Jurnal 19.IX.2015 —» Leo Butnaru
10:46:00MARE-MIC FOTBAL: BRAZILIA – GERMANIA 1-7! Jurnal / 9.VII.2014 —» Leo Butnaru
17:22:37Fără Titlu —» Путепроводные Заметки
16:03:41Fără Titlu —» Путепроводные Заметки
10:56:00Cum ne salvăm în caz de incendiu – sfaturi esențiale pentru toate situațiile —» Eleonora Lisnic în versuri
09:39:00Energocom a semnat un contract pe 3 ani cu OMV Petrom pentru livrarea de gaze naturale din Marea Neagră —» Eleonora Lisnic în versuri
05:33:00DIN POEMELE RECENTE —» Leo Butnaru
18:41:09Fără Titlu —» Путепроводные Заметки
14:59:33Fără Titlu —» Путепроводные Заметки
13:45:40Din loc, de pe loc, în loc de – câte fețe are cuvântul „loc”? (2) —» Moldova Creștină
12:10:54Premieră: spumant Orange —» Fine Wine
09:29:00ȘEICUL ȘI SÂNGELE SEMIT. SAU INVERS, SĂ NU SE SUPERE NIMENI —» Leo Butnaru
22:27:30Fără Titlu —» Путепроводные Заметки
20:26:57Fără Titlu —» Путепроводные Заметки
19:52:41Fără Titlu —» Путепроводные Заметки
19:50:36Fără Titlu —» Путепроводные Заметки
05:36:59Start admitere in clasa a 5-a, anul de studii 2025 -2026 —» Liceul Teoretic “Mihail Sadoveanu”, Călăraşi
05:33:36Start admitere in clasa V-a, anul de studii 2025-2026 —» Centrul Comunitar Instruire, Acces Informaţie Călăraşi
05:22:00LA PLECAREA LUI VLADIMIR BEȘLEAGĂ —» Leo Butnaru
23:58:50Fără Titlu —» Путепроводные Заметки
18:27:23©️ Smerenia unui om mare —» Licurici de suflet
15:41:37Fără Titlu —» Путепроводные Заметки
05:38:00MARI SAVANȚI „ÎNSTRĂINAȚI” APĂRÂND LIMBA ROMÂNĂ LA... MOSCOVA / Jurnal, 7.I.2014 —» Leo Butnaru
02:08:53Fără Titlu —» Путепроводные Заметки
19:06:26Fără Titlu —» Путепроводные Заметки