pv: Progress Bar for md5sum et al.


Tools that know they will take a long time often come with a built-in progress indicator, but there are other utilities on Linux that often leave the user frustratedly tapping their fingers, wondering how much longer they will have to wait.

Luckily, there is a nifty little tool called pv that will donate a progress bar to any program that can read from standard input or a pipe. pv probably stands for pipe viewer.

1. Simple example: figure out how long an md5sum will take:

pv eternal.avi |md5sum

will display something like

96.5MB 0:00:05 [25.3MB/s] [=======>                                    ]  9% ETA 0:00:48

Notes:

  • pv reads from file and prints to stdout.
  • md5sum reads from stdin.
  • pv outputs the progress bar to stderr so as not to interfere with the piped data. See the man page for ways to customize pv’s output.
  • since the bottleneck of such an operation is the media you’re reading from, not the CPU, there will be no noticeable overhead.

2. Complex example: add a progress bar to tar/bzip2 compression/decompression:

tar cf - mydir | pv -n -s $(du -sb mydir | awk ‘{print $1}’) | bzip2 >mydir.tar.bz2

Notes:

  • this example is adapted from the pv man page.
  • the -n switch makes pv output only percentage values.
  • no file is passed to pv, so it reads from stdin (piped to the output of tar).
  • on a system with good cache and enough memory, doing the extra du -s mydir shouldn’t hurt much, since tar will go through the entire directory anyway.

Now let’s decompress it:

pv mydir.tar.bz2 |tar xjf -

By now you realize how awesome this is.

3. Fun example: measure /dev/null throughput:

pv /dev/zero >/dev/null

is close to 3.3GB/s on my 3-year-old system.

Notes:

  • this is not a benchmark ™.
  • pv can’t know the size of its input in this case (infinity), so it obviously can’t display an ETA.

pv is a brilliant example of the UNIX philosophy: simple puzzle pieces combining to create useful results. A couple of last-word remarks:

  • There is apparently a very similar tool called cpipe.
  • There are, unfortunately, programs for which you will not be able to use pv. One example is dpkg, which apparently tries seeking in its input, thus failing to work with pipes.
  • Thanks to my boss for pointing me to this awesome tool.
Posted in Linux, Shell, Technology   Tagged: bzip, bzip2, compression, decompression, md5, md5 sum, md5sum, pipe, progress, progress bar, pv, tar, throughput, tool, trick, unzip, zip   

Sursa
2009-02-05 00:00:00



Comenteaza





Ultimele 25 posturi adăugate

19:00:00Consilierul (IV) —» Un PUNCT din .FR
18:45:29Ratacit —» Alexander's Poems… | Nu exista minciuni doar adevar fara dovezi…
18:38:03Fum printre ginduri —» Alexander's Poems… | Nu exista minciuni doar adevar fara dovezi…
17:43:11Şi la Iaşi sunt firme funerare fără drept de funcţionare —» Curaj.TV | Media alternativă
13:35:47Artistul plastic bulgar, Krassimir Savov a organizat o expoziție de picturi la bibliotecă —» BPR Ungheni's Blog
13:13:20Что стоит знать про коронавирус 2019-nCoV —» Biblioteca Publică or.Rîşcani
13:03:02Atenție! Se fac controale la sosirea în Germania. Cei care sunt certați cu legea plătesc amenzi ca să nu ajungă după gratii —» Elena Robu
12:33:47Циничный грабёж кишинёвцев закончится техногенной катастрофой —» Блог Михаила Полянского
11:09:34Un nou canal de propagandă rusă în R. Moldova: A fost lansat primul post de televiziune creștin-ortodox care va difuza conținut de la televiziunea patriarhului Chiril al Moscovei —» Elena Robu
10:30:05Campania „Promo Dual VET” a luat start —» Asociaţia Obştească "Demos"
08:59:13Burger fără carne cu sos Torchyn (chifteluță veggie) —» Life is too short to wear boring clothes. - A blog about travel, lifestyle, healthy life and fashio
08:04:58Regruparile de forta din cadrul PDM —» Roman Mihăeş
07:02:57BRÂNCUȘI LA PARIS, ÎN UNIVERS.... —» Leo Butnaru
06:06:36I-au cerut mii de euro pentru eliberare şi n-a mai rezistat —» Curaj.TV | Media alternativă
03:38:0025 de ani. Cele ami bune cărți Cartier (XI) —» Gheorghe Erizanu
02:49:33Odă americii —» Viata pe Tinder – Viata amoroasa pe Tinder
20:57:58(VIDEO) Traian Băsescu surprinde din nou: Iată ce a spus despre românii din diaspora —» Elena Robu
20:25:05Lenevia și ipocrizia —» Victor Timotin | creează, inovează, dezvoltă și fii exemplu
20:14:56La Radio Chișinău despre finanțarea științei în Republica Moldova —» Gheorghe Cuciureanu
18:47:57Servicii funerare ilegale la Mizil —» Curaj.TV | Media alternativă
18:45:15Sa fii Activist Civic sau sa fii Primar e mai bine? —» Curaj.TV | Media alternativă
18:05:28Campania de Informare și Educație Electorală „EU VOTEZ” – în liceul nostru —» Liceul Teoretic “Mihail Sadoveanu”, Călăraşi
16:49:17Patriotism and the Gospel —» Erik and Elena Brewer's Weblog
14:56:32Suma imensă pe care Rusia trebuie să o plătească foștilor acționari Yukos: 50 de miliarde de dolari —» Elena Robu
14:24:45Coronavirus: Al treilea francez contaminat la bordul navei de croazieră Diamond Princess —» Elena Robu