Linux: Using the parallel command to use all your cores

Linux has a cool utility named parallel that let's you run many tasks simultaneously. It's useful for older tasks that aren't multi-threaded. I often use it to encode MP3s in parallel because lame only uses one core. On a modern machine with 8+ cores, it's much more efficient to use them all at the same time. You need to feed parallel a list of files and then use the {} pragma to replace the string with the incoming filename. Parallel has similar syntax to xargs.

find src/dir -type f -iname *.mkv | parallel vid2mp3 '{}' --track 1 --out /var/tmp/

This sample command will invoke parallel command, detecting how many cores are available, and spawn that many threads of the output command. All references to {} will be replaced with the incoming filename.

Leave A Reply
All content licensed under the Creative Commons License