Linux: Using the parallel command to use all your cores

Linux has a cool utility named parallel that let's you run many tasks simultaneously. It's useful for older tasks that aren't multi-threaded. I often use it to encode MP3s in parallel because lame only uses one core. On a modern machine with 8+ cores, it's much more efficient to use them all at the same time. You need to feed parallel a list of files and then use the {} pragma to replace the string with the incoming filename. Parallel has similar syntax to xargs.

find src/dir -type f -iname *.mkv | parallel vid2mp3 '{}' --track 1 --out /var/tmp/

This sample command will invoke parallel command, detecting how many cores are available, and spawn that many threads of the output command. All references to {} will be replaced with the incoming filename.



Note: Replies will be formatted with PHP Markdown Extra syntax.

Name: Email (Not Required):
 
Logged IP: 35.171.22.220
To prevent spam please submit by clicking the kitten: