Continuing the journey to improve my shell scripting skills from beginner to intermediate.
Problem
I was working to improve the performance of my NAS daily sync mechanism. Currently, it uses rsync commands at the folder level. There are around 5 folders to sync right now. It issues the rsync commands one by one starting at 1 AM and outputs the results to a log file. After completing all rsync commands it sends a summary email. Obviously, it takes time. The easy quick fix is to parallelize.
Once all the rsync commands are complete in parallel execute the script to send the summary email
Running shell commands in parallel
If it was PowerShell we need to use Jobs or the ForEach-Object -Parallel iteration. But in shell, it is as easy as combining commands with &. Below is the code to demo the concept using a simple number-counting scenario.
Code to count from the numbers passed into the script
When it runs, it outputs the numbers to numbers.txt.
Below is the code to invoke the counter.sh by passing start and end numbers
This will be adding 2 additional lines to the numbers.txt file
The magic is done by the ampersand & symbol at the end of each command. & executes in the background
Executing & results
When we run without the & we can see the numbers.txt will have numbers in order. But with & the numbers will be saved out of order
Also, note the "End counting" line is saved only after all the numbers as it waits for the parallel invocations to complete.
References
