28th Nov 2017
Controlling Loops
Controlling and Quitting Loops in Shell Scripts
Controlling for loops in shell scripts is pretty much all about the break and continue commands. These are fairly straightforward, once you've got the hang of them. There is a little gotcha with while loops, however.
1. For Loops
First of all, let's look at for loops. Here is a deliberately simplistic loop, counting 1-9:
#!/bin/bash
for x in 1 2 3 4 5 6 7 8 9
do
# "echo -n" means to stay on this line after
# displaying "x is (whatever)"
echo -n "x is ${x}. "
if [ "$x" -eq "4" ]; then
# Don't process this loop for "4" any further
# ... instead, move on to "5".
# The "echo" below moves down to a new line:
echo
continue
fi
echo "Most numbers get here, but 4 doesn't."
if [ "$x" -eq "7" ]; then
# Once we get to 7, exit the loop completely.
# This overrides the fact that the loop was
# going to continue up to "9".
break
fi
done
echo "We have finished the loop now."
echo "As an aside, x is now ${x}."
Download loop1.sh
The output of the above script is like this:
x is 1. Most numbers get here, but 4 doesn't. x is 2. Most numbers get here, but 4 doesn't. x is 3. Most numbers get here, but 4 doesn't. x is 4. x is 5. Most numbers get here, but 4 doesn't. x is 6. Most numbers get here, but 4 doesn't. x is 7. Most numbers get here, but 4 doesn't. We have finished the loop now. As an aside, x is now 7.
When the loop gets to "4", the "continue" statement tells it to continue with the next item in the loop; it runs a blank "echo" to put the cursor down to the next line (for tidiness of the output), and continues with "5".
When it gets to 7, the "break" statement tells it to break out of the loop completely. This means that "8" and "9" will never be processed. Because a "for" loop gets its list of items at the start, this can be a useful method to get out of a loop that was going to have been running for longer.
A further, and more extreme option, is to exit. This will terminate the entire script.
The next variant on the script checks for "6" and calls "exit" when it finds that "x" is "6". Notice that even the "We have finished the loop now" message doesn't get executed:
#!/bin/bash for x in 1 2 3 4 5 6 7 8 9 do # "echo -n" means to stay on this line after # displaying "x isDownload loop2.sh" echo -n "x is ${x}. " if [ "$x" -eq "4" ]; then # Don't process this loop for "4" any further # ... instead, move on to "5". # The "echo" below moves down to a new line: echo continue fi echo "Most numbers get here, but 4 doesn't." if [ "$x" -eq "6" ]; then # Quit the entire script: exit fi if [ "$x" -eq "7" ]; then # Once we get to 7, exit the loop completely. # This overrides the fact that the loop was # going to continue up to "9". break fi done echo "We have finished the loop now." echo "As an aside, x is now ${x}."
Notice that the whole script aborts when its gets to "6":
x is 1. Most numbers get here, but 4 doesn't. x is 2. Most numbers get here, but 4 doesn't. x is 3. Most numbers get here, but 4 doesn't. x is 4. x is 5. Most numbers get here, but 4 doesn't. x is 6. Most numbers get here, but 4 doesn't.
To restate: As soon as we said "exit", the entire script bombed out. Nothing further was processed, not even the code after the loop.
2. While Loops
while loops act in a similar way to for loops. For example, this loop will continue until you say "finish". It has another special word, too: If you say "silent", then it will increment the counter, but use continue to stop this iteration around the loop (so it skips the display part), and will start a new iteration of the loop. Finally, so that we can demo all of the "break, continue and exit" features, it has a third special word: "abort". If you say "abort" then it will exit the script, and - just like the for loop above, it won't even display the summary information at the end. Execution terminates immediately.
Note that the "read -p MESSAGE x" syntax means "Display MESSAGE and save the user's response in the variable x".
#!/bin/bash
COUNT=0
echo "Please type 'finish' to quit this demo."
# Prompt the user with "Say something> " and
# read the value into variable "x"
while read -p "Say something> " x
do
if [ "${x}" == "finish" ]; then
break
fi
# Increment the counter
let COUNT++
# if we say "silent", then we count the word,
# but do not display it:
if [ "${x}" == "silent" ]; then
continue
fi
echo "You said: '${x}'."
echo "You have said ${COUNT} things."
done
echo "All done. You have said ${COUNT} things to me."
Download loop3.sh
When it is run, it looks something like this, if I give it input of "Happy Christmas" then "silent" then "night", followed by "finish":
Please type 'finish' to quit this demo. Say something> Happy Christmas You said: 'Happy Christmas'. You have said 1 things. Say something> silent Say something> night You said: 'night'. You have said 3 things. Say something> finish All done. You have said 3 things to me.
Note that it counted 3 things, "Happy Christmas", "silent" and "night", but not "finish", because "finish" wasn't an interaction which got counted.
Another example, if we give it input of "the aliens are coming!" and "I don't have time for this demo!" followed by "abort", then it will terminate the script immediately, and does not continue with the rest of the script, which would otherwise have shown the final message of "All done. You have said 2 things to me":
Please type 'finish' to quit this demo. Say something> the aliens are coming! You said: 'the aliens are coming!'. You have said 1 things. Say something> I don't have time for this demo! You said: 'I don't have time for this demo!'. You have said 2 things. Say something> abort
2.1) While Loops in a pipe
There is another way to execute a while loop, and while it looks fairly similar, it has subtly different behaviour. If you run the script above from an input file, you might implement it the same way, but call the while loop like this (we don't need the "-p Say something>" prompt, because we're not going interactive any more):
cat input.txt | while read x
That's a very small change to the script, and it means that "Happy Christmas" will still be treated as a single input ("for x in $(cat input.txt)" would not, but that's a subject for another post!). In fact it all looks pretty similar:
$ cat input.txt Happy Christmas silent night finish $ ./loop4.sh Please type 'exit' to quit this demo. You said: 'Happy Christmas'. You have said 1 things. You said: 'night'. You have said 3 things. All done. You have said 0 things to me. $
But wait - the final line says "You have said 0 things to me." - what is going on?!
Because we piped the "cat input.txt" into the while loop, and a pipe involves two separate processes talking to each other, a new instance of the shell was spawned to run the while loop. This is referred to as a subshell. This doesn't normally affect anything, much of the time, but as an implementation detail, there are two key things where this matters: The scope of variables, and the behaviour of exit.
We will deal with these two issues in the rest of this post.
2.1.1) Scope of Variables
Because the while loop is running in a separate process, any changes it makes to variables will be lost when the loop exits. That process is destroyed, along with the state of all its variables. So $COUNT has been incremented properly within the while loop. But as soon as we end the while loop, it dies, along with all of its state.
The original process, which had set COUNT=0, and has never done anything with COUNT since then, still has COUNT=0 in its state.
This is a pain in the backside, quite frankly! Workarounds include things like writing to files, and then reading the results back from those files after the loop has finished, but it's all a bit messy. One tip is to do something like this (where you're setting the variables before calling the loop, so both processes share the same value):
#!/bin/bash
SHARED_FILE=$(mktemp)
COUNT=0
cat input.txt | while read x
do
let COUNT++
echo ${COUNT} > ${SHARED_FILE}
# presumably do something useful too, or
# you are just reimplementing "wc -l"
done
COUNT=$(cat ${SHARED_FILE})
rm -f ${SHARED_FILE}
2.1.2) Behaviour of exit
Another, and less commonly-encountered side-effect, is that the exit statement doesn't have the same impact. When we say "abort" to the script, we expect it to terminate completely. But when the while loop is in a subshell, it simply exits the loop (just like the break statement does), and returns control to the calling process. This is pretty major, because we normally expect exit to cause the entire thing to terminate immediately. This totally threw me for a while today, and I'm supposed to know all about this stuff!...
$ cat input.txt Happy Christmas abort silent night finish $ ./loop4.sh Please type 'finish' to quit this demo. You said: 'Happy Christmas'. You have said 1 things. All done. You have said 0 things to me.
Here, even though we said "abort", which in the earlier example aborted the whole thing, now we just quit the subshell, and return control to the calling process.
Despite getting an "abort" input, it still continued processing the original script, and ran the "All done. You have said 0 things to me." message.
In this one change, we have changed the behaviour of the whole script, just by the way we called the while loop!
One way around this is to send a return code back to the calling process. We have added the ABORT_CODE variable, which we check for after the loop has returned:
#!/bin/bash
COUNT=0
ABORT_CODE=123
echo "Please type 'finish' to quit this demo."
# Prompt the user with "Say something> " and
# read the value into variable "x"
cat input.txt | while read -p "Say something> " x
do
if [ "${x}" == "finish" ]; then
break
fi
if [ "${x}" == "abort" ]; then
exit $ABORT_CODE
fi
# Increment the counter
let COUNT++
# if we say "silent", then we count the word,
# but do not display it:
if [ "${x}" == "silent" ]; then
continue
fi
echo "You said: '${x}'."
echo "You have said ${COUNT} things."
done
LOOP_RETURN_CODE=$?
if [ "$LOOP_RETURN_CODE" -eq "$ABORT_CODE" ]; then
# exit the calling script, too!
exit
fi
echo "All done. You have said ${COUNT} things to me."
Download loop5.sh
When we run this script, it acts like the earlier, interactive one:
$ cat input.txt Happy Christmas abort silent night finish $ ./loop5.sh Please type 'finish' to quit this demo. You said: 'Happy Christmas'. You have said 1 things. $
3. Conclusion
The exit call is very well known, and is generally well-understood. The break and continue directives are less well-known, and can give you fine-grained control over the execution of your loops.
The thing about while loops in subshells is also relatively well known, at least in how it affects the scope of variables. The implications of exit in a while loop subshell is, I think less obvious, and something to be aware of as you write scripts which make use of such facilities.
Hopefully you found this useful and interesting, and hopefully there was something new in there which you did not know before. Feedback is always most welcome via email.
Invest in your career. Buy my Shell Scripting Tutorial today:
