If you’re looking for a way to communicate with someone remotely, you may want to consider using semaphores. Semaphores are a type of synchronization mechanism that allow two or more processes to share data without having to use shared memory. In this article, we’ll show you how to create and use a semaphore in Bash. First, we’ll need to install the semaphore package: sudo apt-get install semaphore Once the semaphore package is installed, we can create our first semaphore. We’ll use the semctl command to create a new semaphore with the name mysem: semctl mysem create 1 The mysem variable will hold the ID of our newly created semaphore. We can now use this ID to reference our semaphore from within Bash: mysem = SEM_CREATE_ID () echo “Semaphore created: $mysem " ..
Are you developing a multi-threaded application? Sooner or later, you will likely need to use a Semaphore. In this article, you will learn what a Semaphore is, how to create/implement one in Bash, and more.
What Is a Semaphore?
A semaphore is a programming construct that is used in computer programs that employ multiple processing threads (computer processing threads which each execute source code from the same program or same suite of programs) to achieve exclusive use of a common resource at a given point in time. Said in a much easier way, think about this like “one at the time, please.”
A semaphore was first defined towards the end of the 1960s by the late computer scientist Edsger Dijkstra from Rotterdam in the Netherlands. You have likely used semaphores many times in your life without specifically realizing you were doing so!
Staying in the Netherlands for a bit, a country full of small waterways and many movable bridges (called a drawbridge in American English), one can see many excellent real-life world examples of a semaphore; consider the handle of a drawbridge operator: up or down. That handle is the semaphore variable protecting either the waterway or the road from accidents. Thus, the waterway and roads could be seen as other variables protected by the semaphore.
If the handle is up and the bridge is open, the exclusive use of the water/road intersection is given to the ship or ships passing through the channel of water. When the handle is down, the bridge is closed, and the exclusive use of the water/road intersection is given to the cars which pass over the bridge.
The semaphore variable can control access to another set of variables. For example, the up state of the handle prevents the $cars_per_minute and $car_toll_booth_total_income variables from being updated, etc.
We can take the example a little further and clarify that when the handle operates up or down, a matching light is visible to both ship captains on the water and people driving in cars and trucks on the road: all operators can read a common variable for a specific state.
While the scenario described here is not only a semaphore, but it is also a simple mutex. A mutex is another common programming construct that is very similar to a semaphore, with the additional condition that a mutex may only be unlocked by the same task or thread which locked it. Mutex stands for “mutually exclusive.”
In this case, this applies to our example as the bridge operator is the only one with control over our semaphore and mutex up/down handle. In contrast, if the guardhouse at the end of the bridge had a bridge override switch, we would still have a semaphore setup, but not a mutex.
Both constructs are used regularly in computer programming when multiple threads are used to ensure only a single process or task access a given resource at any none time. Some semaphores may be ultra-fast switching, for example, when employed in multithreaded financial market trading software, and some may be much slower, only changing state every few minutes, like when used in an automated drawbridge or in a road train crossing.
Now that we have a better understanding of semaphores let’s implement one in Bash.
Implementing a Semaphore in Bash: Easy, or Not?
Implementing a semaphore in Bash is so easy that it can even be done directly from the command line, or so it seems…
Let’s start simple.
In this code, the variable BRIDGE holds our bridge status. When we set it to up, ships may pass and when we set it to down, cars may pass. We could also read out the value of our variable at any point to see whether the bridge is really up or down. The shared/common resource, in this case, is our bridge.
However, this example is single-threaded, and thus we never ran into a semaphore required situation. Another way to think about this is that our variable can never be up and down at the exact same point in time as code is executed sequentially, i.e., step by step.
Another thing to note is that we did not really control access to another variable (like a semaphore usually would do), and so our BRIDGE variable is not really a true semaphore variable, though it comes close.
Finally, as soon as we introduce multiple threads which can affect the BRIDGE variable we run into issues. For example, what if, directly after the BRIDGE=up command, another thread issues BRIDGE=down which would then result in the message Cars may pass! output, even though the first thread would expect the bridge to be up, and in reality the bridge is still moving. Dangerous!
You can see how things can quickly become murky and confusing, not to mention complex, when working with multiple threads.
The situation where multiple threads try and update the same variable either at the same time or at least close enough in time for another thread to get the situation wrong (which in the case of drawbridges can be quite a while) is called a race condition: two threads racing to updating or reporting some variable or status, with the result that one or more threads may get it quite wrong.
We can make this code much better by flowing the code into procedures and by using a real semaphore variable which will restrict access to our BRIDGE variable depending on the situation.
Creating a Bash Semaphore
Implementing a full multi-threaded, which is thread-safe (a computing term to describe software that is thread-safe or developed in such a way that threads cannot negatively/incorrectly affect each other when they should not) is not an easy feat. Even a well-written program that employs semaphores is not guaranteed to be fully thread-safe.
The more threads there are, and the higher the frequency and complexity of thread interactions, the more likely it is that there will be race conditions.
For our small example, we will look at defining a Bash semaphore when one of the drawbridge operators lowers a bridge handle, thereby indicating he or she wants to lower the bridge. Avid readers may have noticed the reference to operators instead of operator: there are now multiple operators which can lower the bridge. In other words, there are multiple threads or tasks which all run at the same time.
Here we have a lower_bridge function which will do a number of things. Firstly, let’s assume another operator has recently moved the bridge up within the last minute. As such, there is another thread executing code in a function similar to this one called raise_bridge.
In fact, that function has finished raising the bridge but has instituted a mandatory 5-minute wait which all operators previously agreed on and which was hard coded into the source code: it prevents the bridge from going up/down all the time. You can also see this mandatory 5-minute wait implemented in this function as sleep 300.
So, when that raise_bridge function is operating, it will have set the semaphore variable BRIDGE_SEMAPHORE to 1, just like we do in the code here (directly after the echo “Lower bridge command accepted, locking semaphore and lowering bridge” command), and – by means of the first if conditional check in this code – the infinite loop present in this function will continue (ref continue in the code) to loop, with pauses of 2 minutes, as the BRIDGE_SEMAPHORE variable is 1.
As soon as that raise_bridge function finishes raising the bridge and finishing its five minute sleep, it will set the BRIDGE_SEMAPHORE to 0, allowing our lower_bridge function co commence executing the functions execute_lower_bridge and subsequent wait_for_bridge_to_come_down whilst having first re-locked our semaphore to 1 to prevent other functions from taking over bridge control.
There are, however, shortcomings in this code, and race conditions that may have far-reaching consequences for bridge operators are possible. Can you spot any?
The “Lower bridge command accepted, locking semaphore and lowering bridge” is not thread-safe!
If another thread, for example raise_bridge is executing at the same time and trying to access the BRIDGE_SEMAPHORE variable, it could be (when BRIDGE_SEMAPHORE=0 and both threads running reach their respective echo‘s at the exact same time that the bridge operators see “Lower bridge command accepted, locking semaphore and lowering bridge” and “Raise bridge command accepted, locking semaphore and raising bridge”. Directly after each other on the screen! Scary, no?
More scary still is the fact that both threads can proceed to BRIDGE_SEMAPHORE=1, and both threads can continue executing! (There is nothing to stop them from doing so) The reason is that there is not much protection yet for such scenarios. While this code implements a semaphore, it is thus by no means thread-safe. As stated, multi-threaded coding is complex and requires much expertise.
While the timing required in this case is minimal (1-2 lines of code take only a few milliseconds to run), and given the likely low number of bridge operators, the possibility of this happening is very small. However, the fact it is possible is what makes it dangerous. Creating thread-safe code in Bash is not an easy feat.
This could be improved further by, for example, introducing a pre-lock and/or by introducing some form of delay with subsequent re-check (though this will likely require an additional variable) or making a regular re-check before actual bridge execution, etc. Another option is to make a priority queue or a counter variable that checks how many threads have locked control of the bridge etc.
Another commonly used approach, for example, when running multiple bash scripts that could interact, is to use mkdir or flock as base locking operations. There are various examples of how to implement these available online, for example, What Unix commands can be used as a semaphore/lock?.
Wrapping up
In this article, we have a look at what a semaphore is. We also briefly touched on the subject of a mutex. Finally, we looked at implementing a semaphore in Bash using the practical example of multiple bridge operators operating a movable bridge/drawbridge. We also explored how complex implementing a reliable semaphore-based solution is.
If you enjoyed reading this article, have a look at our Asserts, Errors, and Crashes: What’s the Difference? article.
