Unix - Lev Popov
Lev Popov
TR12/rm304
Martin Rinard/Rob Seater
Unix Pipes

In Unix, pipes are a convenient, lightweight way of performing inter-process communication (IPC), commonly used in execution of shell commands for filter pipelines. While simple and efficient for the pipeline scenario, pipe use for more complex systems can become difficult due to the pipes' design limitation that requires for a pipe used by a pair of processes to be created by a common ancestor of the two processes. This limitation prevents pipes from being used in a lot of cases with non-trivial process communication patterns, or requires adding to system complexity to cope with the constraint.

Unless special care is taken, the desired piped process communication pattern will dictate the process launching order, possibly leading to faulty designs. For example, consider a scenario with a persistent back-end process and a front-end process. Since the back-end is persistent, it must be the back-end process that launches the front-end, in order to communicate through pipes. This creates an inelegant design, and the system modularity suffers. If the system was to have two redundant back-ends, there would be no way for the client to gracefully switch to the backup.

Another aspect of this limitation is that a pipe has to be created before the child process that will use it, which is a problem for numerous designs. An example is a system performing distributed computation with a varying number of participating worker processes and a single manager process. If all worker processes must communicate, the pipes for them must be created before the processes are started. But if the number of processes is dynamic, and unknown beforehand, the manager has no way of guaranteeing pipes for all its children.

There is, of course, a workaround for the problems in the examples above which actually lifts the common ancestor constraint in the general case. One could design a system with a top level wrapper process that handles the launching of all other processes and relays communication between them. This way, any pair of processes can communicate with no limitations through the wrapper that is acting as a proxy. Unfortunately, there is a cost - significant overhead that is hurting performance and increasing complexity. Pipes are an elegant and efficient way of doing IPC for simple scenarios, like pipelines, but for more complex systems, a heavier solution, like sockets, might be more appropriate.

Acknowledgments: I have discussed ideas for this paper with Ashwind Deshpande