09/17/87  Multics Simplified I/O Redirection (pipes)


The pipe facility of the command processor provides the ability to
redirect stream I/O.  The delimiter string semi-colon, vertical bar
(";|") instructs the command processor to redirect the I/O attachments
of the standard switches user_input and user_output.  The pipe
component delimiter is interpreted similarly to the command line
delimiter ";" with pipe streams limited to one command line.  The
command processor pipe facility is designed to provide simplified data
flow control.


Notes on basic usage:

The simplest form of a pipe is,

cmd1 ;| cmd2

where cmd1 is the name of a command which outputs to user_output and
cmd2 is a command that gets data from user_input and may output data to
user_output.


The command line,

pwd ;| sm User_id

results in the output of the command print_wd(pwd) being routed as
input to the command send_message (sm).  Some commands will not
execute correctly without an end of information token.  To use the pipe
facility with these commands it is necessary to add the value at the
end of the input.


The command line,

pwd ;| input; ioa_ "." ;| input ;| sdm User_id -sj "the pathname"

routes the output of the print_wd (pwd) command to the file "input".
Then ioa_ is used to add a period to the file and finally the file is
routed as input to the send_mail (sdm) command.


Notes on commands and files usage:

In addition to the ability to route data from one command to another,
the pipe facility is available for more complex functions.

Data can be routed from a command to a file.

The command line,

list ;| list_command_output

is expanded to,

list ;| vfile_ [wd]>list_command_output -extend

The expanded command line instructs the command processor to attach the
user_output switch to the file list_command_output in the current
working directory.  The output of the list command is then placed into
the file.


Notes on commands and I/O module usage:

It is possible to specify an I/O module in the description of an output
data store.

The command line,

list ;| tape_mult_ M9999 -write -den 6250

will route the output of the list command to the magnetic tape M9999
through the Multics standard format tape I/O module tape_mult_.


Any Multics I/O module may be specified.  For a description of the
Multics I/O modules see the Multics manual "Multics Subroutines and I/O
Modules" (AG93) and Section 4 of "Multics Programmer's Reference
Manual" (AG91).


Data can be routed from a file to a command.

The command line,

input_file ;| sm GWMay

expands to,

vfile_ [wd]>input_file -extend ;| sm GWMay

The expanded command line instructs the command processor to route the
data contained in the file input_file as input to the command
send_message (sm).

Any Multics I/O module may be given to specify the source of stored
data.


Notes on interfile usage:

Data can be routed from one file to another.

vfile_ [wd]>input_file -extend ;| tape_mult_ m9999 -write -den 6250

results in the file input_file being copied to the tape m9999.

The command line,

file1 ;| file2

is expanded to,

vfile_ [wd]>file1 -extend ;| vfile_ [wd]>file2 -extend

The expanded command line instructs the command processor to copy the
contents of the file file1 to the file file2.  The pipe facility will
allow the same file name to be given for input and output.

file ;| file

will result in the contents of the file being appended to the end of
itself.


Because most data storing done on the Multics system is handled with
magnetic disk, the defaulting of file names to the I/O module vfile_ is
provided for ease of use.  If desired, control arguments for vfile_ may
be specified with the file name.  Pathnames may also be given.

The command line,

>udd>Multics>file1 ;| file2 -truncate

expands to,

vfile_ udd>Multics>file1 -extend ;| vfile_ [wd]>file2
                                       -extend -truncate


The pipe facility may be used for more complex functions.

The various components of a pipe may be chained together in order
to create a stream of data that accomplishes many tasks.

The command line,

ls ;| list_file ;| sm GWMay

expands to,

ls ;| vfile_ [wd]>list_file -extend ;| sm GWMay

The expanded  command line results in  several functions.  First,
the  output  of  the  list  (ls)  command  is  placed in the file
list_file.   Then the file  list_file is routed  as input to  the
command send_message (sm).

There  is no  limit on  the number  of pipe  delimiters used in a
command line.


Notes on default pipe output

The  pipe facility  provides for   the default  output of  a pipe
stream.

The command line,

ls ;|

is expanded to,

ls ;| vfile_ [wd]>pipeout -extend

The output of the list command  is placed into the file "pipeout"
in the current working directory.


Notes on iteration with pipes:

The  pipe facility  can be   combined with  iteration.  The  pipe
facility limits iteration to a single delimited pipe component.

The command line,

(pwd ls who) ;| output

The  effect of  the command  line is  to route  the output of all
three commands to the file "output".

The command line,

input ;| (cmd1 cmd2 file1) ;| output

instructs the command processor to route the data from "input" to
the commands "cmd1" and "cmd2" and the file "file1" where "input"
is either  a command or file.   The output of all  three are then
routed to "output" which may be a command or file.


A possible  use of this command  line would be when  the commands
cmd1 and cmd2 use input from  "input" but do not output any data.
In such  a case, the file  "file" would contain the  same data as
was routed out  of "input".  This makes it possible  to route the
data through the pipe to "output".


The command line,

a (;|b-c -d;|e) ;| f

is not an acceptable use of iteration and pipe facility.


Notes on active strings with pipes:

The pipe facility can be used in active strings.

The command line,

([segs **]);| file

will copy the contents of each file and output of each command in
the current working directory to the file "file".


The command line,

pl1 ([ls >udd>pl1_dir>ab*.pl1 -no_header -primary -name;|])

instructs  the command  processor to   return the  output of  the
command  ls as the  active return string.   The output of  a pipe
stream is returned only when the active string is terminated with
the pipe delimiter just before the ending right bracket "]".


By  default, new  line characters   are removed  from the  active
function return string.  When it  is desirable to retain new line
characters, an additional vertical bar is appended.

The command line,

value_set command_results ||[string [ls -all -sort;||]

will return the output of the ls command with new line characters
unaltered.


The pipe facility, iteration and active string can be combined in
a single command line.

The command line,

sm ([who ;| match /Multics/;|]) [pwd;|]

routes the  output of the who  command to a filter  "match".  The
names  output by  match are   returned to  the send_message  (sm)
command in  the active function return string  as the destination
of  the message.   The output  of the  print_wd (pwd)  command is
returned as the message to be sent.


Notes on restrictions on active strings:

It is not  possible to intermix pipes and  normal active function
invocations within the same active function bracket pair.

The command line,

string [time; pwd;|]

attempts to call  the time active function then  execute the pipe
pwd;| and is in error.  The correct usage is,

string [time][pwd;|]

Active functions can be used within pipes as long as they are not
within the same active string level as the pipe.  For example:

string [ls [hd]>*.[date];|]


Notes on alternate I/O types

The pipe  facility can be used  with I/O types other  than stream
I/O.   In  order  to  access  sequential  data  type  files,  the
record_stream_ I/O module must be  used as the intermediary.  The
command line,

ls ;| record_stream_ -length 80
  -target "tape_nstd_ m9999 -block 80 -write -den 1600"

will output the stream data of the list command to the sequential
data store m9999 controlled by the tape_nstd_ I/O module.


Notes on command/file conflicts in pipes:

The simplicity of the pipe  facility syntax may lead to conflicts
between file and command names.  In order to supply the simplest
possible  syntax, the  ability  to  explicitly identify  files and
commands can be overridden.  When using the pipe facility, a pipe
string may begin with one of the following:

1) a Multics command name.

2) a Multics I/O module name.

3) a Multics file name that can be followed by
   vfile_ control arguments.


The Multics pipe facility uses the current search list to automatically
determine the type of entry that starts a pipe string.  A segment is
determined to be a command when it contains a linkage section and an
entrypoint matching the name given in the command line.


A Multics I/O module is a special type of object segment that does not
contain an entrypoint that matches the name given on the command line.
All I/O modules do however contain an entrypoint which is a composite
of the name given with an attach identifier.


To facilitate the fewest number of key strokes, lines which start with
a name that is not a command or an I/O module are considered filenames
by default.


Files have no particular identifying features other than that they do
not meet the requirements of an object segment.

Because of the defaulting of names to be files, there may be occasions
when a file name is given when a command was intended and the other way
around.  When this happens, the pipe command line may perform
differently than was intended.


Notes on locating a conflict

When a pipe command line does not perform as expected, the where(wh)
command may help determine the cause.

With the command line,

ls ;| list

the user may have intended the output of the "ls" command to be placed
into a file named "list".  The actual execution of this line will
result in the ls command being executed twice.  With the second
execution displaying to the terminal.


By typing,

where ls

and

where list

the user will easily see where the list entry is being taken from.


To determine if the entry is a file or command, the print_link_info
(pli) command can be used.  Using the example from above with the pli
command results in the command line:

pli [wh list] -he

If the list entry is an object segment, the information displayed by
the command line will indicate so.  If the entry is not an object, an
inconsistency in the segment is diagnosed.


Notes on resolving command/file conflicts:

The conflicts between files and commands can be resolved by explicitly
stating the I/O module in the attachment and by using complete
pathnames.

If the user really does want to override the list command and create a
file named "list" then the command line:

ls ;| [wd]>list

should be used.


If by chance there is a command in the current working directory named
"list", the I/O module name should be included in the attachment.  The
command line:

ls ;| vfile_ [wd]>list

will replace the command with a file named "list.


Whenever it is necessary to override commands with filenames or when a
specific location of a file is important, complete pathnames should be
given.  For example, when using the pipe facility in exec_coms or
abbreviations, it is a good idea to use complete pathnames to make sure
that the proper files and commands are referenced.


                                          -----------------------------------------------------------


Historical Background

This edition of the Multics software materials and documentation is provided and donated
to Massachusetts Institute of Technology by Group BULL including BULL HN Information Systems Inc. 
as a contribution to computer science knowledge.  
This donation is made also to give evidence of the common contributions of Massachusetts Institute of Technology,
Bell Laboratories, General Electric, Honeywell Information Systems Inc., Honeywell BULL Inc., Groupe BULL
and BULL HN Information Systems Inc. to the development of this operating system. 
Multics development was initiated by Massachusetts Institute of Technology Project MAC (1963-1970),
renamed the MIT Laboratory for Computer Science and Artificial Intelligence in the mid 1970s, under the leadership
of Professor Fernando Jose Corbato. Users consider that Multics provided the best software architecture 
for managing computer hardware properly and for executing programs. Many subsequent operating systems 
incorporated Multics principles.
Multics was distributed in 1975 to 2000 by Group Bull in Europe , and in the U.S. by Bull HN Information Systems Inc., 
as successor in interest by change in name only to Honeywell Bull Inc. and Honeywell Information Systems Inc. .

                                          -----------------------------------------------------------

Permission to use, copy, modify, and distribute these programs and their documentation for any purpose and without
fee is hereby granted,provided that the below copyright notice and historical background appear in all copies
and that both the copyright notice and historical background and this permission notice appear in supporting
documentation, and that the names of MIT, HIS, BULL or BULL HN not be used in advertising or publicity pertaining
to distribution of the programs without specific prior written permission.
    Copyright 1972 by Massachusetts Institute of Technology and Honeywell Information Systems Inc.
    Copyright 2006 by BULL HN Information Systems Inc.
    Copyright 2006 by Bull SAS
    All Rights Reserved