This is Info file pm.info, produced by Makeinfo version 1.68 from the input file bigpm.texi.  File: pm.info, Node: IPC/ClearTool, Next: IPC/LDT, Prev: IPC/ChildSafe, Up: Module List run a bidirectional pipe to a cleartool process *********************************************** NAME ==== IPC::ClearTool, ClearTool - run a bidirectional pipe to a cleartool process SYNOPSIS ======== use IPC::ClearTool; my $CT = IPC::ClearTool->new; $CT->cmd("pwv"); $CT->cmd("lsview"); $CT->cmd("lsvob -s"); for ($CT->stdout) { print } $CT->finish; ALTERNATE SYNOPSES ================== use IPC::ClearTool; $rc = $CT->cmd("pwv"); # Assign return code to $rc $CT->notify; # "notify mode" is default; $rc = $CT->cmd("pwv"); # same as above $CT->store; # "Store mode" - hold stderr for $rc = $CT->cmd("pwv -X"); # later retrieval via $CT->stderr @errs = $CT->stderr; # Retrieve it now $CT->ignore; # Discard all stdout/stderr and $CT->cmd("pwv"); # ignore nonzero return codes $CT->cmd("ls foo@@"); # In void context, store stdout, # print stderr immediately, # exit on error. my %results = $CT->cmd("pwv"); # Place all results in %results, # available as: # @{$results{stdout}} # @{$results{stderr}} # @{$results{status}} $CT->cmd(); # Clear all accumulators $CT->stdout; # In void context, print stored output DESCRIPTION =========== This module invokes the ClearCase 'cleartool' command as a child process and opens pipes to its standard input, output, and standard error. Cleartool commands may be sent "down the pipe" via the $CT->cmd() method. All stdout resulting from commands is stored in the object and can be retrieved at any time via the $CT->stdout method. By default, stderr from commands is sent directly to the real (parent's) stderr but if the store attribute is set as shown above, stderr will accumulate just like stdout and must be retrieved via $CT->stderr. If $CT->cmd is called in a void context it will exit on error unless the *ignore* attribute is set, in which case all output is thrown away and error messages suppressed. If called in a scalar context it returns the exit status of the command. When used with no arguments and in a void context, $CT->cmd simply clears the stdout and stderr accumulators. The $CT->stdout and $CT-stderr methods behave just like arrays; when used in a scalar context they return the number of lines currently stored. When used in an array context they return, well, an array containing all currently stored lines, and then clear the internal stack. The $CT->finish method ends the child process and returns its exit status. This is only a summary of the documentation. There are more advanced methods for error detection, data return, etc. documented as part of IPC::ChildSafe. Note that IPC::ClearTool is simply a small subclass of ChildSafe; it provides the right defaults to ChildSafe's constructor for running cleartool and adds a few ClearCase-specific methods. In all other ways it's identical to ChildSafe, and all ChildSafe documentation applies. BUGS ==== * Comments Comments present a special problem. If a comment is prompted for, it will likely hang the child process by interrupting the tag/eot sequencing. So we prefer to pass comments on the command line with -c. Unfortunately, the quoting rules of cleartool are insufficient to allow passing comments with embedded newlines using -c. The result being that there's no clean way to handle multi-line comments. To work around this, a method `$CT-'comment> is provided which registers a comment *to be passed to the next `$CT-'cmd()* command>. It's inserted into the stdin stream with a "\n.\n" appended. The subsequent command must have a `-cq' flag, e.g.: $CT->comment("Here's a\nmultiple line\ncomment"); $CT->cmd("ci -cq foo.c"); If your script hangs and the comment for the last element checked in is `"pwd -h"', then you were burned by such a sync problem. * UNIX/Win32 Semantics Skew On UNIX, this module works by running cleartool as a child process. On Windows, the ClearCase Automation Library (a COM API) is used instead. This provides the same interface *but be warned that there's a subtle semantic difference!* On UNIX you can send a setview command to the child and it will run in the new view while the parent's environment remains unchanged. On Windows there's no subprocess; thus the setview would change the context of the "parent process". The same applies to chdir commands. It's unclear which behavior is "better" overall, but in any case portable applications must take extra care in using such stateful techniques. As a partial remedy, a chdir method is provided. This simply does the cd in both parent and child processes in an attempt to emulate the in-process behavior. Emulating an in-process `setview' is harder because on UNIX, setview is implemented with a fork/chroot/exec sequence so (a) it's hard to know how a single-process setview should behave and (b) I wouldn't know how to do it anyway, especially lacking the privileges required by chroot(2). Of course in most cases you could work around this by using chdir to work in view-extended space rather than a set view. Of course, in some cases the ability to set the child process into a different view or directory is a feature, and no attempt is made to stop you from doing that. * Win32::OLE Behavior with IClearTool Due to the way Win32::OLE works, on Windows the results of each command are passed back as a single string, possibly with embedded newlines. For consistency, in a list context we split this back into lines and return the list. However, it's possible for this to result in a different line count for the same command in UNIX and NT, if one of the lines would have had embedded newlines in it anyway. * Win32/CC 3.2.1 COM Bug The ClearCase/COM API wasn't documented until CC 4.0, but it's actually present (at least the IClearTool interface, which is what IPC::ClearTool uses) in 3.2.1 with one major bug - all output arrives backwards! So we provide a class method IPC::ClearTool->cc_321_hack; which, if called, will determine whether it's running on CC 3.2.1 and if so reverse the order of output. This is called once at the start of the program to set the state; it's a no-op if CC 4.0 or above is in use. AUTHOR ====== David Boyce dsb@world.std.com SEE ALSO ======== perl(1), "perldoc IPC::ChildSafe"  File: pm.info, Node: IPC/LDT, Next: IPC/Locker, Prev: IPC/ClearTool, Up: Module List implements a length based IPC protocol ************************************** NAME ==== *IPC::LDT* - implements a length based IPC protocol SCRIPT DATA =========== This manual describes version *2.03*. DESCRIPTION =========== Interprocess communication often uses line (or record) oriented protocols. FTP, for example, usually is such a protocol: a client sends a command (e.g. "LS") which is completed by a carriage return. This carriage return is included in the command which is sent to the server process (FTP deamon) which could implement its reading in a way like this: while ($cmd=) { chomp($cmd); performCommand($cmd); } Well, such record oriented, blocked protocols are very useful and simply to implement, but sometimes there is a need to transfer more complex data which has no trailing carriage return, or data which may include more carriage returns inside the message which should not cause the reciepient to think the message is already complete while it is really not. Even if you choose to replace carriage returns by some obscure delimiters, the same could happen again until you switch to a protocol which does not flag the end of a message by special strings. On the other hand, if there is no final carriage return (or whatever flag string) within a message, the end of the message has to be marked another way to avoid blocking by endless waiting for more message parts. A simple way to provide this is to precede a message by a prefix which includes the length of the remaining (real) message. A reciepient reads this prefix, decodes the length information and continues reading until the announced number of bytes came in. *IPC::LDT* provides a class to build objects which transparently perform such "length driven transfer". A user sends and receives messages by simple method calls, while the LDT objects perform the complete translation into and from LDT messages (with prefix) and all the necessary low level IO handling to transfer stream messages on non blocked handles. *IPC::LDT* objects can be configured to transfer simle string messages as well as complex data structures. Additionally, they allow to delay the transfer of certain messages in a user defined way. SYNOPSIS ======== Load the module as usual: use IPC::LDT; Make an LDT object for every handle that should be used in an LDT communication: my $asciiClient=new IPC::LDT(handle=>HANDLE); my $objectClient=new IPC::LDT(handle=>HANDLE, objectMode=>1); Now you can send and receive data: $data=$asciiClient->receive; @objects=$objectClient->receive; $asciiClient=$client->send("This is", " a message."); $objectClient=$client->send("These are data:", [qw(a b c)]); Exports ------- No symbol is exported by default. You can explicitly import LDT_CLOSED, LDT_READ_INCOMPLETE, LDT_WRITE_INCOMPLETE, LDT_OK and LDT_INFO_LENGTH which are described in section CONSTANTS. Global Variables ================ Settings -------- Traces You may set the module variable *$IPC::LDT::Trace* before the module is loaded (that means in a BEGIN block before the "use" statement) to activate the built in trace code. If not prepared this way, all runtime trace settings (e.g. via the constructor parameter traceMode) will take *no effect* because the trace code will have been filtered out at compile time for reasons of performance. (This means that no trace message will appear.) If *$IPC::LDT::Trace* is set before the module is loaded, te builtin trace code is active and can be deactivated or reactivated at runtime globally (for all objects of this class) by unsetting or resetting of this module variable. Alternatively, you may choose to control traces for certain objects by using the constructor parameter traceMode. So, if you want to trace every object, set *$IPC::LDT::Trace* initially and load the module. If you want to trace only certain objects, additionally unset *$IPC::LDT::Trace* after the module is loaded and construct these certain objects with constructor flag traceMode. Assertions It is a good tradition to build self checks into a code. This makes code execution more secure and simplifies bug searching after a failure. On the other hand, self checks decrease code performance. That's why you can filter out the self checking code (which is built in and activated by default) by setting the module variable *$IPC::LDT::noAssert* before the module is loaded. The checks will be removed from the code before they reach the compiler. Setting or unsetting this variable after the module was loaded takes *no effect*. CONSTANTS ========= Error codes ----------- LDT_CLOSED a handle related to an LDT object was closed when reading or writing should be performed on it; LDT_READ_INCOMPLETE a message could not be (completely) read within the set number of trials; LDT_WRITE_INCOMPLETE a message could not be (completely) written within the set number of trials; METHODS ======= new() ----- The constructor builds a new object for data transfers. All parameters except of the class name are passed named (this means, by a hash). Parameters: Class name the first parameter as usual - passed implicitly by Perl: my $asciiClient=new IPC::LDT(...); The method form of construtor calls is not supported. handle The handle to be used to perform the communication. It has to be opened already and will not be closed if the object will be destroyed. Example: handle => SERVER A closed handle is not accepted. You can use whatever type of handle meets your needs. Usually it is a socket or anything derived from a socket. For example, if you want to perform secure IPC, the handle could be made by Net::SSL. There is only one precondition: the handle has to provide a *fileno()* method. (You can enorce this even for Perls default handles by simply using FileHandle.) objectMode Pass a true value if you want to transfer data structures. If this setting is missed or a "false" value is passed, the object will transfer strings. Data structures will be serialized via Storable for transfer. Because of this, such a communication is usually restricted to partners which could use Storable methods as well to reconstruct the data structures (which means that they are written in Perl). String transfer objects, on the other hand, can be used to cimmunicate with any partner who speaks the LDT protocol. We use Java and C clients as well as Perl ones, for example. Example: objectMode => 1 The transfer mode may be changed while the object is alive by using the methods setObjectMode() and setAsciiMode(). startblockLength sets the length of the initial info block which preceds every LDT message coding the length of the remaining message. This setting is done in bytes. If no value is provided, the builtin default value *LDT_INFO_LENGTH* is used. (This value can be imported in your own code, see section "Exports" for details.) *LDT_INFO_LENGTH* is designed to meet usual needs. Example: startblockLength => 4 traceMode Set this flag to a true value if you want to trace to actions of the module. If set, messages will be displayed on STDERR reporting what is going on. Traces for objects of this class can be activated (regardless of this constructor parameter) via *$IPC::LDT::Trace*. This is described more detailed in section "CONSTANTS". Example: traceMode => 1 *Return value:* A successfull constructor call replies the new object. A failed call replies an undefined value. Examples: my $asciiClient=new IPC::LDT(handle=>HANDLE); my $objectClient=new IPC::LDT(handle=>HANDLE, objectMode=>1); setObjectMode() --------------- Switches the LDT object to "object trasnfer mode" which means that is can send and receive Perl data structures now. Runtime changes of the transfer mode have to be exactly synchronized with the partner the object is talking with. See the constructor (new()) description for details. Parameters: object An LDT object made by new(). Example: $asciiClient->setObjectMode; setAsciiMode() -------------- Switches the LDT object to "ASCII trasnfer mode" which means that is sends and receives strings now. Runtime changes of the transfer mode have to be exactly synchronized with the partner the object is talking with. See the constructor (new()) description for details. Parameters: object An LDT object made by new(). Example: $objectClient->setAsciiMode; delay() ------- Sometimes you do not want to send messages immediatly but buffer them for later delivery, e.g. to set up a certain send order. You can use delay() to install a filter which enforces the LDT object to delay the delivery of all matching messages until the next call of undelay(). The filter is implemented as a callback of send(). As long as it is set, send() calls it to check a message for sending or buffering it. You can overwrite a set filter by a subsequent call of delay(). Messages already collected will remain collected. To send delayed messages you have to call undelay(). If the object is detroyed while messages are still buffered, they will not be delivered but lost. Parameters: object An LDT object made by new(). filter A code reference. It should await a reference to an array which will contain the message (possibly in parts). It should reply a true or false value to flag if the passed message has to be delayed. It is recommended to provide a fast function because it will be called everytime send() will be invoked. Example: $ldt->delay(\&filter); with filter() defined as sub filter { # get and check parameters my ($msg)=@_; confess "Missed message parameter" unless $msg; confess "Message parameter is no array reference" unless ref($msg) and ref($msg) eq 'ARRAY'; # check something $msg->[0] eq 'delay me'; } See undelay() for a complete example. undelay() --------- Sends all messages collected by a filter which was set by delay(). The filter is removed, so that every message will be sent by send() immediatly afterwards again. In case of no buffered message and no set filter, a call of this message takes no effect. Parameters: object An LDT object made by new(). *Beispiel:* $ldt->undelay; Here comes a complete example to illustrate how delays can be used. filter definition: sub filter { # check something $msg->[0] eq 'delay me'; } usage: # send messages $ldt->send('send me', 1); # sent; $ldt->send('delay me', 2); # sent; # activate filter $ldt->delay(\&filter); # send messages $ldt->send('send me', 3); # sent; $ldt->send('delay me', 4); # delayed; $ldt->send('send me', 5); # sent; $ldt->send('delay me', 6); # delayed; # send collected messages, uninstall filter $ldt->undelay; # sends messages 4 and 6; # send messages $ldt->send('send me', 7); # sent; $ldt->send('delay me', 8); # sent; send() --------------- Sends the passed message via the related handle (which was passed to new()). The message, which could be passed as a list of parts, is sent as a (concatenated) string or as serialized Perl data depending on the settings made by the constructor flag objectMode and calls of setObjectMode() or *setAsciiMode*, respectively. In case of an error, the method replies an undefined value and stores both an error code and an error message inside the object which could be accessed via the object variables "rc" and "msg". (See CONSTANTS for a list of error codes.) An error will occur, for example, if the handle related to the LDT object was closed (possibly outside the module). An error is detected as well if a previous call of send() or receive() already detected an error. This behaviour is implemented for reasons of security, however, if you want to try it again regardless of the objects history, you can reset the internal error state by reset(). For reasons of efficiency, sent messages may be splitted up into parts by the underlaying (operating or network) system. The reciepient will get the message part by part. On the other hand, the sender might only be able to send them part by part as well. That is why this send() method retries writing attempts to the associated handle until the complete message could be sent. Well, in fact it stops retries earlier if an inacceptable long period of time passed by without being successfull. If that happens, the method replies undef and provides an error code in the object variable "rc". *The caller should be* *prepared to handle such cases. Usually further access to the associated handle* *is useless or even dangerous.* Parameters: object An LDT object made by new(). message (a list) All list elements will be combined to the resulting message as done by print() or *warn()* (that means, *without* separating parts by additional whitespaces). Examples: $asciiClient->send('Silence?', 'Maybe.') or die $asciiClient->{'msg'}; $objectClient->send({oops=>1, beep=>[qw(7)]}, $scalar, \@array); Note: If the connection is closed while the message is sent, the signal *SIGPIPE* might arrive and terminate the complete program. To avoid this, *SIGPIPE* is ignored while this method is running. The handle associated with the LDT object is made *non blocking* during data transmission. The original mode is restored before the method returns. reset ----- If an error occurs while data are transmitted, further usage of the associated handle is usually critical. That is why send() and receive() stop operation after a transmission error, even if you repeat their calls. This should *protect* your program and make it more stable (e.g. writing to a closed handle migth cause a fatal error and even terminate your program). Nevertheless, if you really want to retry after an error, here is the reset() method which resets the internal error flags - unless the associated handle was not already closed. Parameters: object An LDT object made by new(). Example: $ldtObject->reset; receive() --------- reads a message from the associated handle and replies it. In case of an error, the method replies an undefined value and provides both a return code (see CONSTANTS) and a complete message in the object variables "rc" and "msg", respectively, where you can read them. An error will occur, for example, if the handle related to the LDT object was closed (possibly outside the module). An error is detected as well if a previous call of send() or receive() already detected an error. This behaviour is implemented for reasons of security, however, if you want to try it again regardless of the objects history, you can reset the internal error state by reset(). For reasons of efficiency, sent messages may be splitted up into parts by the underlaying (operating or network) system. The reciepient will get the message part by part. That is why this receive() method retries reading attempts to the associated handle until the complete message could be read. Well, in fact it stops retries earlier if an inacceptable long period of time passed by without being successfull. If that happens, the method replies undef and provides an error code in the object variable "rc". *The caller should be* *prepared to handle such cases. Usually further access to the associated handle* *is useless or even dangerous.* Parameters: object An LDT object made by new(). The received message is replied as a string in ASCII mode, and as a list in object mode. Example: $msg=$asciiClient->receive or die $asciiClient->{'msg'}; @objects=$objectClient->receive or die $objectClient->{'msg'}; Note: If the connection is closed while the message is read, the signal *SIGPIPE* might arrive and terminate the complete program. To avoid this, *SIGPIPE* is ignored while this method is running. The handle associated with the LDT object is made *non blocking* during data transmission. The original mode is restored before the method returns. version() --------- replies the modules version. It simply replies $IPC::LDT::VERSION and is implemented only to provide compatibility to other object modules. Example: # get version warn "[Info] IPC is performed by IPC::LDT ", IPC::LDT::version, ".\n"; ENVIRONMENT =========== FILES ===== SEE ALSO ======== NOTES ===== EXAMPLE ======= To share data between processes, you could embed a socket into an LDT object. my $ipc=new IO::Socket(...); my $ldt=new IPC::LDT(handle=>$ipc, objectMode=>1); Now you are able to send data: my $dataRef=[{o=>1, lal=>2, a=>3}, [[qw(4 5 6)], [{oo=>'ps'}, 7, 8, 9]]]; $ldt->send($dataRef) or die $ldt->{'msg'}; or receive them: @data=$ldt->receive or die $ldt->{'msg'}; AUTHOR ====== Jochen Stenzel (perl@jochen-stenzel.de) COPYRIGHT ========= Copyright (c) 1998-2000 Jochen Stenzel. All rights reserved. This program is free software, you can redistribute it and/or modify it under the terms of the Artistic License distributed with Perl version 5.003 or (at your option) any later version. Please refer to the Artistic License that came with your Perl distribution for more details.  File: pm.info, Node: IPC/Locker, Next: IPC/Locker/Server, Prev: IPC/LDT, Up: Module List Distributed lock handler ************************ NAME ==== IPC::Locker - Distributed lock handler SYNOPSIS ======== use IPC::Locker; my $lock = IPC::Locker->lock(lock=>'one_per_machine', host=>'example.std.com', port=>223); if ($lock->lock()) { something; } if ($lock->locked()) { something; } $lock->unlock(); DESCRIPTION =========== `IPC::Locker' will query a remote server to obtain a lock. This is useful for distributed utilities which run on many machines, and cannot use file locks or other such mechanisms due to NFS or lack of common file systems. new ([parameter=>value ...]); Create a lock structure. lock ([parameter=>value ...]); Try to obtain the lock, return the lock object if successful, else undef. locked () Return true if the lock has been obtained. unlock () Remove the given lock. This will be called automatically when the object is destroyed. break_lock () Remove current locker for the given lock. owner ([parameter=>value ...]); Returns a string of who has the lock or undef if not currently . Note that this information is not atomic, and may change asyncronously; do not use this to tell if the lock will be available, to do that, try to obtain the lock and then release it if you got it. PARAMETERS ========== block Boolean flag, true indicates wait for the lock when calling lock() and die if a error occurs. False indicates to just return false. Defaults to true. family The family of transport to use, either INET or UNIX. Defaults to INET. host The name of the host containing the lock server. It may also be a array of hostnames, where if the first one is down, subsequent ones will be tried. port The port number (INET) or name (UNIX) of the lock server. Defaults to 'lockerd' looked up via /etc/services, else 1751. lock The name of the lock. print_broke A function to print a message when the lock is broken. The only argument is self. Defaults to print a message if verbose is set. print_obtained A function to print a message when the lock is obtained after a delay. The only argument is self. Defaults to print a message if verbose is set. print_waiting A function to print a message when the lock is busy and needs to be waited for. The first argument is self, second the name of the lock. Defaults to print a message if verbose is set. print_down A function to print a message when the lock server is unavailable. The first argument is self. Defaults to a croak message. timeout The maximum time in seconds that the lock may be held before being forced open, passed to the server when the lock is created. Thus if the requestor dies, the lock will be released after that amount of time. Defaults to 10 minutes. user Name to request the lock under, defaults to host_pid_user verbose True to print messages when waiting for locks. Defaults false. SEE ALSO ======== `lockerd', DISTRIBUTION ============ This package is distributed via CPAN. AUTHORS ======= Wilson Snyder  File: pm.info, Node: IPC/Locker/Server, Next: IPC/MM, Prev: IPC/Locker, Up: Module List Distributed lock handler server ******************************* NAME ==== IPC::Locker::Server - Distributed lock handler server SYNOPSIS ======== use IPC::Locker::Server; IPC::Locker::Server->start_server(port=>1234,); DESCRIPTION =========== `IPC::Locker::Server' provides the server for the IPC::Locker package. start_server ([parameter=>value ...]); Starts the server. Does not return. PARAMETERS ========== family The family of transport to use, either INET or UNIX. Defaults to INET. port The port number (INET) or name (UNIX) of the lock server. Defaults to 'lockerd' looked up via /etc/services, else 1751. SEE ALSO ======== `lockerd', `IPC::Locker', DISTRIBUTION ============ This package is distributed via CPAN. AUTHORS ======= Wilson Snyder  File: pm.info, Node: IPC/MM, Next: IPC/Msg, Prev: IPC/Locker/Server, Up: Module List Perl interface to Ralf Engelschall's mm library *********************************************** NAME ==== IPC::MM - Perl interface to Ralf Engelschall's mm library SYNOPSIS ======== use IPC::MM; $MMSIZE = 65536; $MMFILE = 'mm_file'; $mm = mm_create($MMSIZE, $MM_FILE); $scalar = mm_make_scalar($mm); tie $tied_scalar, 'IPC::MM::Scalar', $scalar; $tied_scalar = 'hello'; $btree = mm_make_btree_table($mm); tie %tied_hash, 'IPC::MM::BTree', $btree; $tied_hash{key} = 'val'; $hash = mm_make_hash($mm); tie %tied_hash, 'IPC::MM::Hash', $hash; $tied_hash{key} = 'val'; $num = mm_maxsize(); $num = mm_available($mm); $errstr = mm_error(); mm_display_info($mm); mm_destroy($mm); DESCRIPTION =========== IPC::MM provides an interface to Ralf Engelschall's mm library, allowing memory to be shared between multiple processes in a relatively convenient way. IPC::MM provides methods to create and destoy shared memory segments and to access data structures within those shared memory segments, as well as miscellaneous methods. Additionally, it provides a tied interface for scalars and hashes. METHODS ======= $mm = mm_create($size, $file) This method creates a shared memory segment. It corresponds to the function in mm of the same name. $size is the size of the shared memory segment, in bytes. A size of 0 means to allocate the maximum allowed size which is platform dependent. $file is a filesystem path to a file which may be used as a lock file for synchronizing access. $rc = mm_permission($mm, $mode, $owner, $group) This method sets the filesystem mode, owner, and group for the shared memory segment mm. It will only do anything when the underlying shared memory segment is based on files. It corresponds to the function in mm of the same name. $mm is the shared memory segment returned by *mm_create*. $mode, *$owner*, and *$group* are passed directly to chown and chmod. mm_destroy($mm) This method destroys a shared memory segment created by *mm_create*. $mm is the shared memory segment returned by *mm_create*. $scalar = mm_make_scalar($mm) mm_free_scalar($scalar) $val = mm_scalar_get($scalar) $rc = mm_scalar_set($scalar, $val) This family of methods provides a data structure for use by scalar variables. *mm_make_scalar* allocates the data structure used by the scalar. *mm_free_scalar* frees a data structure created by *mm_make_scalar*. *mm_scalar_get* returns the contents of the scalar, $scalar. *mm_scalar_set* sets the contents of the scalar, $scalar, to $val. $val is simply a Perl scalar value, meaning that it can be a string, a number, a reference, et al. It is possible for *mm_scalar_set* to fail if there is not enough shared memory. It is possible to make the scalar a tied variable, like so: tie $tied_scalar, 'IPC::MM::Scalar', $scalar; $btree = mm_make_btree_table($mm) mm_clear_btree_table($btree) mm_free_btree_table($btree) $val = mm_btree_table_get($btree, $key) $rc = mm_btree_table_insert($btree, $key, $val) $oldval = mm_btree_table_delete($btree, $key) $rc = mm_btree_table_exists($btree, $key) $key = mm_btree_table_first_key($btree) $key = mm_btree_table_next_key($btree, $key) This family of methods provides a btree data structure for use by hashes. *mm_make_btree_table* allocates the data structure. *mm_clear_btree_table* clears the data structure, making it empty. *mm_free_btree_table* frees the data structure. *mm_btree_table_get* returns the value associated with $key. *mm_btree_table_insert* inserts a new entry into the btree, with $key equal to $val. *mm_btree_table_delete* deletes the entry in the btree identified by $key. *mm_btree_table_exists* tests for the existence of an entry in the btree identified by $key. *mm_btree_table_first_key* returns the first key in the btree. *mm_btree_table_next_key* returns the next key after $key in the btree. It is possible to tie a btree to a hash, like so: tie %tied_hash, 'IPC::MM::BTree', $btree; One interesting characteristic of the btree is that it is presorted, so keys %tied_hash will return a sorted list of items. $hash = mm_make_hash($mm) mm_hash_clear($hash) mm_free_hash($hash) $val = mm_hash_get($hash, $key) $rc = mm_hash_insert($hash, $key, $val) $oldval = mm_hash_delete($hash, $key) $rc = mm_hash_exists($hash, $key) $key = mm_hash_first_key($hash) $key = mm_hash_next_key($hash, $key) This family of methods implements a shared memory hash list. These hash lists are not presorted like btrees, but they can be faster than btrees (especially unbalanced btrees). *mm_make_hash* allocates the data structure. *mm_clear_hash* clears the data structure, making it empty. *mm_free_hash* frees the data structure. *mm_hash_get* returns the value associated with $key. *mm_hash_insert* inserts a new entry into the hash, with $key equal to $val. *mm_hash_delete* deletes the entry in the hash identified by $key. *mm_hash_exists* tests for the existence of an entry in the hash identified by $key. *mm_hash_first_key* returns the first key in the hash. *mm_hash_next_key* returns the next key after $key in the hash. It is possible to tie a shared-memory hash to a perl hash, like so: tie %tied_hash, 'IPC::MM::Hash', $hash; $rc = mm_lock($mm, $mode) This method locks the shared memory pool $mm for the current process in order to perform either shared/read-only (mode MM_LOCK_RD) or exclusive/read-write (mode MM_LOCK_RW) operations. $rc = mm_unlock($mm) This method unlocks the shared memory pool $mm. $num = mm_maxsize This method returns the maximum allowable size for a shared memory segment. It corresponds to the function of the same name in the mm library. $num = mm_available($mm) This method returns the number of free bytes left in a shared memory segment. It corresponds to the function of the same name in the mm library. $mm is a shared memory segment created by mm_create. $errstr = mm_error This method returns an error string, if any. It corresponds to the function of the same name in mm. mm_display_info($mm) This method displays some miscellaneous information about a shared memory segment. It corresponds to the function of the same name in mm. BUGS ==== No effort is made to balance the btree. AUTHOR ====== Copyright (c) 1999, Arthur Choung . All rights reserved. This module is free software; you may redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO ======== `mm' in this node, *Note IPC/Shareable: IPC/Shareable,, perl. perl(1).  File: pm.info, Node: IPC/Msg, Next: IPC/Open2, Prev: IPC/MM, Up: Module List SysV Msg IPC object class ************************* NAME ==== IPC::Msg - SysV Msg IPC object class SYNOPSIS ======== use IPC::SysV qw(IPC_PRIVATE S_IRWXU S_IRWXG S_IRWXO); use IPC::Msg; $msg = new IPC::Msg(IPC_PRIVATE, S_IRWXU | S_IRWXG | S_IRWXO); $msg->snd(pack("L a*",$msgtype,$msg)); $msg->rcv($buf,256); $ds = $msg->stat; $msg->remove; DESCRIPTION =========== METHODS ======= new ( KEY , FLAGS ) Creates a new message queue associated with KEY. A new queue is created if * KEY is equal to `IPC_PRIVATE' * KEY does not already have a message queue associated with it, and `*FLAGS* & IPC_CREAT' is true. On creation of a new message queue FLAGS is used to set the permissions. id Returns the system message queue identifier. rcv ( BUF, LEN [, TYPE [, FLAGS ]] ) Read a message from the queue. Returns the type of the message read. See `msgrcv' in this node remove Remove and destroy the message queue from the system. set ( STAT ) set ( NAME => VALUE [, NAME => VALUE ...] ) set will set the following values of the stat structure associated with the message queue. uid gid mode (oly the permission bits) qbytes set accepts either a stat object, as returned by the stat method, or a list of name-value pairs. snd ( TYPE, MSG [, FLAGS ] ) Place a message on the queue with the data from MSG and with type TYPE. See `msgsnd' in this node. stat Returns an object of type `IPC::Msg::stat' which is a sub-class of Class::Struct. It provides the following fields. For a description of these fields see you system documentation. uid gid cuid cgid mode qnum qbytes lspid lrpid stime rtime ctime SEE ALSO ======== *Note IPC/SysV: IPC/SysV, *Note Class/Struct: Class/Struct, AUTHOR ====== Graham Barr COPYRIGHT ========= Copyright (c) 1997 Graham Barr. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.  File: pm.info, Node: IPC/Open2, Next: IPC/Open3, Prev: IPC/Msg, Up: Module List open a process for both reading and writing ******************************************* NAME ==== IPC::Open2, open2 - open a process for both reading and writing SYNOPSIS ======== use IPC::Open2; $pid = open2(\*RDRFH, \*WTRFH, 'some cmd and args'); # or without using the shell $pid = open2(\*RDRFH, \*WTRFH, 'some', 'cmd', 'and', 'args'); # or with handle autovivification my($rdrfh, $wtrfh); $pid = open2($rdrfh, $wtrfh, 'some cmd and args'); # or without using the shell $pid = open2($rdrfh, $wtrfh, 'some', 'cmd', 'and', 'args'); DESCRIPTION =========== The open2() function runs the given $cmd and connects $rdrfh for reading and $wtrfh for writing. It's what you think should work when you try $pid = open(HANDLE, "|cmd args|"); The write filehandle will have autoflush turned on. If $rdrfh is a string (that is, a bareword filehandle rather than a glob or a reference) and it begins with `< '& >>, then the child will send output directly to that file handle. If $wtrfh is a string that begins with `< <& '>, then $wtrfh will be closed in the parent, and the child will read from it directly. In both cases, there will be a dup(2) instead of a pipe(2) made. If either reader or writer is the null string, this will be replaced by an autogenerated filehandle. If so, you must pass a valid lvalue in the parameter slot so it can be overwritten in the caller, or an exception will be raised. open2() returns the process ID of the child process. It doesn't return on failure: it just raises an exception matching `/^open2:/'. However, exec failures in the child are not detected. You'll have to trap SIGPIPE yourself. open2() does not wait for and reap the child process after it exits. Except for short programs where it's acceptable to let the operating system take care of this, you need to do this yourself. This is normally as simple as calling `waitpid $pid, 0' when you're done with the process. Failing to do this can result in an accumulation of defunct or "zombie" processes. See `waitpid', *Note Perlfunc: (perl.info)perlfunc, for more information. This whole affair is quite dangerous, as you may block forever. It assumes it's going to talk to something like *bc*, both writing to it and reading from it. This is presumably safe because you "know" that commands like *bc* will read a line at a time and output a line at a time. Programs like sort that read their entire input stream first, however, are quite apt to cause deadlock. The big problem with this approach is that if you don't have control over source code being run in the child process, you can't control what it does with pipe buffering. Thus you can't just open a pipe to `cat -v' and continually read and write a line from it. The IO::Pty and Expect modules from CPAN can help with this, as they provide a real tty (well, a pseudo-tty, actually), which gets you back to line buffering in the invoked command again. WARNING ======= The order of arguments differs from that of open3(). SEE ALSO ======== See *Note IPC/Open3: IPC/Open3, for an alternative that handles STDERR as well. This function is really just a wrapper around open3().  File: pm.info, Node: IPC/Open3, Next: IPC/Run, Prev: IPC/Open2, Up: Module List open a process for reading, writing, and error handling ******************************************************* NAME ==== IPC::Open3, open3 - open a process for reading, writing, and error handling SYNOPSIS ======== $pid = open3(\*WTRFH, \*RDRFH, \*ERRFH, 'some cmd and args', 'optarg', ...); my($wtr, $rdr, $err); $pid = open3($wtr, $rdr, $err, 'some cmd and args', 'optarg', ...); DESCRIPTION =========== Extremely similar to open2(), open3() spawns the given $cmd and connects RDRFH for reading, WTRFH for writing, and ERRFH for errors. If ERRFH is false, or the same file descriptor as RDRFH, then STDOUT and STDERR of the child are on the same filehandle. The WTRFH will have autoflush turned on. If WTRFH begins with `< <& '>, then WTRFH will be closed in the parent, and the child will read from it directly. If RDRFH or ERRFH begins with `< '& >>, then the child will send output directly to that filehandle. In both cases, there will be a dup(2) instead of a pipe(2) made. If either reader or writer is the null string, this will be replaced by an autogenerated filehandle. If so, you must pass a valid lvalue in the parameter slot so it can be overwritten in the caller, or an exception will be raised. open3() returns the process ID of the child process. It doesn't return on failure: it just raises an exception matching `/^open3:/'. However, exec failures in the child are not detected. You'll have to trap SIGPIPE yourself. open2() does not wait for and reap the child process after it exits. Except for short programs where it's acceptable to let the operating system take care of this, you need to do this yourself. This is normally as simple as calling `waitpid $pid, 0' when you're done with the process. Failing to do this can result in an accumulation of defunct or "zombie" processes. See `waitpid', *Note Perlfunc: (perl.info)perlfunc, for more information. If you try to read from the child's stdout writer and their stderr writer, you'll have problems with blocking, which means you'll want to use select() or the IO::Select, which means you'd best use sysread() instead of readline() for normal stuff. This is very dangerous, as you may block forever. It assumes it's going to talk to something like *bc*, both writing to it and reading from it. This is presumably safe because you "know" that commands like *bc* will read a line at a time and output a line at a time. Programs like sort that read their entire input stream first, however, are quite apt to cause deadlock. The big problem with this approach is that if you don't have control over source code being run in the child process, you can't control what it does with pipe buffering. Thus you can't just open a pipe to `cat -v' and continually read and write a line from it. WARNING ======= The order of arguments differs from that of open2().  File: pm.info, Node: IPC/Run, Next: IPC/Run/IO, Prev: IPC/Open3, Up: Module List Run subprocesses w/ piping, redirection, and pseudo-ttys ******************************************************** NAME ==== IPC::Run - Run subprocesses w/ piping, redirection, and pseudo-ttys SYNOPSIS ======== use IPC::Run ( qw( run ), # Batch mode API qw( harness start pump finish ), # Intermittent API qw( io timer timeout ), # Helper objects. ) ; my @cat = qw( cat in.txt - ) ; ## Running a command with stdin closed, output redirected ## to scalars and a 10 second timeout: run \@cat, \undef, \$out, \$err, timeout( 10 ) or die "cat returned $?"; ## Redirecting stdin from various scalars: $in = "input" ; run \@cat, \$in, \$out, \$err, ... ; run \@cat, \"input", \$out, \$err, ... ; run \@cat, \<', $out_name, '2>', $err_name ... ; ## Redirecting using psuedo-terminals. run \@cat, 'pty>', \$out_and_err ; run \@cat, 'pty>', \&out_and_err ; # Incrementally read from / write to scalars. Note that $in_q # is a queue that is drained as it is used, which is not the # way run() treats inputs. $h is for "harness". my $h = start( \@cat, \$in_q, \$out_q, \$err_q, timeout( name => "process timeout", 10 ), $stall_timeout = timeout( name => "stall timeout", 5 ), ) ; $in_q .= "some input\n" ; $stall_timeout->start ; pump $h until $out_q =~ /input/ ; warn $err_q ; $err_q = '' ; $out_q = '' ; $stall_timeout->start ; $in_q .= "some more input\n" ; pump $h until $out_q =~ /input/ ; warn $err_q ; $err_q = '' ; $in_q .= "some final input\n" ; $h->timeout( "1:00" ) ; $stall_timeout->start ; finish $h or die "cat returned $?" ; warn $err_q ; print $out_q ; # Piping between children run \@cat, '|', [qw(gzip)] ; # Multiple children simultaneously (run() blocks until all # children exit, use start() for background execution): run [qw(foo1)], '&', [qw(foo2)] ; # Spawning a subroutine (only on systems with true fork()/exec()) run \&child_sub, \$in, \$out ; # Calling \&set_up_child in the child before it executes the # command (only works on systems with true fork() & exec()) run \@cat, \$in, \$out, init => \&set_up_child ; # Append output to named files run( \@cat, '>>', $out_f_name, '2>>', $err_f_name ) ; # Read from / write to file handles you open and close open IN, 'out.txt' or die $! ; print OUT "preamble\n" ; run \@cat, \*IN, \*OUT or die "cat returned $?" ; print OUT "postamble\n" ; close IN ; close OUT ; # Create pipes for you to read / write (like IPC::Open2 & 3). $h = start \@cat, 'pipe', \*OUT, '2>pipe', \*ERR or die "cat returned $?" ; print IN "some input\n" ; close IN ; print , ; finish $h ; # Mixing input and output modes run \@cat, 'in.txt', \&catch_some_out, \*ERR_LOG ) ; # Other redirection constructs run \@cat, '>&', \$out_and_err ; run \@cat, '2>&1' ; run \@cat, '0<&3' ; run \@cat, '<&-' ; run \@cat, '3<', \$in3 ; run \@cat, '4>', \$out4 ; # etc. # Passing options: run \@cat, 'in.txt', debug => 1 ; # Call this system's shell, returns TRUE on 0 exit code # THIS IS THE OPPOSITE SENSE OF system()'s RETURN VALUE run "cat a b c" or die "cat returned $?" ; # Launch a sub process directly, no shell. Can't do redirection # with this form, it's here to behave like system() with an # inverted result. $r = run "cat a b c" ; # Read from a file in to a scalar run io( "filename", 'r', \$recv ) ; run io( \*HANDLE, 'r', \$recv ) ; DESCRIPTION =========== IPC::Run allows you run and interact with child processes, files, pipes, and pseudo-ttys. Both event-loop and procedural techniques are supported and may be mixed. Likewise, functional and OO API styles are both supported and may be mixed. Various redirection operators reminiscent of those seen on common Unix and DOS command lines are provided. Harnesses --------- Child processes and I/O handles are gathered in to a harness, then run until the processing is finished or aborted. run() vs. start(); pump(); finish(); ------------------------------------ There are two modes you can run harnesses in: incremental and batch. In incremental mode, you use start() and pump() to activate the harness and interact with it, then switch to batch mode and use run() or finish() to let the harness run to completion. In batch mode, you just use run() or start() and finish() and let things run to completion. In batch mode, all data to be sent to the harness is set up in advance, the harness is run and all output is collected from it, then any child processes are waited for: run \@cmd, \<finish ; You can compile the harness with harness() prior to start()ing, and you may omit start() between harness() and pump(). You might want to do these things if you compile your harnesses ahead of time. You may call timeout() at any time in this process. If it is called before start() (or the first pump(), if start() is omitted), then the timer is started just after all the child processes have been fork()ed or spawn()ed. If you call it once the children have been start()ed, the timeout timer is restarted. So this sequence: my $h = harness \@cat, \$in_q, \$out_q, \$err_q ; $h->timeout( 10 ) ; sleep 11 ; # No effect: timer not running yet start $h ; $in_q = "foo\n" ; pump $h until ! length $in_q ; ## 10 (or little more) may pass til this line. $in_q = "bar\n" ; $h->timeout( 1 ) ; ## Timer is now running, but will elapse after 1 second pump $h until ! length $in_q ; ## 1 seconds (or a little more) may pass til this line. $h->timeout( 30 ) ; $h->finish ; The timeout does not include the time it takes to wait for the children to exit, either. Syntax ------ run(), start(), and harness() can all take a harness specification as input. A harness specification is either a single string to be passed to the systems' shell: run "echo 'hi there'" ; or a list of commands, io operations, and/or timers/timeouts to execute. Consecutive commands must be separated by a pipe operator '|' or an '&'. External commands are passed in as array references, and, on systems supporting fork(), Perl code may be passed in as subs: run \@cmd ; run \@cmd1, '|', \@cmd2 ; run \@cmd1, '&', \@cmd2 ; run \&sub1 ; run \&sub1, '|', \&sub2 ; run \&sub1, '&', \&sub2 ; '|' pipes the stdout of \@cmd1 the stdin of \@cmd2, just like a shell pipe. '&' does not. Child processes to the right of a '&' will have their stdin closed unless it's redirected-to. *Note IPC/Run/IO: IPC/Run/IO, objects may be passed in as well, whether or not child processes are also specified: run io( "infile", ">", \$in ), io( "outfile", "<", \$in ) ; as can L objects: run \@cmd, io( "outfile", "<", \$in ), timeout( 10 ) ; Commands may be followed by scalar, sub, or i/o handle references for redirecting child process input & output: run \@cmd, \undef, \$out ; run \@cmd, \$in, \$out ; run \@cmd1, \&in, '|', \@cmd2, \*OUT ; run \@cmd1, \*IN, '|', \@cmd2, \&out ; This is known as succinct redirection syntax, since run(), start() and harness(), figure out which file descriptor to redirect and how. File descriptor 0 is presumed to be an input for the child process, all others are outputs. The assumed file descriptor always starts at 0, unless the command is being piped to, in which case it starts at 1. To be explicit about your redirects, or if you need to do more complex things, there's also a redirection operator syntax: run \@cmd, '<', \undef, '>', \$out ; run \@cmd, '<', \undef, '>&', \$out_and_err ; run( \@cmd1, '<', \$in, '|', \@cmd2, \$out ) ; Operator syntax is required if you need to do something other than simple redirection to/from scalars or subs, like duping or closing file descriptors or redirecting to/from a named file. The operators are covered in detail below. After each \@cmd (or \&foo), parsing begins in succinct mode and toggles to operator syntax mode when an operator (ie plain scalar, not a ref) is seen. Once in operator syntax mode, parseing only reverts to succinct mode when a '|' or '&' is seen. In succinct mode, each parameter after the \@cmd specifies what to do with the next highest file descriptor. These File descriptor start with 0 (stdin) unless stdin is being piped to (`'|', \@cmd'), in which case they start with 1 (stdout). Currently, being on the left of a pipe (`\@cmd, \$out, \$err, '|'') does not cause stdout to be skipped, though this may change since it's not as DWIMerly as it could be. Only stdin is assumed to be an input in succinct mode, all others are assumed to be outputs. If no piping or redirection is specified for a child, it will inherit the parent's open file descriptors as dictated by your system's close-on-exec behavior and the $^F flag, except that processes after a '&' will not inherit the parent's stdin. Such processes will have their stdin closed unless it has been redirected-to. If you want to close a child processes stdin, you may do any of: run \@cmd, \undef ; run \@cmd, \"" ; run \@cmd, '<&-' ; run \@cmd, '0<&-' ; Redirection is done by placing redirection specifications immediately after a command or child subroutine: run \@cmd1, \$in, '|', \@cmd2, \$out ; run \@cmd1, '<', \$in, '|', \@cmd2, '>', \$out ; If you omit the redirection operators, descriptors are counted starting at 0. Descriptor 0 is assumed to be input, all others are outputs. A leading '|' consumes descriptor 0, so this works as expected. run \@cmd1, \$in, '|', \@cmd2, \$out ; The parameter following a redirection operator can be a scalar ref, a subroutine ref, a file name, an open filehandle, or a closed filehandle. If it's a scalar ref, the child reads input from or sends output to that variable: $in = "Hello World.\n" ; run \@cat, \$in, \$out ; print $out ; Scalars used in incremental (start()/pump()/finish()) applications are treated as queues: input is removed from input scalers, resulting in them dwindling to ", and output is appended to output scalars. This is not true of harnesses run() in batch mode. It's usually wise to append new input to be sent to the child to the input queue, and you'll often want to zap output queues to " before pumping. $h = start \@cat, \$in_q ; $in_q = "line 1\n" ; pump $h ; $in_q .= "line 2\n" ; pump $h ; $in_q .= "line 3\n" ; finish $h ; The final call to finish() must be there: it allows the child process(es) to run to completion and waits for their exit values. OBSTINATE CHILDREN ================== Interactive applications are usually optimized for human use. This can help or hinder trying to interact with them through modules like IPC::Run. Frequently, programs alter their behavior when they detect that stdin, stdout, or stderr are not connected to a tty, assuming that they are being run in batch mode. Whether this helps or hurts depends on which optimizations change. And there's often no way of telling what a program does in these areas other than trial and error and, occasionally, reading the source. This includes different versions and implementations of the same program. All hope is not lost, however. Most programs behave in reasonably tractable manners, once you figure out what it's trying to do. Here are some of the issues you might need to be aware of. * fflush()ing stdout and stderr This lets the user see stdout and stderr immediately. Many programs undo this optimization if stdout is not a tty, making them harder to manage by things like IPC::Run. Many programs decline to fflush stdout or stderr if they do not detect a tty there. Some ftp commands do this, for instance. If this happens to you, look for a way to force interactive behavior, like a command line switch or command. If you can't, you will need to use a pseudo terminal ('pty>'). * false prompts Interactive programs generally do not guarantee that output from user commands won't contain a prompt string. For example, your shell prompt might be a '$', and a file named '$' might be the only file in a directory listing. This can make it hard to guarantee that your output parser won't be fooled into early termination of results. To help work around this, you can see if the program can alter it's prompt, and use something you feel is never going to occur in actual practice. You should also look for your prompt to be the only thing on a line: pump $h until $out =~ /^\s?\z/m ; (use `(?!\n)\Z' in place of `\z' on older perls). You can also take the approach that IPC::ChildSafe takes and emit a command with known output after each 'real' command you issue, then look for this known output. See new_appender() and new_chunker() for filters that can help with this task. If it's not convenient or possibly to alter a prompt or use a known command/response pair, you might need to autodetect the prompt in case the local version of the child program is different then the one you tested with, or if the user has control over the look & feel of the prompt. * Refusing to accept input unless stdin is a tty. Some programs, for security reasons, will only accept certain types of input from a tty. su, notable, will not prompt for a password unless it's connected to a tty. If this is your situation, use a pseudo terminal ('pty>'). * Not prompting unless connected to a tty. Some programs don't prompt unless stdin or stdout is a tty. See if you can turn prompting back on. If not, see if you can come up with a command that you can issue after every real command and look for it's output, as IPC::ChildSafe does. There are two filters included with IPC::Run that can help with doing this: appender and chunker (see new_appender() and new_chunker()). * Different output format when not connected to a tty. Some commands alter their formats to ease machine parsability when they aren't connected to a pipe. This is actually good, but can be suprising. PSEUDO TERMINALS ================ On systems providing pseudo terminals under /dev, IPC::Run can use IO::Pty (available on CPAN) to provide a terminal environment to subprocesses. This is necessary when the subprocess really wants to think it's connected to a real terminal. CAVEATS ------- Psuedo-terminals are not pipes, though they are similar. Here are some differences to watch out for. Echoing Sending to stdin will cause an echo on stdout, which occurs before each line is passed to the child program. There is currently no way to disable this. Shutdown IPC::Run cannot close a pty until all output has been collected. This means that it is not possible to send an EOF to stdin by half-closing the pty, as we can when using a pipe to stdin. This means that you need to send the child process an exit command or signal, or run() / finish() will time out. Be careful not to expect a prompt after sending the exit command. Command line editing Some subprocesses, notable shells that depend on the user's prompt settings, will reissue the prompt plus the command line input so far once for each character. '>pty>' means '&>pty>', not '1>pty>' The pseudo terminal redirects both stdin and stdout unless you specify a file desciptor. If you want to grab stderr separately, do this: start \@cmd, 'pty>', \$out, '2>', \$err ; stdin, stdout, and stderr not inherited Child processes harnessed to a pseudo terminal have their stdin, stdout, and stderr completely closed before any redirection operators take effect. This casts of the bonds of the controlling terminal. This is not done when using pipes. Right now, this affects all children in a harness that has a pty in use, even if that pty would not affect a particular child. That's a bug and will be fixed. Until it is, it's best not to mix-and-match children. Redirection Operators --------------------- Operator SHNP Description ======== ==== =========== <, N< SHN Redirects input to a child's fd N (0 assumed) >, N> SHN Redirects output from a child's fd N (1 assumed) >>, N>> SHN Like '>', but appends to scalars or named files >&, &> SHN Redirects stdout & stderr from a child process pty, N>pty S Like '>', but uses a pseudo-tty instead of a pipe N<&M Dups input fd N to input fd M M>&N Dups output fd N to input fd M N<&- Closes fd N pipe, N>pipe P Pipe opens H for caller to read, write, close. 'N' and 'M' are placehodlers for integer file descriptor numbers. The terms 'input' and 'output' are from the child process's perspective. The SHNP field indicates what parameters an operator can take: S: \$scalar or \&function references. Filters may be used with these operators (and only these). H: \*HANDLE or IO::Handle for caller to open, and close N: "file name". P: \*HANDLE opened by IPC::Run as the parent end of a pipe, but read and written to and closed by the caller (like IPC::Open3). Redirecting input: [n]<, [n], [n]>>, [n]>&[m], [n]>pipe You can redirect any output the child emits to a scalar variable, subroutine, file handle, or file name. You can have &run truncate or append to named files or scalars. If you are redirecting stdin as well, or if the command is on the receiving end of a pipeline ('|'), you can omit the redirection operator: @ls = ( 'ls' ) ; run \@ls, \undef, \$out or die "ls returned $?" ; run \@ls, \undef, \&out ; ## Calls &out each time some output ## is received from the child's ## when undef is returned. run \@ls, \undef, '2>ls.err' ; run \@ls, '2>', 'ls.err' ; The two parameter form guarantees that the filename will not be interpreted as a redirection operator: run \@ls, '>', "&more" ; run \@ls, '2>', ">foo\n" ; You can pass file handles you've opened for writing: open( *OUT, ">out.txt" ) ; open( *ERR, ">err.txt" ) ; run \@cat, \*OUT, \*ERR ; Passing a scalar reference and a code reference requires a little more work, but allows you to capture all of the output in a scalar or each piece of output by a callback: These two do the same things: run( [ 'ls' ], '2>', sub { $err_out .= $_[0] } ) ; does the same basic thing as: run( [ 'ls' ], '2>', \$err_out ) ; The subroutine will be called each time some data is read from the child. The >pipe operator is different in concept than the other '>' operators, although it's syntax is similar: $h = start \@cat, $in, '>pipe', \*OUT, '2>pipe', \*ERR ; $in = "hello world\n" ; finish $h ; print ; print ; close OUT ; close ERR ; causes two pipe to be created, with one end attached to cat's stdout and stderr, respectively, and the other left open on OUT and ERR, so that the script can manually read(), select(), etc. on them. This is like the behavior of IPC::Open2 and IPC::Open3. Duplicating output descriptors: >&m, n>&m This duplicates output descriptor number n (default is 1 if n is ommitted) from descriptor number m. Duplicating input descriptors: <&m, n<&m This duplicates input descriptor number n (default is 0 if n is ommitted) from descriptor number m Closing descriptors: <&-, 3<&- This closes descriptor number n (default is 0 if n is ommitted). The following commands are equivalent: run \@cmd, \undef ; run \@cmd, '<&-' ; run \@cmd, ', >&, &>pipe, >pipe& The following pairs of commands are equivalent: run \@cmd, '>&', \$out ; run \@cmd, '>', \$out, '2>&1' ; run \@cmd, '>&', 'out.txt' ; run \@cmd, '>', 'out.txt', '2>&1' ; etc. File descriptor numbers are not permitted to the left or the right of these operators, and the '&' may occur on either end of the operator. The '&>pipe' and '>pipe&' variants behave like the '>pipe' operator, except that both stdout and stderr write to the created pipe. Redirection Filters Both input redirections and output redirections that use scalars or subs as endpoints may have an arbitrary number of filter subs placed between them and the child process. This is useful if you want to receive output in chunks, or if you want to massage each chunk of data sent to the child. To use this feature, you must use operator syntax: run( \@cmd '<', \&in_filter_2, \&in_filter_1, $in, '>', \&out_filter_1, \&in_filter_2, $out, ) ; This capability is not provided for IO handles or named files. Two filters are provided by IPC::Run: appender and chunker. Because these may take an argument, you need to use the constructor fuinctions new_appender() and new_chunker() rather than using \& syntax: run( \@cmd '<', new_appender( "\n" ), $in, '>', new_chunker, $out, ) ; Just doing I/O -------------- If you just want to do I/O to a handle or file you open yourself, you may specify a filehandle or filename instead of a command in the harness specification: run io( "filename", '>', \$recv ) ; $h = start io( $io, '>', \$recv ) ; $h = harness \@cmd, '&', io( "file", '<', \$send ) ; Options ------- Options are passed in as name/value pairs: run \@cat, \$in, debug => 1 ; If you pass the debug option, you may want to pass it in first, so you can see what parsing is going on: run debug => 1, \@cat, \$in ; debug Enables debugging output in parent and child. Children emit debugging info over a separate, dedicated pipe to the parent, so you must pump(), pump_nb(), or finish() if you use start(). This pipe is not created unless debugging is enabled. RETURN VALUES ============= harness() and start() return a reference to an IPC::Run harness. This is blessed in to the IPC::Run package, so you may make later calls to functions as members if you like: $h = harness( ... ) ; $h->start ; $h->pump ; $h->finish ; $h = start( .... ) ; $h->pump ; ... Of course, using method call syntax lets you deal with any IPC::Run subclasses that might crop up, but don't hold your breath waiting for any. run() and finish() return TRUE when all subcommands exit with a 0 result code. *This is the opposite of perl's system() command*. All routines raise exceptions (via die()) when error conditions are recognized. A non-zero command result is not treated as an error condition, since some commands are tests whose results are reported in their exit codes. ROUTINES ======== run Run takes a harness or harness specification and runs it, pumping all input to the child(ren), closing the input pipes when no more input is available, collecting all output that arrives, until the pipes delivering output are closed, then waiting for the children to exit and reaping their result codes. You may think of `run( ... )' as being like start( ... )->finish() ; , though there is one subtle difference: run() does not set \$input_scalars to " like finish() does. harness Takes a harness specification and returns a harness. This harness is blessed in to IPC::Run, allowing you to use method call syntax for run(), start(), et al if you like. harness() is provided so that you can pre-build harnesses if you would like to, but it's not required.. You may proceed to run(), start() or pump() after calling harness() (pump() calls start() if need be). Alternatively, you may pass your harness specification to run() or start() and let them harness() for you. You can't pass harness specifications to pump(), though. close_terminal This is used as (or in) an init sub to cast off the bonds of a controlling terminal. It must precede all other redirection ops that affect STDIN, STDOUT, or STDERR to be guaranteed effective. start $h = start( \@cmd, \$in, \$out, ..., timeout( 30, name => "process timeout" ), $stall_timeout = timeout( 10, name => "stall timeout" ), ) ; $h = start \@cmd, '<', \$in, '|', \@cmd2, ... ; start() accepts a harness or harness specification and returns a harness after building all of the pipes and launching (via fork()/exec(), or, maybe someday, spawn()) all the child processes. It does not send or receive any data on the pipes, see pump() and finish() for that. You may call harness() and then pass it's result to start() if you like, but you only need to if it helps you structure or tune your application. If you do call harness(), you may skip start() and proceed directly to pump. start() also starts all timers in the harness. See *Note IPC/Run/Timer: IPC/Run/Timer, for more information. start() flushes STDOUT and STDERR to help you avoid duplicate output. It has no way of asking Perl to flush all your open filehandles, so you are going to need to flush any others you have open. Sorry. Here's how if you don't want to alter the state of $| for your filehandle: $ofh = select HANDLE ; $of = $| ; $| = 1 ; $| = $of ; select $ofh; If you don't mind leaving output unbuffered on HANDLE, you can do the slightly shorter $ofh = select HANDLE ; $| = 1 ; select $ofh; Or, you can use IO::Handle's flush() method: use IO::Handle ; flush HANDLE ; Perl needs the equivalent of C's fflush( (FILE *)NULL ). pump pump $h ; $h->pump ; Pump accepts a single parameter harness. It blocks until it delivers some input or recieves some output. It returns TRUE if there is still input or output to be done, FALSE otherwise. pump() will automatically call start() if need be, so you may call harness() then proceed to pump() if that helps you structure your application. Calling pump() when there is no more i/o do be done causes a "process ended prematurely" exception to be thrown. This allows fairly simple scripting of external applications without having to add lots of error handling code at each step of the script: $h = harness \@smbclient, \$in_q, \$out_q, $err_q ; $in_q = "cd /foo\n" ; $h->pump until $out_q =~ /^smb.*> \Z/m ; die "error cding to /foo:\n$out_q" if $out_q =~ "ERR" ; $out_q = '' ; $in_q = "mget *\n" ; $h->pump until $out_q =~ /^smb.*> \Z/m ; die "error retrieving files:\n$out_q" if $out_q =~ "ERR" ; $h->finish ; warn $err_q if $err_q ; pump_nb pump_nb $h ; $h->pump_nb ; "pump() non-blocking", pumps if anything's ready to be pumped, returns immediately otherwise. This is useful if you're doing some long-running task in the foreground, but don't want to starve any child processes. pumpable Returns TRUE if calling pump() won't throw an immediate "process ended prematurely" exception. Reflects the value returned from the last call to pump(). finish This must be called after the last start() or pump() call for a harness, or your system will accumulate defunct processes and you may "leak" file descriptors. finish() returns TRUE if all children returned 0 (and were not signaled and did not coredump, ie ! $?), and FALSE otherwise (this is like run(), and the opposite of system()). Once a harness has been finished, it may be run() or start()ed again, including by pump()s auto-start. result $h->result ; Returns the first non-zero result code (ie $? >> 8). See `' in this node to get the $? value for a child process. To get the result of a particular child, do: $h->result( 0 ) ; # first child's $? >> 8 $h->result( 1 ) ; # second child or ($h->results)[0] ($h->results)[1] Returns undef if no child processes were spawned and no child number was specified. Throws an exception if an out-of-range child number is passed. results Returns a list of child exit values. See `' in this node if you want to know if a signal killed the child. Throws an exception if the harness is not in a finished state. =cut sub results { &_assert_finished ; my IPC::Run $self = shift ; return map $_->{RESULT} >> 8, @{$self->{KIDS}} ; } full_result $h->full_result ; Returns the first non-zero $?. See `' in this node to get the first $? >> 8 value for a child process. To get the result of a particular child, do: $h->full_result( 0 ) ; # first child's $? >> 8 $h->full_result( 1 ) ; # second child or ($h->full_results)[0] ($h->full_results)[1] Returns undef if no child processes were spawned and no child number was specified. Throws an exception if an out-of-range child number is passed. full_results Returns a list of child exit values as returned by wait. See `' in this node if you don't care about coredumps or signals. Throws an exception if the harness is not in a finished state. =cut sub full_results { &_assert_finished ; my IPC::Run $self = shift ; croak "Harness not run" unless $self->{STATE} >= _finished ; croak "Harness not finished running" unless $self->{STATE} == _finished ; return map $_->{RESULT}, @{$self->{KIDS}} ; } ## ## Filter Scaffolding ## use vars ( '$filter_op', ## The op running a filter chain right now '$filter_num' ## Which filter is being run right now. ) ; ## ## A few filters and filter constructors ## FILTERS ======= These filters are used to modify input our output between a child process and a scalar or subroutine endpoint. new_chunker This breaks a stream of data in to chunks, based on an optional scalar or regular expression parameter. The default is the Perl input record separator in $/, which is a newline be default. run( \@cmd, '>', new_chunker, \&lines_handler, ) ; run( \@cmd, '>', new_chunker( "\r\n" ), \&lines_handler, ) ; Because this uses $/ by default, you should always pass in a parameter if you are worried about other code (modules, etc) modifying $/. If this filter is last in a filter chain that dumps in to a scalar, the scalar must be set to " before a new chunk will be written to it. As an example of how a filter like this can be written, here's a chunker that splits on newlines: sub line_splitter { my ( $in_ref, $out_ref ) = @_ ; return 0 if length $$out_ref ; return input_avail && do { while (1) { if ( $$in_ref =~ s/\A(.*?\n)// ) { $$out_ref .= $1 ; return 1 ; } my $hmm = get_more_input ; unless ( defined $hmm ) { $$out_ref = $$in_ref ; $$in_ref = '' ; return length $$out_ref ? 1 : 0 ; } return 0 if $hmm eq 0 ; } } } ; new_appender This appends a fixed string to each chunk of data read from the source scalar or sub. This might be useful if you're writing commands to a child process that always must end in a fixed string, like "\n": run( \@cmd, '<', new_appender( "\n" ), \&commands, ) ; Here's a typical filter sub that might be created by new_appender(): sub newline_appender { my ( $in_ref, $out_ref ) = @_ ; return input_avail && do { $$out_ref = join( '', $$out_ref, $$in_ref, "\n" ) ; $$in_ref = '' ; 1 ; } } ; timer $h = start( \@cmd, \$in, \$out, $t = timer( 5 ) ) ; pump $h until $out =~ /expected stuff/ || $t->is_expired ; Instantiates a non-fatal timer. pump() returns once each time a timer expires. Has no direct effect on run(), but you can pass a subroutine to fire when the timer expires. See `' in this node for building timers that throw exceptions on expiration. See `timer', *Note IPC/Run/Timer: IPC/Run/Timer, for details. timeout $h = start( \@cmd, \$in, \$out, $t = timeout( 5 ) ) ; pump $h until $out =~ /expected stuff/ ; Instantiates a timer that throws an exception when it expires. If you don't provide an exception, a default exception that matches /^IPC::Run: .*timed out/ is thrown by default. You can pass in your own exception scalar or reference: $h = start( \@cmd, \$in, \$out, $t = timeout( 5, exception => 'slowpoke' ), ) ; or set the name used in debugging message and in the default exception string: $h = start( \@cmd, \$in, \$out, timeout( 50, name => 'process timer' ), $stall_timer = timeout( 5, name => 'stall timer' ), ) ; pump $h until $out =~ /started/ ; $in = 'command 1' ; $stall_timer->start ; pump $h until $out =~ /command 1 finished/ ; $in = 'command 2' ; $stall_timer->start ; pump $h until $out =~ /command 2 finished/ ; $in = 'very slow command 3' ; $stall_timer->start( 10 ) ; pump $h until $out =~ /command 3 finished/ ; $stall_timer->start( 5 ) ; $in = 'command 4' ; pump $h until $out =~ /command 4 finished/ ; $stall_timer->reset; # Prevent restarting or expirng finish $h ; See `' in this node for building non-fatal timers. See `timer', *Note IPC/Run/Timer: IPC/Run/Timer, for details. FILTER IMPLEMENTATION FUNCTIONS =============================== These functions are for use from within filters. input_avail Returns TRUE if input is available. If none is available, then &get_more_input is called and it's result returned. This is usually used in preference to &get_more_input so that the calling filter removes all data from the $in_ref before more data gets read in to $in_ref. get_more_input is usually used as part of a return expression: return input_avail && do { ## process the input just gotten 1 ; } ; This technique allows get_more_input to return the undef or 0 that a filter normally returns when there's no input to process. If a filter stores intermediate values, however, it will need to react to an undef: my $got = input_avail ; if ( ! defined $got ) { ## No more input ever, flush internal buffers to $out_ref } return $got unless $got ; ## Got some input, move as much as need be return 1 if $added_to_out_ref ; get_more_input This is used to fetch more input in to the input variable. It returns undef if there will never be any more input, 0 if there is none now, but there might be in the future, and TRUE if more input was gotten. get_more_input is usually used as part of a return expression, see `' in this node for more information. filter_tests my @tests = filter_tests( "foo", "in", "out", \&filter ) ; $_->() for ( @tests ) ; This creates a list of test subs that can be used to test most filters for basic functionality. The first parameter is the name of the filter to be tested, the second is sample input, the third is the test(s) to apply to the output(s), and the rest of the parameters are the filters to be linked and tested. If the filter chain is to be fed multiple inputs in sequence, the second parameter should be a reference to an array of thos inputs: my @tests = filter_tests( "foo", [qw(1 2 3)], "123", \&filter ) ; If the filter chain should produce a sequence of outputs, then the thrid parameter should be a reference to an array of those outputs: my @tests = filter_tests( "foo", "1\n\2\n", [ qr/^1$/, qr/^2$/ ], new_chunker ) ; See t/run.t and t/filter.t for an example of this in practice. TODO ==== These will be addressed as needed and as time allows. Stall timeout. Expose a list of child process objects. When I do this, each child process is likely to be blessed into IPC::Run::Proc. $kid->abort(), $kid->kill(), $kid->signal( $num_or_name ). Write tests for /(full_)?results?/ subs. Currently, pump() and run() only work on systems where select() works on the filehandles returned by pipe(). This does *not* include Win32. I'd like to rectify that, suggestions and patches welcome. Likewise start() only fully works on fork()/exec() machines (well, just fork() if you only ever pass perl subs as subprocesses). There's some scaffolding for calling Open3::spawn_with_handles(), but that's not that useful with limited select(). Support for `\@sub_cmd' as an argument to a command which gets replaced with /dev/fd or the name of a temporary file containing foo's output. This is like <(sub_cmd ...) found in bash and csh (IIRC). Allow multiple harnesses to be combined as independant sets of processes in to one 'meta-harness'. Allow a harness to be passed in place of an \@cmd. Ability to add external file descriptors w/ filter chains and endpoints. Ability to add timeouts and timing generators (i.e. repeating timeouts). High resolution timeouts. LIMITATIONS =========== No support for ';', '&&', '||', '{ ... }', etc: use perl's, since run() returns TRUE when the command exits with a 0 result code. Does not provide shell-like string interpolation. No support for cd, `setenv', or export: do these in an init() sub run( \cmd, ... init => sub { chdir $dir or die $! ; $ENV{FOO}='BAR' } ) ; Timeout calculation does not allow absolute times, or specification of days, months, etc. INSPIRATION =========== Well, select() and waitpid() badly needed wrapping, and open3() isn't open-minded enough for me. API inspired by a message Russ Allbery sent to perl5-porters, which included: I've thought for some time that it would be nice to have a module that could handle full Bourne shell pipe syntax internally, with fork and exec, without ever invoking a shell. Something that you could give things like: pipeopen (PIPE, [ qw/cat file/ ], '|', [ 'analyze', @args ], '>&3'); Message ylln51p2b6.fsf@windlord.stanford.edu, on 2000/02/04. AUTHOR ====== Barrie Slaymaker , with numerous suggestions by p5p.