MULTICS DESIGN DOCUMENT MDD-024 To: MDD Distribution From: Ed Sharpe Date: February 1, 1986 Subject: System Logging Abstract: Describes the system logging mechanisms. This document also provides a foundation for MDD-029 "Security Auditing". Revisions: REVISION DATE AUTHOR initial 86-02-01 Ed Sharpe _________________________________________________________________ Multics Design Documents are the official design descriptions of the Multics Trusted Computing Base. They are internal documents, which may be released outside of Multics System Development only with the approval of the Director. i MDD-024 System Logging CONTENTS Page Section 1 Introduction . . . . . . . . . . . . . . 1-1 Section 2 Goals . . . . . . . . . . . . . . . . . . 2-1 Section 3 Overview . . . . . . . . . . . . . . . . 3-1 Log Message . . . . . . . . . . . . . 3-1 Log Segment . . . . . . . . . . . . . 3-1 Lockless Operation . . . . . . . . 3-2 Log Family . . . . . . . . . . . . . . 3-2 Log History . . . . . . . . . . . . . 3-2 Programmatic Interface . . . . . . . . 3-2 Command Interfaces . . . . . . . . . . 3-3 Standard System Logs . . . . . . . . . 3-3 AS Log . . . . . . . . . . . . . . 3-4 Admin Log . . . . . . . . . . . . . 3-4 MC Logs . . . . . . . . . . . . . . 3-4 DM Log . . . . . . . . . . . . . . 3-4 Syserr Log . . . . . . . . . . . . 3-5 The Syserr Log Partition . . . . 3-5 The Wired Syserr Log . . . . . . 3-6 Section 4 Implementation . . . . . . . . . . . . . 4-1 Data Structures . . . . . . . . . . . 4-1 Internal Database Structures . . . 4-1 External Interface Structures . . . 4-5 Internal Interface Structures . . . 4-10 Modules . . . . . . . . . . . . . . . 4-12 bound_log_active_ . . . . . . . . . 4-13 bound_log_support_ . . . . . . . . 4-15 bound_log_tools_ . . . . . . . . . 4-19 System Support for AS Log . . . . . . 4-23 System Support for Admin Log . . . . . 4-23 System Support for MC Logs . . . . . . 4-24 System Support for DM Log . . . . . . 4-24 System Support for Syserr Log . . . . 4-25 Syserr Data Structures . . . . . . 4-26 Syserr Modules . . . . . . . . . . 4-33 Special Support in Standard Log Software . . . . . . . . . . . . . 4-39 Compatability . . . . . . . . . . . 4-40 ii System Logging MDD-024 CONTENTS (cont) Page Section 5 Security . . . . . . . . . . . . . . . . 5-1 Access Properties on New Log Segments 5-1 Access for Multiple Writers . . . . . 5-1 Access for Syserr Log . . . . . . . . 5-2 Section 6 Future Enhancements . . . . . . . . . . . 6-1 Selective Log Trimming . . . . . . . . 6-1 Multiple Log Perusal . . . . . . . . . 6-2 Replacement of Wired Syserr Log . . . 6-2 New syserr$binary Interface . . . . . 6-2 Remove syserr_log_util_ Interface . . 6-2 iii System Logging MDD-024 SECTION 1 INTRODUCTION This document describes the function and implementation of system logs in Multics. It covers the structure of data in the logs and all programs which manipulate them. This document also forms a basis for the presentation of MDD-029 "Security Auditing". The reader may be interested in the content of MTB-666 which describes the original proposal for the new logging mechanisms. However, note that information in that MTB may be obsolete and useful only for historical purposes. This document supersedes the MTB. 1-1 System Logging MDD-024 SECTION 2 GOALS Logs are intended to retain information about system events for later perusal. In particular, hardware and software failures are noted so that corrective action can be taken. Also, any changes in operating conditions (e.g. configuration) are noted to provide additional input to the failure resolution process. In a secure system environment, additional requirements are placed on the system logs. All security related events, such as use of system objects, must be noted. These entries in the log must include information about the object and the subject (user). For more information see MDD-029. The logging facilities must be robust: loss of information may make failure analysis difficult or security auditing useless. The logging facilities must be secure: users must not be able to corrupt nor view the information unless authorized. The logging facilities must be efficient: high overhead in storage or CPU resources will significantly degrade overall system performance. The logging facilities must provide perusal facilities: administrators should be able to select log entries by process (user), object, or event. 2-1 System Logging MDD-024 SECTION 3 OVERVIEW This section describes, in general, the logging mechanisms. First, the structure of the log data is presented. Interfaces for manipulating a log follows. Finally, the standard logs generated by the system are described. LLLOOOGGG MMMEEESSSSSSAAAGGGEEE Logs are composed of a number of entries called messages. Each message contains three general pieces of information: Identification (sequence number, timestamp, userid) Text (the message itself) Binary data (optional) The text and binary portions are supplied by the process wishing to log an event. The identification is supplied by the log support software (although it is not protected). LLLOOOGGG SSSEEEGGGMMMEEENNNTTT Messages are accumulated in "log segments". A log segment may be created anywhere in the Multics storage system hierarchy. The internal structure exhibits a header containing information about the number of messages, limits, etc. and an array of log messages. Log segments are protected by the standard Multics security mechanisms (ACLs, rings, and AIM). A process must posess "write" effective access to add messages to a log segment. It must posess "read" effective access to peruse the contents of a log segment. It must posess "append" effective access to a directory in which a new log segment is to be created. Section 5 "Security" describes security concerns in more detail. 3-1 MDD-024 System Logging LLLoooccckkkllleeessssss OOOpppeeerrraaatttiiiooonnn One feature of a log segment is that multiple processes may simultaneously add messages. This is made possible by a mechanism that uses the STACQ to reserve space in the segment. Multiple processes may fill information in their respective reserved areas. This avoids the need for log segment locks and their associated overhead. LLLOOOGGG FFFAAAMMMIIILLLYYY Logs typically grow beyond what can be contained in a single segment. When a log segment fills, it is marked as such, renamed, and a new segment is created to take its place. The resulting collection of segments is called a "log family". The most recently created segment in a family, the "live" log segment, bears the name of the log (e.g. "admin_log"). Segments which have been filled are renamed to include the date/time of the last message they contain (e.g. "admin_log.19851216.185704"). All log perusal routines deal with log families so that the user is presented with the illusion of a single, large, sequential log. LLLOOOGGG HHHIIISSSTTTOOORRRYYY The primary location of a log family may not always be the optimum place for long-term storage. For instance, the live answering service log must be on the root logical volume. However, saving the messages over a month would place unacceptable storage requirement on the root volume. Thus, the log mechanisms allow a log family to be split into multiple directories (to use space on different volumes). PPPRRROOOGGGRRRAAAMMMMMMAAATTTIIICCC IIINNNTTTEEERRRFFFAAACCCEEE The following modules provide the basic subroutine support for maintaining logs. They are described in more detail in the next section "Implementation". log_write_ Is the primary interface for writing logs. The caller supplies text and binary data. This routine takes responsibility for entering the log message and maintaining the log family (i.e. renaming full segments, creating new segments, etc). log_output_ Is an I/O module interface to log_write_. 3-2 System Logging MDD-024 log_read_ Is the primary module for perusal of a log family. CCCOOOMMMMMMAAANNNDDD IIINNNTTTEEERRRFFFAAACCCEEESSS This section describes the command interfaces associated with system logs. They are all described in the published Multics documentation and/or in info segs. print_sys_log Is the primary tool for log perusal. Through use of control arguments the user can select messages based on time, sequence, data type, or message contents. monitor_sys_log Is similar to print_sys_log except that messages are processed in a polled manner. That is, every N seconds, the prior N second interval of the specified log is scanned. Messages are displayed that meet the selection criteria. summarize_sys_log Provides a summary of messages within a given interval that appear in a log. This is useful for counting occurrances of similar messages. It is typically used by system administrators to provide system activity reports on a daily basis. move_log_segments Moves the timestamped members of a log family to another directory. It appropriately updates the log segment headers to provide a trail for log perusal tools. set_log_history_dir Is a tool used to update log segment headers when the history trail must be repaired or changed. display_log_segment Displays the contents of a log segment. In particular, it displays the log segment header information (which is not available through the log perusal tools). This is meant as a debugging/maintenance tool. SSSTTTAAANNNDDDAAARRRDDD SSSYYYSSSTTTEEEMMM LLLOOOGGGSSS The following paragraphs describe the standard logs generated by the system. 3-3 MDD-024 System Logging AAASSS LLLoooggg The answering service log family appears in the directory >sc1>as_logs. Its primary name is "log". The log perusal tools recognize the log selection control argument "-answering_service" ("-as") to refer to this log. The AS log is writable by the Initializer process only. It contains entries for events such as user authentication, process creations, accounting updates, communications channel attachment, operator commands, etc. Some of these messages are considered part of the security audit trail. AAAdddmmmiiinnn LLLoooggg This log family appears in the directory >sc1>as_logs. Its primary name is "admin_log". The log perusal tools recognize the log selection control argument "-admin" in reference to this log. The admin log is writable by the Initializer process only. It contains a script of all operator console interaction including that while in "admin" mode. It also records the use and response from commands entered via the "send_admin_command" AS request. All messages in this log are considered part of the security audit trail. MMMCCC LLLooogggsss There are an arbitrary number of message coordinator log families, each corresponding to a message coordinator log stream. They appear in the directory >sc1>as_logs. The log perusal tools recognize the log selection control argument "-mc_log NAME" ("-mcl NAME") in reference to these logs (where "NAME" is replaced by the entryname of the log). The various message coordinator logs are writable by the Initializer process only. They contain a script of all text output on the associated I/O stream. Creation and maintenance of these logs depends upon the stream routing set up at the site. Logs associated with trusted daemon processes (e.g# printer) are considered part of the security audit trail. DDDMMM LLLoooggg A data management log family appears in the directory >site>Data_Management>ACCESS_CLASS>logs (where ACCESS_CLASS is the AIM level/ categories of the processes which add messages to this particular log). Its primary name is "dm_system_log". The log perusal tools recognize the log selection control argument 3-4 System Logging MDD-024 "-dm_system" ("-dms") in reference to this log. There is a DM log for each DM system (generally for each access class). The DM system log is writable by all processes in ring 2. Read access is controlled by the access on a gate used to read this log. The DM log contains messages about daemon actions such as DM initialization, shutdown, and recovery. It also contains messages about user actions (inner ring) such as failures to open a database and abandoned transactions. SSSyyyssseeerrrrrr LLLoooggg This log appears in the directory >sl1. Its primary name is "syserr_log". The log perusal tools recognize the log selection control argument "-syserr" in reference to this log. The syserr log is writable by all processes in ring 0. It contains messages about hardware status, hardware and software failures, resource requests, and security audit. The syserr log is different from all other logs. The handling of the full log segment condition is handled in a special manner. In particular, the live log segment corresponds to the "log" partition which appears on the root physical volume (RPV). It cannot simply be renamed and a new log created. THE SYSERR LOG PARTITION Three segments are mapped onto the log partition (via "fabricate" keywords in the hardcore header). "syserr_log_data" is a single page which contains control information. The rest of the pages of the partition are divided integrally between two log segments. One of these will be the "live" syserr_log segment which appears as "syserr_log" in the directory ">sl1". The other will appear as "syserr_log.empty". When the log segment "syserr_log" becomes full (within a threshold), it is renamed to include a timestamp (e.g. "syserr_log.19851216.180734"). The "syserr_log.empty" segment is renamed to "syserr_log" and thus becomes the "live" syserr log segment.(1) These two segments form the immediate log family. The next portion of the family appears in the directory ">sc1>syserr_log". The log segments here are created by the Initializer. At the time the log segments in >sl1 are renamed (as described above) a _________________________________________________________________ (1) Note that these segments appear in the hardcore header as syserr_log_laurel and syserr_log_hardy. While their names in the hierarchy will change, their names in the SLT will not. 3-5 MDD-024 System Logging wakeup is sent to the Initializer. Its task is to migrate the individual log messages from the timestamped log into the segments in >sc1>syserr_log which are created as required. When the timestamped segment in >sl1 is empty, the Initializer will rename it to "syserr.empty" thereby restarting the cycle. As this migration of messages is being performed by the Initializer, other processes may append messages to the segment named >sl1>syserr_log. Note that performance of this log is dependant on the speed with which this migration can be accomplished. (So far, this has not been a problem). THE WIRED SYSERR LOG Note that the segments mapped onto the syserr log partition are paged. It is not always possible to handle a page fault at the time some message must be placed in the log (some messages are generated while wired and masked). Messages generated while certain locks are set or while the processor is masked are temporarily deposited in the wired syserr log. This is a small wired memory area which does not appear in the hierarchy (it appears in the SLT as "syserr_data"). A dedicated hardcore process "Syserr_Logger.SysDaemon" is sent a wakeup whenever a message is deposited in the wired log. The Syserr_Logger will migrate these messages to the paged ">sl1>syserr_log" segment. Any process which is not wired, and has no locks set, can migrate these messages. This will occur when the process wishes to place a message in the syserr log, and finds the wired syserr log non-empty. Ensuring that the wired log is empty before appending to the paged log will maintain the ordering of message sequence. 3-6 System Logging MDD-024 SECTION 4 IMPLEMENTATION This major section covers the actual implementation of the log databases and programs. Descriptions are limited to high level descriptions of each module. For details, one should refer to the source code. Of course, any discrepancy is resolved by the source code. DDDAAATTTAAA SSSTTTRRRUUUCCCTTTUUURRREEESSS This section describes the data structures associated with the logging software. IIInnnttteeerrrnnnaaalll DDDaaatttaaabbbaaassseee SSStttrrruuuccctttuuurrreeesss The following include files define the format of logs. These are considered internal and subject to change without notice. log_message.incl.pl1 This defines the format of a log message. Basically there are three sections: a header; the message text; and optional binary data. The structure as of this writing is: /* last modified 84-04-25 W. Olin Sibert */ dcl 1 log_message_header aligned based, 2 sentinel bit (36) aligned, 2 sequence fixed bin (35), 2 severity fixed bin (8) unaligned, 2 data_class_lth fixed bin (9) unaligned unsigned, 2 time fixed bin (53) unaligned, 2 text_lth fixed bin (17) unaligned, 2 data_lth fixed bin (17) unaligned, 2 process_id bit (36) aligned; dcl 1 log_message aligned based (log_message_ptr), 2 header aligned like log_message_header, 2 text char (log_message_text_lth 4-1 MDD-024 System Logging refer (log_message.text_lth)) unaligned, 2 data_class char (log_message_data_class_lth refer (log_message.data_class_lth)) unaligned, 2 data dim (log_message_data_lth refer (log_message.data_lth)) bit (36) aligned; where: sentinel Is a bit pattern used to indicate a new message, a completed message, and as a salvaging aid. sequence Is the sequence number of the message. Sequence numbers are increased by one as each message is appended. They start at 100000. severity Indicates the relative importance of the event which causes the message to be appended. It may be used to select messages of particular importance by the log perusal tools. data_class_lth The number of characters in the variable-length data_class field. time The real-time clock value at the time of message creation. These are generally increasing in a log. However, because multiple processes may fill in the messages, these values may appear disordered for small samples of messages. text_lth The number of characters in the variable-length text field. data_lth The number of words in the variable-length data field. process_id The process-ID of the process which appended the log message. text The textual part of the message as given to log_write_. There may be conventions used for the format (e.g. syserr_log messages are typically "MODULE_NAME: 4-2 System Logging MDD-024 MESSAGE"). But these are not enforced by the log support software. data_class A 0-16 character identifier. This defines the format of the binary data field. The log perusal tools use this field for message selection and data expansion. (The expansion is carried out by routines named "expand_XXXXXX_msg_" where "XXXXXXX" is the value of the data_class field). data An arbitrarily dimensioned array of binary data. Its interpretation is left to the associated data expansion module. The log_write_ caller need only supply its class identifier and its length. log_segment.incl.pl1 This defines the format of a log segment. Basically it is a header followed by an array of messages. Its format as of this writing is: /* last modified 84-05-03 W. Olin Sibert */ dcl 1 log_segment aligned based (log_segment_ptr), 2 header aligned like log_segment_header, 2 data dim (log_segment_max_size refer (log_segment.max_size)) bit (36) aligned; dcl 1 log_segment_header aligned based, 2 version char (8) unaligned, 2 time_created fixed bin (71), 2 previous_log_dir char (168) unaligned, 2 limits, 3 first_sequence fixed bin (35), 3 last_sequence fixed bin (35), 3 first_time fixed bin (71), 3 last_time fixed bin (71), 2 alloc_info, 3 word_1 fixed bin (18), 3 word_2 bit (36) aligned, 2 max_size fixed bin (18), 2 listeners_registered bit (1) aligned, 2 listener_bootload_time fixed bin (71), 2 listener (25), 3 process_id bit (36) aligned, 3 event_channel fixed bin (71) unaligned, 2 last_wakeup_time fixed bin (71), 2 wakeup_delta fixed bin (71), 2 pad (6) fixed bin (71); 4-3 MDD-024 System Logging where: data Is the data area reserved for appending messages. version Is a version indicator. time_created Is the real-clock time when the log segment header was initialized. previous_log_dir Is the pathname of the directory to which older members of the log family have been moved. This is updated by move_log_segments and set_log_history_dir. Only the oldest log in a particular directory need have this set (although all members should have a value here). A null value indicates there is no older member online. first_sequence Is the sequence number of the first message appended to this log segment. last_sequence Is the sequence number of the last message appended to this log segment. first_time Is the real-time clock value of the first message in this log segment. last_time Is the real-time clock value of the last message completed in this log segment. This is the value used to generate a timestamp name for when the log segment becomes full. alloc_info Is the structure of the double word used to allocate messages in the data area. It is used with the STACQ instruction so that no locking is required. Its format is known only by the modules "log_segment_" and "log_salvage_". At the time of this writing its structure is: dcl 1 sequence_info aligned, 2 pad bit (17) unaligned, 2 number fixed bin (35) unaligned, 2 in_service bit (1) unaligned, 2 words_used fixed bin (18) unsigned unaligned; 4-4 System Logging MDD-024 where: "number" is the message sequence number; "in_service" indicates whether the segment is the live log segment or a history member; and "words_used" is the number of words already allocated out of the data area. See the description for "log_segment_" for details on how this double word is used. max_size Is the total number of words in the data area. listeners_registered Indicates that the listeners list is non-empty. listener_bootload_time Is the value of the system bootload time at the last listener registration. This is used to determine when the listener array is to be cleared. listener Is an array of listening processes. Each element of the array contains the process-ID and event channel for sending wakeups. last_wakeup_time Unknown purpose. This value is only copied to new logs from their immediate ancestors. (See conjecture of purpose below for "wakeup_delta"). wakeup_delta Unknown purpose. This value is only copied to new logs from their immediate ancestors. (I suspect now that this value, and the "last_wakeup_time", were meant to limit the number of wakeups sent for particularly busy logs - this would group the messages covered by a wakeup. Of course, this implies that listeners may miss messages when a log suddenly quiesces - unless a timer_manager_ wakeup would go off to kick us into action - but there's no single process to set this timer! I guess we simply have to send a wakeup for each message as it is appended.) EEExxxttteeerrrnnnaaalll IIInnnttteeerrrfffaaaccceee SSStttrrruuuccctttuuurrreeesss These structures define the information passed from user commands and I/O modules to the log support subroutines. They do not appear in published documentation. log_limit_info.incl.pl1 This structure is passed to log_limit_scan_. It contains the -from/-to control arguments on the print_sys_log or 4-5 MDD-024 System Logging summarize_sys_log command lines. The format as of this writing is: /* last modified 84-12-16 BIM */ dcl 1 log_limit_info aligned based (log_limit_info_ptr), 2 version char (8) aligned, 2 to_opt char (50) varying, 2 from_opt char (50) varying, 2 for_opt char (50) varying, 2 last_opt char (50) varying, 2 first_msg pointer, 2 last_msg pointer, 2 msg_count fixed bin (35); where: version Is a structure version identifier. to_opt Is the argument to the "-to" control arg. It may be a date-time or a sequence number. from_opt Is the argument to the "-from" control arg. It may be a date-time or a sequence number. for_opt Is the argument to the "-for" control arg. It is an integral number of messages. last_opt Is the argument to the "-last" control arg. It is an integral number of messages. first_msg Is a pointer to the first message of the interval defined by the command line arguments. This is filled in by log_limit_scan_. last_msg Is a pointer to the last message of the interval defined by the command line arguments. This is filled in by log_limit_scan_. msg_count Is the number of messages in the interval. This is filled in by log_limit_scan_. log_message_format.incl.pl1 This structure is filled in by the log perusal tools 4-6 System Logging MDD-024 print_sys_log, monitor_sys_log, and summarize_sys_log. The information is used by format_log_message_, expand_log_message_, and some of the binary data interpretation modules. Its format as of this writing is: /* last modified 84-07-03 W. Olin Sibert */ dcl 1 log_message_format aligned based (log_message_format_ptr), 2 caller char (32) unaligned, 2 line_lth fixed bin, 2 indentation fixed bin, 2 equal_sw bit (1) aligned, 2 error_sw bit (1) aligned, 2 prefix char (40) varying, 2 number_format char (20) varying, 2 time_format char (40) varying, 2 date_format char (40) varying, 2 continuation_indent fixed bin, 2 pad (59) bit (36) aligned, /* ONLY format_log_message_$adjust ever changes the values below */ 2 internal_info aligned, 3 area_ptr pointer, 3 date_ioa_string char (100) varying, 3 message_ioa_string char (100) varying, 3 real_continuation_indent fixed bin, 3 prefix_lth fixed bin, 3 date_modulus fixed bin (36), 2 end fixed bin; where: caller Is the name of the calling module (primarily for use in error messages). line_lth Is the maximum length for output lines. Longer lines will be folded. indentation Is the number of spaces to leave blank at the beginning of each output line. equal_sw Indicates whether to suppress display of messages with identical text. error_sw Indicates whether to print error messages via com_err_. 4-7 MDD-024 System Logging prefix Is a prefix for printing messages. This is especially useful for monitor_sys_log where messages from multiple logs may be interleaved in the output stream. number_format Is a date_time_$format format string for message times. date_format Is a date_time_$format format string for message dates. continuation_indent Is the number of spaces to indent continuation (folded) lines of the message. If zero, the continuation lines are lined up under the start of the message text. area_ptr Is a pointer to the area in which this structure is allocated. date_ioa_string Format for date breaks. That is, when the messages span dates and dates are not normally displayed. message_ioa_string Format for first lines of messages. This contains controls for the sequence number, severity, date, and time of a message. prefix_lth Is the sum of the lengths of: the indentation value; the prefix; the time; the sequence; and the severity. date_modulus Is a modulus for checking whether dates are equal. log_output_binary_info.incl.pl1 Is the structure used with the log_output_ "set_binary_info" and "get_binary_info" control orders. It is used to supply binary data for log messages appended using the I/O module. It format at the time of this writing is: /* last modified 1984-10-22 BIM */ dcl 1 log_output_binary_info aligned based (log_output_binary_info_ptr), 2 version char (8) aligned, 2 data_class char (16) varying, 2 data_ptr pointer, 2 data_length fixed bin (18); where: 4-8 System Logging MDD-024 version Is a structure version identifier. data_class Is the data class to be associated with the binary data in the log message. data_ptr Is a pointer to the data area in which the binary data resides. data_length Is the length, in words, of the binary data. log_read_open_info.incl.pl1 This structure is used for the log_read_$open_long entrypoint to provide additional information for the opening. It is filled in by print_sys_log, monitor_sys_log, and summarize_sys_log. Its format at the time of this writing is: /* last modified 1984-12-15 BIM */ dcl 1 log_read_open_info aligned based (log_read_open_info_ptr), 2 version char (8) aligned, 2 reader_procedure char (32) varying, 2 allocation_area_ptr pointer, 2 allocate_copies bit (1) aligned; where: version Is a structure version identifier. reader_procedure Is the name of a module which will provide the same functions as log_read_. This module will be transferred to upon each log_read_ invocation. This is useful for reading inner ring logs in the user ring. In this case, "reader_procedure" will gate into the inner ring. The reading of the data management log is accomplished in this manner. allocation_area_ptr This is passed on to the reader_procedure. It is a pointer to the area in which the reader procedure should allocate storage. allocate_copies Indicates that copies of messages are to be allocated in the area accessible by the caller. (Normally, 4-9 MDD-024 System Logging log_read_ operations pass a pointer to the actual message in the log segment. This is not possible if the log is in a lower ring than the caller.) log_salvage_arg.incl.pl1 This structure is used to supply information to log_salvage_. Its format at the time of this writing is: /* last modified 85-04-09 EJ Sharpe */ dcl 1 log_salvage_arg aligned based (log_salvage_arg_ptr), 2 version char (8), 2 reporter_proc entry (char (*)) variable, 2 from_time fixed bin (71), 2 flags, 3 migrating bit (1) unal, 3 pad bit (35) unal; where: version Is a structure version identifier. reporter_proc Is a procedure supplied by the caller which is used to report errors. It is called upon encountering an error and is passed a character string containing an explanation. This string may be printed, logged, etc. from_time Is a real time clock value which describes the beginning point of the salvage. migrating Indicates that this log family receives only message migrated from some other live log. Thus, only timestamped family members should exist in the directory (as for the syserr log). IIInnnttteeerrrnnnaaalll IIInnnttteeerrrfffaaaccceee SSStttrrruuuccctttuuurrreeesss These structures define the format of data used within the log support software. They are not documented in any published documantation and may change without notice. The following paragraphs provide only an overview of these structures, not actual PL/1 formats. log_read_data.incl.pl1 This structure maintains information about the state of a log opened for reading. It is used by log_read_, 4-10 System Logging MDD-024 log_list_history_, log_salvage_, and syserr_log_man_ (for its own internal salvager). Callers of log_read_$open are returned a pointer to the structure. This pointer is to be supplied to any other log_read_ entrypoint referenced. Use of static data in log_read_ is avoided. The header of the structure contains information about the opening: - number of segments in family - whether to copy messages read - the current message pointer - routines to perform operations - log limits time of very first message time of very last message sequence of very first message sequence of very last message The remainder of the structure is an array with an entry for each segment in the log family. These entries contain: - segment path - pointer to segment - time of last message in segment - UID of parent directory log_write_data.incl.pl1 This structure contains information about a log used for write operations. It is used by log_write_, log_create_, and log_salvage_. It is filled in by log_write_$open, and a pointer is passed back to the caller. The caller is to supply this pointer to the other log_write_ entrypoints. As for log_read_, use of static storage in log_write_ is avoided. The structure contains: - the name of the log - a pointer to the live log segment - the pathname of the log - the ACL on the log - the ring brackets of the log - the maximum length of the log segment - the access class of the log - a flag indicating whether the log is multi-class Most of the items are used when creating a new log segment upon exhaustion of space in the live log segment. log_output_attach_data_.incl.pl1 This structure contains data used by the log_output_ I/O 4-11 MDD-024 System Logging module and its companion routine log_ouptut_io_. It contains: - attach description - open description - pointer to log_write_data - pointer to text buffer - pointer to binary data buffer monitor_sys_log_info_.incl.pl1 This contains structures used by the monitor_sys_log command and the monitor_sys_log_wakeup_ timer handler. The monitor_sys_log_data_ structure is allocated in per-process external static. It contains the wakeup interval, the wakeup event channel, the number of logs being monitored, and a pointer to the monitor_sys_log_array structure. The monitor_sys_log_array structure is allocated in the per-process area. It contains valid entries for each log being monitored. For each log, the information maintained is: - the log pathname - the log_read_data_ptr from log_read_$open - flags indicating whether the log is the admin, AS, syserr, or DM log - flags which represent the control arguments supplied on the monitor_sys_log command line - pointers to data for message selection - pointers to data for message expansion - pointers to data for message formatting - an iocb pointer for printing the messages - pointers to the last two messages read - optional event channel for wakeups - command line to call for messages selected log_data_.cds The external structure "log_data_" is initialized by this CDS. The values are constant and may be referenced by any program. It contains: - default log parameters (size, max text/data length) - log message sentinel values - syserr log info (see "Extra Syserr Modules" below) MMMOOODDDUUULLLEEESSS This sections describes the function of all modules which provide log support services. 4-12 System Logging MDD-024 bbbooouuunnnddd_llloooggg_aaaccctttiiivvveee_ This bound unit is found in hardcore collection 2 with ring brackets of 0,5,5. It provides the lowest level primitives for log support. log_data_.cds This program initializes the log_data_ external structure. log_initialize_ This module initializes log segments. The log segment header is either setup with default values, or setup with the values of the previous segment in the log family. log_name_ This module supports log segment names, especially the data/time suffix for non-live family members. The log_name_$name entrypoint, given a clock reading and a log name, returns the appropriately suffixed log name. The log_name_$starname entrypoint, given a log name, returns a starname which matches all suffixed log segments in the family. The log_name_$time entrypoint, given a suffixed log name, returns the wall clock time represented by that suffix. Incidentally, all log name suffixes represent a GMT value. log_position_ This module provides primitives for motion within a log segment. There are entrypoints for moving backward and forward by a single message from a supplied reference. Other entrypoints provide absolute positioning by message time or sequence number. The algorithms will find the closest message to a given time/sequence. log_segment_ This module implements the lowest level primitives in the logging software. It is responsible for the creation of message within a single log segment. If the segment becomes full or damaged, an error code is returned to the caller to take appropriate action (e.g. log_write_ will create a new segment). log_segment_ may be called in any ring, in any environment. Messages are allocated in a log through use of an allocation doubleword and the STACQ test-and-set instruction. The allocation doubleword was described in the subsection "log_segment.incl.pl1" above (the "alloc_info" structure member). The basic approach is to: - read the current allocation word into a temporary - generate a pointer to the next free word - increase the sequence number by one 4-13 MDD-024 System Logging - increase the words used by the size of the message - rewrite the temporary via STACQ - if STACQ fails then repeat previous steps - otherwise return pointer to allocated message area The actual code is slightly more complex and should be consulted for details on how this mechanism works. This is the only module which may manipulate the message allocation information in the log segment header. This module must be called to create messages, place segment in service, removed segment from service etc. The following paragraphs describe each entrypoint in detail. log_segment_$create_message This entrypoint is called to allocate a message in a log segment. The caller supplies a pointer to the log segment and the lengths of the text and binary data of the message. A message sequence number is assigned and an appropriately sized area is allocated (via the STACQ hack) at the end of existing data in the log. A pointer to this area is returned to the caller who may then fill in the text and binary data. The message sentinel has a value indicating that this is a new message. log_segment_$create_message_number This entrypoint is the same as log_message_$create_message except that the caller may choose the sequence number. Use of this entrypoint moves responsibility for message sequencing to the caller. This is used by the syserr log software, since sequence numbers are determined by syserr_real (in the wired log). log_segment_$finish_message The caller of log_segment_$create_message(_number) must call this entrypoint after completing the text and binary data of the message. This entrypoint will change the message sentinel to reflect that it is complete. It will also update the log segment header information to reflect a new value for last message time/sequence. log_segment_$initialize_sequence This entrypoint is used to initialize the allocation info data in the log header. The supplied sequence number is placed in the double word, all other fields are zeroed (i.e. the words_used and the service_bit). log_segment_$last_message_info This entrypoint returns the sequence number and the offset of the last message in the log segment. 4-14 System Logging MDD-024 log_segment_$place_in_service log_segment_$remove_from_service log_segment_$get_service_bit These entrypoints manipulate the service bit. Typically, the service bit is turned on in a new log after all other fields are initialized. It is turned offf when the log becomes full and a new log segment has been created. log_wakeup_ This routine manages the sending of wakeups to registered "listeners" of a log segment. log_read_ calls the $register and $deregister entrypoints to register and deregister listeners (up to 25). log_write_ calls the $send_wakeups entrypoint whenever a new message is added to the log. This is not normally used for monitor_sys_log type activity. Rather, it is meant for those cases where processes must be notified immediately of the addition of messages to the log. bbbooouuunnnddd_llloooggg_sssuuuppppppooorrrttt_ This bound unit appears in hardcore collection 3 with ring brackets of 1,5,5. It implements the next layer of log support primitives above those in bound_log_active_. log_create_ This module is used to create logs. It has entrypoints for creating a new log ($new_segment), and for creating a new segment in an existing log family ($duplicate_segment). There is an additional entrypoint to extract information about a log segment such as the ACL, ring brackets, maximum size, etc. ($get_info). log_initiate_ This is the user-ring log segment initiation routine. It checks the log header for validity. It may wait and retry the header check. This is because some other process may be in the middle of initializing the log. log_list_history_ This routine is used to get a list of all log segments in a family. This list is allocated as an array in the log_read_data structure described earlier. The segments will appear in sorted order according to the value of their timestamped names. There are three entrypoints. log_list_history_$all will list the entire family as it appears in all online history directories. log_list_history_$single_dir will list only those family members in the given directory (used by log_salvage_). log_list_history_$single_dir_oldest_and_newest will return the names of the oldest and newest family members in the 4-15 MDD-024 System Logging given directory (used by log_write_ to find the newest segment in a history directory for syserr message migration). log_move_ This routine moves a single log message from one log segment to another. This is currently used for migrating messages from the ring-0 paged syserr log to the syserr log history directory. This routine should not be confused with move_log_segments whose purpose is to move log history segments between history directories. log_read_ This module provides all the interfaces necessary for log perusal in the user ring. It is used by print_sys_log, monitor_sys_log, and summarize_sys_log for this purpose. The following paragraphs describe each entrypoint in detail. log_read_$open This entrypoint must be called before any other log_read_ entrypoint. It sets up the log_read_data structure upon which all other operations depend. Among other things, a complete list of all segments in the log family is set up. log_read_$open_long This entrypoint is similar to log_read_$open except that the caller supplies a structure of additional opening information. In particular, the caller supplies the names of routines to transfer control for all other log_read_ entrypoints. This is used for logs which require special handling such as the DM system log which resides in ring 2. log_read_$close This entrypoint terminates all log segments initiated during the log perusal. It then discards the log_read_data structure. log_read_$next_message Changes a supplied message pointer to locate the immediately succeeding message in the log. This may be in the next log segment, or null if at the end of the log family. log_read_$prev_message Changes a supplied message pointer to locate the immediately preceding message in the log. This may be in the previous log segment, or null if at the beginning of the log family. log_read_$position_sequence Returns a pointer to the message which has the given sequence number. If no message has that sequence (due to log damage, message incomplete, etc.) a pointer to the 4-16 System Logging MDD-024 immediately succeeding or preceding message is returned (depending on the value of a switch supplied by the caller). log_read_$position_time Returns a pointer to the message which was created at the given time. If no message has that particular time value a pointer to the immediately succeeding or preceding message is returned (depending on the value of a switch supplied by the caller). log_read_$update Updates the log_read_data information according to the new state of the log family. This is used between operations where it is desired to peruse messages appended since the call to log_read_$open (e.g. as is the case with monitor_sys_log). log_read_$hold_message log_read_$free_message These entries are used when dealing with logs which require special handling (e.g. the ring-2 DM system log). All entries allocate user-ring copies of messages rather than returning a pointer into the log segment itself (which would be useless). If the caller needs to "remember" a message, retaining a copy of the pointer is not sufficient, since the next message positioning operation will deallocate that copy. These entries provide a way to defer the deallocation (hold) until a more appropriate time (free). log_read_$get_log_uid This entrypoint returns the UID of the live (newest) log segment in the family. log_read_$register log_read_$deregister These entrypoints simply transfer to log_wakeup_$register and log_wakeup_$deregister respectively. User ring programs should reference the log_read_ entrypoints rather than log_wakeup_ directly. The function of log_wakeup_ is described earlier in this document. log_salvage_ This module checks the consistency of a log family. Some forms of inconsistency will be corrected. Only the last few log segments are inspected, previous segments are probably in good order. In most cases, damage necessitates the reinitialization of the live log segment. Efforts are made to preserve the sequencing of messages in the log. Programs which call log_write_$open or attach an I/O switch with log_output_ are responsible for calling log_salvage_. The suggested usage is: 4-17 MDD-024 System Logging call log_write_$open (dir, name, "0"b, datap, code); if code ^= 0 then do; unspec (salv_arg) = "0"b; salv_arg.version = LOG_SALVAGE_ARG_VERSION_1; salv_arg.reporter_proc = print_errors; call log_salvage_ ( dir, name, addr (salv_arg), salv_code); if salv_code = 0 then call log_write_$open ( dir, name, "0"b, datap, code); if code ^= 0 /* still failing? */ then call error (code); /* nothing more to do */ end; Note that the parameter which controls automatic creation of the live log segment is off ("0"b). This is because log_write_ will not ensure that the message sequence numbers follow the log's history. The same is true of the log_output_ I/O module (the "-no_create" argument should appear in the attach description). Creation of the log should be left to log_salvage_. log_write_ This module provides the interfaces used to maintain a log family. Much more is provided than in the log_segment_ primitives (e.g. it will create log segments as needed). The following paragraphs describe the individual entrypoints. log_write_$open This entrypoint must be called before any other log_write_ operation can be used. Its basic task is to fill in the log_write_data structure. A pointer to this structure is passed back to the caller. This pointer must be supplied to any of the other log_write_ entrypoints called. If the log does not exist, it may be created depending upon the supplied arguments. log_write_$open_for_migrate This entrypoint functions similar to log_write_$open. The difference is that it opens a log history segment rather than the live log segment. This is used by the syserr log support mechanisms. The log_write_ interfaces are used to migrate messages from the ring-0 paged syserr log into the syserr log history. log_write_$close This entrypoint terminates the log segment and discards the log_write_data structure. No further operations can take place on a closed log. 4-18 System Logging MDD-024 log_write_$message This entrypoints adds the given textual message to the log. log_write_$data This entrypoints adds a message to the log which contains both textual and binary information. log_write_$general This entrypoint reserves space in the log for a message. The caller supplies the text and optional binary data lengths. A pointer to the new log message area is returned. After filling the body of the message, the caller must call log_segment_$finish_message. bbbooouuunnnddd_llloooggg_tttoooooolllsss_ This bound unit appears in the >tools online library with standard ring brackets of 1,5,5. It implements the remainder of the user ring log support mechanisms. convert_syserr_log This command converts a pre-MR11 syserr log into the new format. It is meant to be used by each site once while installing MR11 on an existing system. The include files syserr_log.incl.pl1 and syserr_message.incl.pl1 were also retained for conversion purposes. convert_old_log (bound_admin_old_) This command (resides in a different bound unit as noted) converts a pre-MR11 log into the new format. It is meant to be used by each site to convert their logs while installing MR11 on an existing system. The include file syslog.incl.pl1 was also retained for conversion purposes. Note that the old log software is retained in >obsolete>bound_old_logging_ in MR11. Although this software was undocumented, it is known to be used by various customer applications (which must be converted). display_log_segment This command displays the internals of a log segment. expand_log_message_ This module is used to expand various parts of a message's binary data into printable form. The binary data may be expanded into octal word dump format or into a readable text format. Expansion into readable text is accomplished by calling the appropriate expansion module as described in the next paragraph. expand_log_message_ also has entrypoints for display of the process_id and data_class associated with a message. The caller must supply a pointer the the 4-19 MDD-024 System Logging log_message_format structure which contains control information (such as line length, etc). expand_XXXXXXX_msg_ These routines are called upon by expand_log_message_ to convert the binary data of a log message into a readable format. There is one module for each defined data class (e.g. expand_access_audit_msg_, expand_segdamage_msg_, etc). A pointer and length of the binary data is supplied. The resulting text is appended to a varying character string also supplied by the caller. All expanded formats are published in the System Maintenance Procedures AM81. expand_YYYYYYY_audit_info_ These routines are called upon by expand_access_audit_msg_ to convert portions of the binary data into a readable format. Messages with the "access_audit" class have binary data that contains a standard header (which is converted by expand_access_audit_msg_) followed by one or more pieces of operation dependant "extended data". It is on the extended data that these routines perform their conversion. The proper routine is indicated by a code contained in the first word of the extended data. It is supplied with the length and a pointer to the data. It converts the first N words of data (where N = the size of the structure), reduces the data length parameter by N, and returns. expand_access_audit_msg_ will call additional routines for any unconverted extended data. Examples of converted access audit messages are shown in System Administration Procedures AK50. format_log_message_ This routine is used by the log perusal tools to arrange the text and expanded binary into the final printable format. It handles addition of message prefixes, folding of the message text over multiple lines, and indenting the continuation lines. It is called after the (optional) work of expand_log_message_ is complete. log_expand_select_ This routine simply manages a list of binary data classes. It is messages with data of these classes which are to be expanded. The log perusal commands print_sys_log and monitor_sys_log support message selection based on data class, and thus utilize this routine. Each data class in the list may have a mode string which controls the format of the expansion. log_format_time_ This routine returns a printable date-time format. It is used by various modules for printing a time interval. It is not used to print the per-message date-time. 4-20 System Logging MDD-024 log_limit_scan_ This routine processes the "-from", "-to", "-for", and "-last" control arguments for the log perusal commands print_sys_log and summarize_sys_log. It also returns pointers to the first and last messages in the interval defined by these controls. The log must be open prior to calling log_limit_scan_. log_match_ This routine manages the selection criteria and the actual selection of log messages for the log perusal tools print_sys_log, monitor_sys_log, and summarize_sys_log. As message selection control arguments (e.g. "-match", "-exclude", "-match_data", etc) are processed, entrypoints in this routine are called to add/remove entries from the internal "log_match_data" structure. Later, as the command sequences through log messages, this routine is called for each log message to determine whether printing is required. log_output_ This module is the primary piece of the log I/O module. It provides the attach, detach, open, and close functions. The attach description must contain the pathname of the log. It may optionally contain "-create" or "-no_create" to control creation of the log if it is not already present ("-create" is the default). At the time of this writing, this interface is not documented. log_output_io_ This module is part of the log I/O module. It provides the put_chars, modes, and control functions. The put_chars operation accumulates text until it encounters a newline. At that point, a message is formed and placed in the log. Only the "severity=N" mode is supported which controls the severity value of messages appended to the log (the default value is zero). The following control orders are supported: set_binary_info This control order, given a pointer and a length, copies binary data to be added to the succeeding log message(s). get_binary_info This control order returns a pointer and length of the binary data currently pending for the succeeding log message(s). io_call This order provides the command level interface (via 4-21 MDD-024 System Logging the io_call command) to all control orders except "get_write_log_data_ptr". flush_pending_output This control order will cause the current message buffers to be written to the log as a single message. (put_chars operations will not write a message until a newline is encountered). get_log_write_data_ptr This control order returns the pointer to the write_log_data structure used to write the log. The caller needs this if it is necessary to call write_log_ directly rather than through the I/O stream. match_strings_ This module is called upon by log_match_ to test a match between a selection string and a log message. It maintains internal lists of selection strings. These strings may be regular expressions. monitor_sys_log This command is used to poll a set of logs for new messages. Facilities are provided for message selection and formatting. Its usage is described fully in the published documentation. monitor_sys_log_wakeup_ This module supports the alarm timer wakeups for the monitor_sys_log command. move_log_segments This command is used to migrate log segments between directories. It helps ensure that the history directories contain succeeding older log intervals. It also sets the proper link values in log headers so that their ancestors may be found. Its usage is described fully in the published documentation. old_syserr_log_util_ This module used to provide the primary interface to the old format syserr log (a vfile_ multi-segment file). Its name was syserr_log_util_. It was adapted for conversion to the new log format (via convert_syserr_log command). print_sys_log This command is used for interactive log perusal. Its usage is described fully in the published documentation. set_log_history_dir This command is used to correct log header information about the location of the log's ancestors (should it become 4-22 System Logging MDD-024 damaged, or if move_log_segments was not properly used). Its usage is described fully in the published documentation. summarize_sys_log This command is used to condense log messages into a more comprehensible report. Its usage is described fully in the published documentation. SSSYYYSSSTTTEEEMMM SSSUUUPPPPPPOOORRRTTT FFFOOORRR AAASSS LLLOOOGGG This section describes the support provided for the answering service log. The AS log is opened during system initialization by sc_init_ which calls log_write_$open. Should an error occur, sc_init_ will invoke log_salvage_ to attempt to correct any problems. If the error persists, sc_init_ will abort initialization and enter the emergency listener. The AS log is never closed. Messages are added th the answering service log from many different modules. They all use the sys_log_ interface which is described below. Some parameters are defined in sys_log_constants.incl.pl1. sys_log_$sys_log This is the main sys_log_ entrypoint. It formats the supplied control string and optional arguments, then appends a message to the AS log. When the severity value is nonzero, the message is also printed on an appropriate message coordinator stream. sys_log_$binary This entrypoint is similar to the main entrypoint but handles binary data for the log message. sys_log_$error_log This entrypoint is similar to the main entrypoint but also takes a standard system error code. This code is converted to its textual string (via convert_status_code_) which is added into the log message. It is used in the answering service in a similar manner to com_err_. sys_log_$type_sv_zero This entrypoint is used for sys_log_ debugging. SSSYYYSSSTTTEEEMMM SSSUUUPPPPPPOOORRRTTT FFFOOORRR AAADDDMMMIIINNN LLLOOOGGG This section describes the system support of the admin log. 4-23 MDD-024 System Logging An I/O switch for the admin log is established by sc_init_ during system initialization. If any error is encountered during the open operation, sc_init_ calls log_salvage_ to attempt correction of the problem. If the problem persists, sc_init_ aborts system initialization and enters the emergency listener. The admin log is never closed. Two of the entrypoints in sys_log_ direct the message to that admin log rather than the AS log. They are described below. Note that these entries use log_write_ (via sc_stat_$admin_log_write_ptr) to append messages to the admin log. Most messages arrive via the sc_stat_$admin_log_iocb switch. sys_log_$command sys_log_$command_error These entrypoints are the same as the sys_log_$sys_log_ and sys_log_$error_log entrypoints except that the target is the admin log and that messages are printed on the admin console rather than through the message coordinator. SSSYYYSSSTTTEEEMMM SSSUUUPPPPPPOOORRRTTT FFFOOORRR MMMCCC LLLOOOGGGSSS This section describes the system support for message coordinator logs. Message coordinator logs are opened by mc_commands_$define. Currently no salvaging is performed in the event of an error (although this should be added in the future). Messages are written into these logs by mc_wakeups_ (which is invoked when the daemons have some output). Message coordinator logs are closed by mc_commands_$undefine. SSSYYYSSSTTTEEEMMM SSSUUUPPPPPPOOORRRTTT FFFOOORRR DDDMMM LLLOOOGGG This section describes the system support for the Data Management Subsystem log. The Data Management Subsystem maintains an inner-ring log which is written by many processes. Interfaces for message writing and log perusal are provided by the modules described below. dm_log_ This is the interface used by the inner-ring data management system to open the DM log and to write messages. The DM log is never closed in a process. dm_log_read_ This is a transfer vector to the corresponding dm_log_read_inner_ (via dm_admin_gate_) entrypoints. It has 4-24 System Logging MDD-024 entrypoints parallel to all the log_read_ entrypoints. These are provided to log_read_$open_long. For every log_read_ entrypoint, control is immediately transfered to the corresponding entrypoint in dm_log_read_. dm_log_read_inner_ This module provides all the log_read_ functions in the inner ring. It is basically an inner ring wrapper for log_read_. The scheme for reading the DM log can be confusing. The following illustration attempts to clarify the approach. caller (log perusal tool) | V log_read_$position_sequence (user ring) | V dm_log_read_$position_sequence (tv) | V dm_admin_gate_$dlr_position_sequence (gate) | V dm_no_firstref_tv_$dlr_position_sequence (tv) | V dm_log_read_inner_$position_sequence (wrapper) | V log_read_$position_sequence (inner ring) The user-ring invocation of log_read_ immediately transfers control to dm_log_read_ since the log cannot be read in that ring. The user-ring caller of log_read_ must supply a pointer to a user-ring area. The inner-ring invocation of log_read_ will allocate a copies of messages in this area. Copying occurs for every message to which the caller positions. Note that this approach will be taken whenever the "-dms" log selection argument is given to the log perusal tools. This is true even if the user is logging in at the DM ring. (Use of the absolute pathname of the DM log, rather than the "-dms" argument, will use the normal, more efficient, reading approach when logged in at the DM ring). SSSYYYSSSTTTEEEMMM SSSUUUPPPPPPOOORRRTTT FFFOOORRR SSSYYYSSSEEERRRRRR LLLOOOGGG This section describes the system support provided for the syserr log. The syserr log is handled very differently from the other logs. It must be writable from any environment. Thus, there is a wired area for log messages for times when the process may take 4-25 MDD-024 System Logging no faults. There is also a paged ring-zero log (log partition) for times when the process may take faults, but cannot deal with the file system. The following sections describe, in detail, the additional data structures used for the syserr log and the modules which support these structures. SSSyyyssseeerrrrrr DDDaaatttaaa SSStttrrruuuccctttuuurrreeesss The following include files define the format of the wired syserr log and the first page of the log partition. syserr_data.cds This is not an include file, but a structure defined in syserr_data.cds. It uses structures in syserr_data.incl.pl1 which is described below. The CDS creates the structure which appears as the segment "syserr_data" in the SLT. All level-2 structure members are thus available as external variables (named syserr_data$MEMBER). /* last modified 84-11-16 K Loepere */ dcl 1 syserr_data aligned based (syserr_datap), 2 log_meters (16) fixed bin, 2 syserr_size fixed bin (18), 2 wired_log_size fixed bin (18), 2 logger_proc_id bit (36), 2 pad fixed bin, 2 logger_ec fixed bin (71), 2 syserr_area aligned like sd, 2 wired_log_area aligned like wlog, 2 pad_extra_wmess (size (wmess)) bit (36); where: log_meters Apparently was meant to contain metering information. As of this writing, the only reference to this array is by init_syserr_log. Only one value is set by that routine. syserr_size There are no references to this value. It is set as the size, in words, of the "syserr_area" substructure by the CDS. wired_log_size This is set to the size, in words, of the "wired_log_area" substructure by the CDS. It is never changed. 4-26 System Logging MDD-024 logger_proc_id Is the process ID of the SyserrLogger.SysDaemon hardcore process. It is set by init_syserr_log and used whenever a message is entered into the wired log. The daemon will wakeup and copy the messages to the ring 0 paged log segment (log partition). logger_ec Is the event channel over which the logger hardcore process will accept wakeups. syserr_area Is simply the "sd" structure described below. It is used in ocdcm_ and real_initializer, as well as in the syserr log support software. wired_log_area This is the actual wired log. It is simply the "wlog" substructure described below. It contains a header and a data area in which log messages are held. syserr_real writes log messages here when the process may not take page faults, the messages are then copied to the ring 0 paged log by the logger daemon. pad_extra_wmess Not available as an external variable. [Its use is unclear. Perhaps it was used in the past as a little "safety zone" used to work around some indexing bug.] syserr_data.incl.pl1 This include file describes the wired syserr log (parts of the hardcore segment "syserr_data"). It is referenced by syserr_data.cds as described above. /* last modified 85-02-18 Keith Loepere */ dcl 1 sd based (sd_ptr) aligned, 2 lock bit (36), 2 log_flag bit (1) unal, 2 char_type_flag bit (1) unal, 2 ocdcm_init_flag bit (1) unal, 2 pad bit (33) unal, 2 prev_text_written char (80); dcl 1 wlog based (wlog_ptr) aligned, 2 head like wlog_header, 2 buffer (wlog.head.bsize) bit (36); dcl 1 wlog_header based aligned, 2 bsize fixed bin, 2 count fixed bin, 2 slog_ptr ptr, 4-27 MDD-024 System Logging 2 seq_num fixed bin (35), 2 next bit (18) unal, 2 pad bit (18) unal; dcl 1 wmess based (wmess_ptr) aligned, 2 header aligned like wmess_header, 2 text char (0 refer (wmess.text_len)), 2 data (0 refer (wmess.data_size)) bit (36); dcl 1 wmess_header based aligned, 2 seq_num fixed bin (35), 2 time fixed bin (71) unal, 2 code fixed bin (11) unal, 2 text_len fixed bin (11) unal, 2 data_size fixed bin (11) unal, 2 data_code fixed bin (11) unal, 2 pad bit (24) unal, 2 process_id bit (36); where: sd This structure overlays syserr_data$syserr_area. sd.lock This is the lock for all of the syserr_data segment. sd.log_flag This flag indicates that the syserr logging mechanism is enabled. It may be turned off by the logger daemon if more than five errors occur when trying to copy messages to the paged log. sd.char_type_flag This is used by ocdcm_ to indicate the type of console (ASCII or BCD). sd.ocdcm_init_flag This is used to indicate whether the system main console may be used. sd.prev_text_written Is the text of the last message written to the console. It is used by syserr_real to determine whether the next console message should be replaced by an equal sign (to indicate repetition on the console). wlog This structure overlays syserr_data$wired_log_area". It is the actual wired log which contains a header and a data area in which log messages are entered. 4-28 System Logging MDD-024 wlog.buffer This is the data area in which log messages are entered. The format of individual messages is defined by the "wmess" structure described below. wlog_header.bsize Is the size, in words, of the wired log buffer (wlog.buffer). wlog_header.count Is the number of messages currently in the wired log buffer. wlog_header.slog_ptr Was a pointer to the ring0 paged log (log partition) in the old syserr mechanism. It is no longer used. wlog_header.seq_num Is the sequence number of the last syserr message logged. This is updated whether the message is entered into the wired log or is directly added to the paged log. wlog_header.next Is the offset relative to the syserr_data segment which indicates where the next message may be added. wmess Is the structure of a message as it appears within wlog.buffer. It consists of a header, the message text, and (optionally) the message binary data. wmess.text Is the textual portion of the message kept in ASCII. wmess.data Is the binary data portion of the message. wmess.seq_num Is the sequence number of this message. wmess.time Is the wall clock time when the message was added. wmess.code Is the syserr code associated with the message (see syserr_constants.incl.pl1). It represents a message severity (tens digit) and an action (ones digit). wmess.text_len Is the length, in characters, of the text portion of the message. 4-29 MDD-024 System Logging wmess.data_size Is the size, in words, of the binary data portion of the message. wmess.data_code Is a fixed bin value which represents the format of the binary data (see syserr_binary_def.incl.pl1). It is converted to a data_class value by syserr_copy as the message is moved to the paged log. wmess.process_id Is the unique ID of the process that entered the message. syserr_log_dcls.incl.pl1 This include file defines the structure of the first page of the log partition. This page is defined as "syserr_log_data" in the SLT. (There are two other segments, "syserr_log_laurel" and "syserr_log_hardy", which overlay other parts of the log partition). /* last modified 84-12-10 BIM */ dcl 1 syserr_log_data aligned based (syserr_log_data_ptr), 2 version char (8) unaligned, 2 old_init_word char (4) unaligned, 2 pad003 bit (1) aligned, 2 live_log fixed bin, 2 pad001 bit (1) aligned, 2 error_count fixed bin (35), 2 swap_time fixed bin (71), 2 messages_copied fixed bin (35), 2 messages_lost fixed bin (35), 2 log_start (2) fixed bin, 2 log_size (2) fixed bin, /* all following is reinitialized every bootload */ 2 per_bootload, 3 log_ptr (2) pointer, 3 log_name (2) char (32) unaligned, 3 log_dir char (168) unaligned, 3 lock aligned, 4 pid bit (36) aligned, 4 event_id fixed bin (35), 4 notify_requested bit (1) unaligned, 4 pad bit (35) unaligned, 3 take_a_fault bit (1) aligned, 3 test_mode bit (1) aligned, 3 copy_disabled bit (1) aligned, 3 drop_severity_5 bit (1) aligned, 3 wakeup_on_printable bit (1) aligned, 3 copy_threshold fixed bin (18), 4-30 System Logging MDD-024 3 copy_channel fixed bin (71), 3 copy_process_id bit (36) aligned, 2 pad fixed bin (71); where: version Is an 8 character structure identifier. old_init_word Overlays the "init" word in the old log partition format. This is used to indiacate an old format partition - which must be reinitialized. live_log Identifies which of the two paged logs is the live log segment. Its value may be 1 or 2. error_count Is used to count the number of errors which arise during log copying (from wired to paged logs). If its value exceeds 5, log copying becomes disabled. swap_time Is set to the wall clock time when the two paged logs are swapped (that happens when the space used in the live log reaches a threshold) and set to zero when all messages have been emptied. messages_copied Is a meter which counts the number of messages copied from the wired log into the paged logs. messages_lost Is the number of messages that were discarded due to lack of room in the paged logs. log_start Is the page offset within the log partition of each of the two logs. log_size Is the size, in pages, of each of the two logs. log_ptr Are pointers to the two log segments. This two element array may be indexed with the value of "live_log" to yield a pointer to the current live log. log_name Are the respective names of the two log segments. Initially "syserr_log_laurel" and "syserr_log_hardy", 4-31 MDD-024 System Logging they are manipulated by syserr_seg_manager to indicate the live log "syserr_log", the empty log "syserr_log.empty", and, while in transition, the timestamped log "syserr_log.YYYYMMDD.HHMMSS". log_dir Is set to the value of log_data_$syserr_log_dir and never changed. It indicates the location in the hierarchy entries for the two ring0 paged log segments. lock.pid lock.event_id lock.notify_requeted Comprise the paged syserr log lock. It provides concurrency control for manipulations of the first page of the log partition and manipulations of the names of the two paged logs. take_a_fault Indicates that the software is to force a fault, for testing fault recovery. test_mode Indicates that the software is running in user-ring test mode. copy_disabled Indicates that no more messages are to be copied into the live log. This is turned on when the log becomes full. Messages from the wired log will not be copied - they will be discarded. It will be turned off when the older of the two logs becomes empty, and the logs are swapped (see syserr_copy). drop_severity_5 Indicates that severity 5 (lowest severity) messages are to be discarded - not copied into the paged log. This is turned on when the log reaches a size threshold, in the hope of retaining more important messages. It is turned off when the other log is emptied and the logs are swapped (see syserr_copy). wakeup_on_printable Indicates that the outer ring copy process (Initializer) is to be sent a wakeup whenever a message is copied from the wired log that was supposed to have been printed on the console. This is so that the outer ring copy process may deal appropriately with these messages while the main console is out of service. (Currently, the Initializer will write them on a message coordinator terminal with a prefix of "HARD", and some bells). 4-32 System Logging MDD-024 copy_threshold Is the threshold, in pages, at which the outer ring copy process (Initializer) will be sent a wakeup. This wakeup indicates that the two ring0 paged logs should be swapped, and the appropriate one emptied (i.e. messages moved into the syserr_log history directory). copy_channel Is the event channel over which the outer ring copy process (Initializer) is accepting wakeups. copy_process_id Is the process ID of the outer ring copy process (Initializer). SSSyyyssseeerrrrrr MMMoooddduuullleeesss The following modules comprise the system support of the syserr log. Note that all modules are in hardcore except syserr_log_man_. syserr_data.cds Creates the syserr_data structure which contains the wired syserr log. init_syserr_log (bound_temp_2) Is called by real_initializer to set up ASTEs for the three segments that comprise the log partition (syserr_log_data, syserr_log_laurel, and syserr_log_hardy). It checks for validity of the data in the partition, if there are discrepancies, the partition (the three segments) are completely reinitialized. The hardcore process SyserrLogger.SysDaemon is created by this routine. syserr (bound_priv_1) Is an ALM wrapper for ring 0 callers of syserr_real. Its major duty is to place the stack frame high in the stack segment if the syserr action code requests a system crash. This is to preserve whatever stack history is available for analysis purposes. syserr_real (bound_priv_1) Places messages in the syserr log. It is called through syserr by ring0 modules. Outer ring modules may call it through admin_gate_ or hphcs_. It performs the following tasks: - Format the control string and arguments via formline_. - If binary data is given, copy it onto our stack. - If a system status code (error code) is given, convert 4-33 MDD-024 System Logging it to a string and append it to the formatted text. - Determine whether copying to the paged log can be done. The rules are that the process cannot be masked, cannot be an idle process, cannot be running on the PRDS, that logging is enabled, that the paged log is not locked by the same process, and that the action code associated with the message is not one to terminate the process nor one to crash the system. - Set the write_flag and alarm_flag according to the given syserr action code (SYSERR_PRINT_ON_CONSOLE, SYSERR_PRINT_WITH_ALARM, respectively). - If copying can be done, the paged log lock is set. - The process is wired and masked. - The wired log is locked. - A console message structure is filled in. - If copying can be done, copy the messages out of the wired buffer onto the stack. - If copying cannot be done, insert the text and binary into the wired log. If there's not enough room, change the console message structure to include "*lost" which indicates that the message, although printed, will not appear in the log (the write_flag is set here). - If the write flag is on, check for a repetitive message. If this message is the same as the previous message printed, replace the text in the console message structure with an equal (=) sign. - Unlock the wired syserr log. - If the write flag is on, call ocdcm_ to print the message as it appears in the console message structure (with HHMM.S timestamp, and possible "*lost"). - If the syserr action code indicates that the process is to be terminated, then: 1) call syserr_real recursively to log/print a process termination message; 2) reset the process mask; 3) call terminate_proc (which will call syserr again recursively). - If the syserr action code indicates that the system is to be crashed, then: 1) call syserr_real recursively to log_print a system crash message; 2) call ocdcm_ to flush the pending output; 3) call BCE. - Unwire and unmask. - If copying can be done, then: 1) call syserr_copy on messages that we previously moved from the wired log to the stack frame; 2) call syserr_copy for the new message (which we haven't put into the wired log); 3) unlock the paged log lock. The differences in each entrypoint are described below. The "ring1" entries are used by gates from ring 1 (admin_gate_) and the user ring (hphcs_). They make more argument checks to prevent misuse. The following include files are useful to callers of syserr: 4-34 System Logging MDD-024 syserr_binary_def (binary data classes/codes) syserr_constants (action/severity codes) syserr_codes (same as syserr_constants, but with different names to avoid collisions) syserr_real$syserr_real syserr_real$ring1 Are the basic syserr entries. They accept a syserr action/ severity code (see syserr_constants.incl.pl1), a control string (acceptable to formline_), and optional arguments to be formatted. syserr_real$binary syserr_real$ring1_binary These entries additionally take a pointer and length of some binary data. The caller also supplies a binary data code which identifies the format of the data (see syserr_binary_def.incl.pl1). syserr_real$multiple_binary syserr_real$ring1_multiple_binary These entries accept an array of pointers and lengths of binary data. These are merged when the message is created. It is meant to preserve time and stack space for the calling module (so it doesn't have to perform this concatenation). syserr_real$error_code syserr_real$ring1_error_code These entries accept a standard system status code (e.g. error_table_ entry). It is converted (via convert_status_code_) to a string and concatenated to the text of the logged message. syserr_real$syserr_reset Unlocks the wired syserr log for emergency shutdown. syserr_real$panic Calls BCE for ocdcm_ and also for syserr_real if called before ocdcm_ is initialized. syserr_log_daemon (bound_error_wired_2) Run by the SyserrLogger.SysDaemon. It is simply a loop that calls syserr_copy_wired_log upon return from pxss$block. syserr_copy_wired_log (bound_error_wired_2) Performs the copy of messages from the wired log to the paged log. The tasks are: - Lock the paged log. - Wire and mask. - Lock the wired log. - Copy the messages in the wired log into the stack (use 4-35 MDD-024 System Logging syserr_copy). Then mark the wired log as empty. - Unlock the wired log. - Unwire and unmask. - Copy messages from stack into paged log (via syserr_copy). - Unlock the paged log. There is an additional entrypoint "syserr_copy_wired_log$adjust_sequence" which is called by syserr_seg_manager during system initialization. This happens during salvaging of the syserr log. Its task is to adjust the message sequence numbers in the wired log. syserr_copy (bound_error_active_2) Is the utility used by syserr_copy_wired_log, syserr_real, and syserr_seg_manager in the tasks of copying messages from the wired log and to the syserr history. It is also responsible for the paged syserr log lock. The entrypoints are described below: syserr_copy$wired_log Is called by syserr_copy_wired_log and syserr_real to move messages from the wired syserr log to the paged log. Both these logs must be locked. If the live paged log reaches a threshold of pages written, it will be swapped for the empty log and a wakeup sent to the outer ring copy process (Initializer). syserr_copy$swap_logs Is called by syserr_seg_manager upon a timer wakeup in the outer ring copy process. It forces a swap of the live and empty logs before the live log reaches a space threshold. (The outer ring copy process (Initializer) will initiate a copy on its own, if the live log does not reach the threshold in a certain interval - e.g. 1 hour). The paged log must be locked. syserr_copy$lock syserr_copy$unlock These two entrypoints control the lock on the paged logs. syserr_log_man_ (bound_as_misc_ in >tools) Is the outer ring copy process (Initializer) module. It has the main responsibility for copying messages from the paged ring0 logs to the syserr log history (currently in >sc1>syserr_log). It makes much use of syserr_seg_manager (via hphcs_) for inner ring (zero) manipulations of the paged log segments. The entrypoints are described below. syserr_log_man_$as_copy_init syserr_log_man_$restart_copying These entrypoints are used to startup the outer ring 4-36 System Logging MDD-024 copying. The first is called during answering service initialization by (you guessed it) as_init_. The second is used to restart the copying mechanism after it has been disabled due to errors (e.g. record-quota-overflow on the syserr log history directory). The second takes no arguments so it may be used as a command (in the outer ring copy process only). These both call syserr_log_man_$start_copying. syserr_log_man_$start_copying Is called from the previous two entrypoints. Its main purpose is to start the outer ring copying mechanism. The tasks are: - Create an event channel. Its name is placed in the syserr_log_data structure by syserr_seg_manager$start_copying. - Setup threshold values for the number of pages in the ring0 log at which to wakeup the outer ring copy process and the maximum time interval to allow before forcing an outer ring copy. - Setup the syserr log history directory if it is not already there. - Ensure that the sequence numbers in the ring0 paged log and the wired log are greater that those in the oldest log in the history directory. - Call syserr_log_man_$as_copy_log (if restarting). - Set up an alarm timer to wake us up if syserr_copy does not. syserr_log_man_$console_recovery Is called by mc_con_rec_ to note that the main console is disabled/re-enabled. It simply calls syserr_seg_manager$set_recovery_flag. When on, syserr_copy will wakeup the outer ring copy process for every message which is to be printed (on the console). This invokes syserr_log_man_$as_copy_log which (as we shall see below) will direct these messages to a message coordinator terminal. syserr_log_man_$as_copy_log This routine is called from four places: - During AS initialization. - During AS shutdown. - Upon wakeup from syserr_copy. - Upon alarm timer wakeup. It is responsible for managing the copying of the messages from the paged ring0 log to the syserr log history. Its sequence of tasks follows: 4-37 MDD-024 System Logging - Setup condition handlers. - If copying is disabled, simply return. - Initiate the timestamped log. - Open the log history (log_write_$open_for_migrate). - Use log_position_$next_message to sequence through the paged log. For segdamage/voldamage messages, make an interpreted (via binary_segmsg_util_) copy to the answering service log (this is what as_meter_ used to do for us). For messages that were to be printed on the console, if the console is disabled call mc_con_rec_$queue_log_message to print it on a message coordinator terminal. Finally, for every message call log_move_ to copy it to the log history. - Rename the timestamped log to "syserr_log.empty" (via syserr_seg_manager$reuse_empty_log). - If the new live log is not empty, force a log swap (via syserr_seg_manager$swap_logs) and repeat the last 4 steps. - Set new alarm timer if we were invoked by an alarm wakeup. - Terminate the paged log pointer and close the log history. syserr_log_man_$print_meters Is an entrypoint which may be used as a command (in the outer ring copy process only) to print a set of meters. syserr_seg_manager (bound_error_active_2) This module is called by syserr_log_man_ (via hphcs_) to perform various manipulations on the paged syserr logs. The individual entrypoints are described below. syserr_seg_manager_$initialize_log_names Sets the initial names on the two log segments as they appear in >sl1. This entrypoint is called by real_initializer during system initialization. (It cannot be called by init_syserr_log since the root dir branches do not necessarily exist at that time). syserr_seg_manager_$verify_sequence Ensures that the message sequence numbers in the paged and wired syserr logs follows that in the syserr log history. It is called during syserr_log_man_ initialization. syserr_seg_manager_$start_copying Enables the outer ring copying mechanism by placing the copying thresholds, the copying process ID, and the event channel into the syserr_log_data structure. syserr_seg_manager_$set_recovery_flag Sets the wakeup_on_printable flag in the syserr_log_data structure. The outer ring copy process will receive a 4-38 System Logging MDD-024 wakeup for every message that was supposed to be printed on the system console (which is apparently disabled). The copying process will route these messages (during the usual copy procedure) to the message coordinator. syserr_seg_manager_$segment_info Returns the value of the swap_time as it appears in the syserr_log_data structure. If this value is non-zero, there is some copying to be done. It also indicates whether the live log is empty. If not, the process may swap the logs again and take another pass at copying. syserr_seg_manager_$swap_logs Will swap the roles of the live and empty logs. It calls upon syserr_copy$swap_logs to accomplish this. syserr_seg_manager_$reuse_empty_log Will mark the timestamped log as empty, so that swaps may be done. This is called after the contents have been copied to the syserr log history. syserr_seg_manager_$verify_lock Is used by verify_lock on crawlout from ring 0. It does not unlock any locks, but disables outer ring copying if it is set (with an appropriate console message). SSSpppeeeccciiiaaalll SSSuuuppppppooorrrttt iiinnn SSStttaaannndddaaarrrddd LLLoooggg SSSoooffftttwwwaaarrreee Some of the code in the standard log software appears to be in support of the syserr log. Certainly, log_data_.cds defines some external variables for syserr log support. All the other special code deals with the fact that log_write_ is used only on the syserr log history directory. It allows various modules to ignore the fact that all log segments in this directory are timestamped (i.e. there appears no "live" log here). log_write_$open_for_migrate allows an opening where all the logs have names exhibiting a timestamp. log_list_history_$single_dir_oldest_and_newest supports log_write_$open_for_migrate in that it finds the newest log - the one where messages should be added. log_move_ has the sole purpose of copying structured log messages from one log segment into another. log_write_$general is used by log_move_ to create messages and supply the sequence number. 4-39 MDD-024 System Logging log_segment_$create_message_number is used by log_write_$general to create messages with the given sequence number. expand_log_message_ handles messages with the data class "syserr" in a special manner. That is, it looks in the first word of binary data itself for a code which describes the data class (this is actually obsolete since no existing code generates messages with that data class - it was a temporary measure used during MR11 development). Except for the support by log_data_.cds and expand_log_message_, all this code may have another use. That is for the "trim_sys_log" function described below under "Future Enhancements". CCCooommmpppaaatttaaabbbiiillliiitttyyy Many system utilities reference the syserr log to produce various reports (e.g. the "heals" facility). To avoid upgrading all these tools to use the new logging interfaces, the old interfaces were retained. These are implemented by the syserr_log_util_ and print_syserr_msg_ subroutines (both reside in bound_admin_tools_). The internal code of these subroutines was changed to reference the new logging interfaces, but convert the messages to the old format. The messages returned are in the format defined in syserr_message.incl.pl1. 4-40 System Logging MDD-024 SECTION 5 SECURITY The security of logs is completely dependant on the standard security features of Multics (ACL, rings, AIM). The following paragraphs describe actions taken in maintaining access lists and labels. AAACCCCCCEEESSSSSS PPPRRROOOPPPEEERRRTTTIIIEEESSS OOONNN NNNEEEWWW LLLOOOGGG SSSEEEGGGMMMEEENNNTTTSSS The initial log segment is created, by log_write_, with the following access attributes: Rings = process validation level Access Class = parent directory access class Multi-Class = false ACL = System Default (User + IACL + SysDaemon) When subsequent log segments are created, these access attributes are copied from the filled segment. Therefore, any changes made (via standard or privileged commands) will be propogated to new log segments. When log segments are moved (via move_log_segments) the access attributes are copied. However, the access_class will be the same as that of the target directory (and the multi-class switch will be off). Note that the log's containing directory must exhibit "sma" effective access for the writer of the log, and "s" for all readers of the log. The details of log segment creation can be found in log_write_, log_create_, and log_salvage_. AAACCCCCCEEESSSSSS FFFOOORRR MMMUUULLLTTTIIIPPPLLLEEE WWWRRRIIITTTEEERRRSSS When a number of users are to write to a log, the log must exhibit "rw" effective access for all those users. Since 5-1 MDD-024 System Logging standard logs are user-ring objects, there is no protection of the logs integrity. Logs which must be protected should be placed in an inner ring. It is expected that the software which enters log messages is also in an inner ring. Therefore, the standard log_write_ interface can be used to write messages. Perusal of inner ring logs, if to be done from the user ring, may be accomplished by writing a gate and a wrapper for log_read_ as was done for the Data Management log (see above). Reading of the log is controlled by the access on the gate. However, if only writing is to be protected, the read bracket may be set to the user ring and no log_read_ wrapper is necessary. AAACCCCCCEEESSSSSS FFFOOORRR SSSYYYSSSEEERRRRRR LLLOOOGGG Access on the syserr log is slightly more complex since there are three logs (history, paged, and wired). The following paragraphs describe the access for each. The syserr log history (presently residing in the directory >sc1>syserr_log) is treated as a standard log. New segments acquire the same access properties as their ancestors. The standard access manipulation commands may be used by system administrators. Access to the log segments should be "rw" for the outer ring copy process (Initializer) and "r" for readers (administrators, system maintenance staff, hardware field engineers, etc). The containing directory should have "sma" access for the outer ring copy process and the system administrators (so move_log_segments will work). It should have "s" access for any log readers. The access on the paged ring0 logs is "rw" for all users in ring 0 when they are used as hardcore segments (through the syserr_copy mechanism). These segments also have directory entries in >sl1. References outside of ring 0 will be made through a different SDW than in ring 0. The access properties are initially setup in the hardcore header. The ring brackets are 4,5,5. The access should be "rw" for the outer ring copy process (Initializer) and "r" for any log readers (the standard header has "r" for SysAdmin, SysMaint, and SysDaemon and "null" for "*"). Because ACL manipulation via the hardcore header is not convenient it is acceptable to change the ACL on these logs via the system_start_up.ec. A command line to set the ACL to the same values as on the history logs is: set_acl >sl1>syserr_log.** [list_acl [do "&1" [segments >sc1>syserr_log>** -absp]]] 5-2 System Logging MDD-024 The wired log is in hardcore segment "syserr_data". It does not have a hierarchy entry. It has "rw" access for all users in ring 0 only. It is completely under the control of syserr_real and syserr_copy_wired_log. There is no way to read the wired log (other than ring_zero_dump). The messages are quickly moved to the paged log by the SyserrLogger daemon where they may be perused. 5-3 System Logging MDD-024 SECTION 6 FUTURE ENHANCEMENTS This section outlines some of the enhancements to be considered in the future. SSSEEELLLEEECCCTTTIIIVVVEEE LLLOOOGGG TTTRRRIIIMMMMMMIIINNNGGG It is desirable to retain certain log messages for a longer duration than others. For instance, messages about CPU hardware errors could be kept for a year or so for historical purposes in analyzing new problems. On the other hand, messages about FNP polling are only useful for a limited time. The cost of log storage would be greatly reduced if only the interesting messages were retained. However, the current implementation of move_log_segments works at the segment level, not with messages. The original design of the logging software intended to perform log moving on a message basis. It would be similar to the message migration technique used for the paged syserr log. In fact, all the support for message migration, while it appears to be specially coded for use with the syserr log, was also intended for use with a "log_migrate_" subroutine. If messages were to be migrated to log history directories on a per-message basis, it would be a simple matter to introduce a filter into that process. This filter would discard uninteresting messages, leaving the log history a little smaller. A "migrate_sys_log" command would be provided to setup the filter (possibly with print_sys_log type arguments - "-match", "-exclude", etc). Note that a log administrator could set up a series of log history directories. Each succeeding directory would contain logs with only the messages which are interesting for greater duration. 6-1 MDD-024 System Logging MMMUUULLLTTTIIIPPPLLLEEE LLLOOOGGG PPPEEERRRUUUSSSAAALLL Often, series of messages in multiple logs are perused in analysis of subsystem behavior. The messages must be collated mentally or through the use of an editor. print_sys_log should be enhanced to peruse multiple logs simultaneously, collating the messages (by time entered) for the display. RRREEEPPPLLLAAACCCEEEMMMEEENNNTTT OOOFFF WWWIIIRRREEEDDD SSSYYYSSSEEERRRRRR LLLOOOGGG Messages appearing in the wired syserr log currently have their own format. This causes some inefficiencies in copying messages to the paged log, and does not contain all the same information (e.g. data codes vs. data classes). The code which supports the wired log (syserr_real, syserr_copy_wired_log, and syserr_copy) should be changed to support standard messages in the wired log. The orignal design of the logging software intended to split the wired log into two buffers to be used in a manner analogous to the two paged logs. The difference is that the hardcore logger daemon would continue to be sent a wakeup for every message, not just when one of the buffers fills to a threshold. This is important to avoid message loss and to ensure the ability to peruse new messages. NNNEEEWWW SSSYYYSSSEEERRRRRR$$$BBBIIINNNAAARRRYYY IIINNNTTTEEERRRFFFAAACCCEEE The syserr$binary and syserr$multiple_binary interfaces should be changed to accept a character string data class argument (instead of the current data code). RRREEEMMMOOOVVVEEE SSSYYYSSSEEERRRRRR_LLLOOOGGG_UUUTTTIIILLL_ IIINNNTTTEEERRRFFFAAACCCEEE The old syserr_log_util_ routine (and its companion print_syserr_msg_) should eventually be removed. All present callers should be upgraded to use log_read_. Moving these interfaces to >obsolete will allow customers to convert at their leisure (although they should do this under MR11). ----------------------------------------------------------- Historical Background This edition of the Multics software materials and documentation is provided and donated to Massachusetts Institute of Technology by Group BULL including BULL HN Information Systems Inc. as a contribution to computer science knowledge. This donation is made also to give evidence of the common contributions of Massachusetts Institute of Technology, Bell Laboratories, General Electric, Honeywell Information Systems Inc., Honeywell BULL Inc., Groupe BULL and BULL HN Information Systems Inc. to the development of this operating system. Multics development was initiated by Massachusetts Institute of Technology Project MAC (1963-1970), renamed the MIT Laboratory for Computer Science and Artificial Intelligence in the mid 1970s, under the leadership of Professor Fernando Jose Corbato. Users consider that Multics provided the best software architecture for managing computer hardware properly and for executing programs. Many subsequent operating systems incorporated Multics principles. Multics was distributed in 1975 to 2000 by Group Bull in Europe , and in the U.S. by Bull HN Information Systems Inc., as successor in interest by change in name only to Honeywell Bull Inc. and Honeywell Information Systems Inc. . ----------------------------------------------------------- Permission to use, copy, modify, and distribute these programs and their documentation for any purpose and without fee is hereby granted,provided that the below copyright notice and historical background appear in all copies and that both the copyright notice and historical background and this permission notice appear in supporting documentation, and that the names of MIT, HIS, BULL or BULL HN not be used in advertising or publicity pertaining to distribution of the programs without specific prior written permission. Copyright 1972 by Massachusetts Institute of Technology and Honeywell Information Systems Inc. Copyright 2006 by BULL HN Information Systems Inc. Copyright 2006 by Bull SAS All Rights Reserved