Jan 25 20:49:10 black-mesa corosync[27062]: [MAIN ] Corosync Cluster Engine ('1.1.2'): started and ready to provide service. Jan 25 20:49:10 black-mesa corosync[27062]: [MAIN ] Corosync built-in features: nss Jan 25 20:49:10 black-mesa corosync[27062]: [MAIN ] Successfully configured openais services to load Jan 25 20:49:10 black-mesa corosync[27062]: [MAIN ] Successfully read main configuration file '/etc/corosync/corosync.conf'. Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Token Timeout (1000 ms) retransmit timeout (238 ms) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] token hold (180 ms) retransmits before loss (4 retrans) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] join (50 ms) send_join (0 ms) consensus (800 ms) merge (200 ms) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] downcheck (1000 ms) fail to recv const (50 msgs) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] seqno unchanged const (30 rotations) Maximum network MTU 1402 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] window size per rotation (50 messages) maximum messages per rotation (17 messages) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] send threads (0 threads) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] RRP token expired timeout (238 ms) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] RRP token problem counter (2000 ms) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] RRP threshold (10 problem count) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] RRP mode set to none. Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] heartbeat_failures_allowed (0) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] max_network_delay (50 ms) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Initializing transport (UDP/IP). Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0). Jan 25 20:49:10 black-mesa corosync[27062]: [MAIN ] Compatibility mode set to whitetank. Using V1 and V2 of the synchronization engine. Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Receive multicast socket recv buffer size (262142 bytes). Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Transmit multicast socket send buffer size (262142 bytes). Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] The network interface [10.7.0.1] is now up. Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Created or loaded sequence id 76.10.7.0.1 for this ring. Jan 25 20:49:10 black-mesa corosync[27062]: [SERV ] Service engine loaded: openais cluster membership service B.01.01 Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Evt exec init request Jan 25 20:49:10 black-mesa corosync[27062]: [SERV ] Service engine loaded: openais event service B.01.01 Jan 25 20:49:10 black-mesa corosync[27062]: [SERV ] Service engine loaded: openais checkpoint service B.01.01 Jan 25 20:49:10 black-mesa corosync[27062]: [SERV ] Service engine loaded: openais availability management framework B.01.01 Jan 25 20:49:10 black-mesa corosync[27062]: [MSG ] [DEBUG]: msg_exec_init_fn Jan 25 20:49:10 black-mesa corosync[27062]: [SERV ] Service engine loaded: openais message service B.03.01 Jan 25 20:49:10 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_exec_init_fn Jan 25 20:49:10 black-mesa corosync[27062]: [SERV ] Service engine loaded: openais distributed locking service B.03.01 Jan 25 20:49:10 black-mesa corosync[27062]: [SERV ] Service engine loaded: openais timer service A.01.01 Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: process_ais_conf: Reading configure Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: config_find_init: Local handle: 5912924842687987714 for logging Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: config_find_next: Processing additional logging options... Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: get_config_opt: Found 'on' for option: debug Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: get_config_opt: Defaulting to 'off' for option: to_file Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: get_config_opt: Found 'daemon' for option: syslog_facility Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: config_find_init: Local handle: 3984067077437652995 for service Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: config_find_next: Processing additional service options... Jan 25 20:49:10 black-mesa last message repeated 7 times Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: get_config_opt: Defaulting to 'no' for option: use_logd Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: get_config_opt: Defaulting to 'no' for option: use_mgmtd Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: pcmk_startup: CRM: Initialized Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] Logging: Initialized pcmk_startup Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: pcmk_startup: Maximum core file size is: 18446744073709551615 Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: pcmk_startup: Service: 9 Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: pcmk_startup: Local hostname: black-mesa.mit.edu Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: pcmk_update_nodeid: Local node id: 16779018 Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: update_member: Creating entry for node 16779018 born on 0 Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: update_member: 0x64a870 Node 16779018 now known as black-mesa.mit.edu (was: (null)) Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: update_member: Node black-mesa.mit.edu now has 1 quorum votes (was 0) Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: update_member: Node 16779018/black-mesa.mit.edu is now: member Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: spawn_child: Forked child 27068 for process stonithd Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: spawn_child: Forked child 27069 for process cib Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: spawn_child: Forked child 27070 for process lrmd Jan 25 20:49:10 black-mesa stonithd: [27068]: info: G_main_add_SignalHandler: Added signal handler for signal 10 Jan 25 20:49:10 black-mesa stonithd: [27068]: info: G_main_add_SignalHandler: Added signal handler for signal 12 Jan 25 20:49:10 black-mesa stonithd: [27068]: info: Stack hogger failed 0xffffffff Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: spawn_child: Forked child 27071 for process attrd Jan 25 20:49:10 black-mesa stonithd: [27068]: debug: pid 27068 locked in memory. Jan 25 20:49:10 black-mesa stonithd: [27068]: info: crm_cluster_connect: Connecting to OpenAIS Jan 25 20:49:10 black-mesa stonithd: [27068]: info: init_ais_connection: Creating connection to our AIS plugin Jan 25 20:49:10 black-mesa cib: [27069]: info: Invoked: /usr/lib/heartbeat/cib Jan 25 20:49:10 black-mesa cib: [27069]: info: G_main_add_TriggerHandler: Added signal manual handler Jan 25 20:49:10 black-mesa cib: [27069]: info: G_main_add_SignalHandler: Added signal handler for signal 17 Jan 25 20:49:10 black-mesa cib: [27069]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest: /var/lib/heartbeat/crm/cib.xml.sig) Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa cib: [27069]: debug: log_data_element: readCibXmlFile: [on-disk] Jan 25 20:49:10 black-mesa pengine: [27072]: info: Invoked: /usr/lib/heartbeat/pengine Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: spawn_child: Forked child 27072 for process pengine Jan 25 20:49:10 black-mesa lrmd: [27070]: info: G_main_add_SignalHandler: Added signal handler for signal 15 Jan 25 20:49:10 black-mesa crmd: [27073]: info: Invoked: /usr/lib/heartbeat/crmd Jan 25 20:49:10 black-mesa cib: [27069]: debug: activateCibXml: Triggering CIB write for start op Jan 25 20:49:10 black-mesa cib: [27069]: info: startCib: CIB Initialization completed successfully Jan 25 20:49:10 black-mesa pengine: [27072]: debug: main: Checking for old instances of pengine Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: spawn_child: Forked child 27073 for process crmd Jan 25 20:49:10 black-mesa lrmd: [27070]: info: G_main_add_SignalHandler: Added signal handler for signal 17 Jan 25 20:49:10 black-mesa crmd: [27073]: info: main: CRM Hg Version: cebe2b6ff49b36b29a3bd7ada1c4701c7470febe Jan 25 20:49:10 black-mesa cib: [27069]: info: crm_cluster_connect: Connecting to OpenAIS Jan 25 20:49:10 black-mesa pengine: [27072]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/pengine Jan 25 20:49:10 black-mesa corosync[27062]: [SERV ] Service engine loaded: Pacemaker Cluster Manager 1.0.6 Jan 25 20:49:10 black-mesa lrmd: [27070]: debug: Enabling coredumps Jan 25 20:49:10 black-mesa crmd: [27073]: info: crmd_init: Starting crmd Jan 25 20:49:10 black-mesa cib: [27069]: info: init_ais_connection: Creating connection to our AIS plugin Jan 25 20:49:10 black-mesa pengine: [27072]: debug: init_client_ipc_comms_nodispatch: Could not init comms on: /var/run/crm/pengine Jan 25 20:49:10 black-mesa corosync[27062]: [SERV ] Service engine loaded: corosync extended virtual synchrony service Jan 25 20:49:10 black-mesa lrmd: [27070]: info: G_main_add_SignalHandler: Added signal handler for signal 10 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: s_crmd_fsa: Processing I_STARTUP: [ state=S_STARTING cause=C_STARTUP origin=crmd_init ] Jan 25 20:49:10 black-mesa pengine: [27072]: debug: main: Init server comms Jan 25 20:49:10 black-mesa corosync[27062]: [SERV ] Service engine loaded: corosync configuration service Jan 25 20:49:10 black-mesa cib: [27069]: info: init_ais_connection: AIS connection established Jan 25 20:49:10 black-mesa lrmd: [27070]: info: G_main_add_SignalHandler: Added signal handler for signal 12 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_LOG Jan 25 20:49:10 black-mesa pengine: [27072]: info: main: Starting pengine Jan 25 20:49:10 black-mesa corosync[27062]: [SERV ] Service engine loaded: corosync cluster closed process group service v1.01 Jan 25 20:49:10 black-mesa attrd: [27071]: info: Invoked: /usr/lib/heartbeat/attrd Jan 25 20:49:10 black-mesa lrmd: [27070]: debug: main: run the loop... Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_STARTUP Jan 25 20:49:10 black-mesa corosync[27062]: [SERV ] Service engine loaded: corosync cluster config database access v1.01 Jan 25 20:49:10 black-mesa cib: [27069]: info: get_ais_nodeid: Server details: id=16779018 uname=black-mesa.mit.edu Jan 25 20:49:10 black-mesa attrd: [27071]: info: main: Starting up Jan 25 20:49:10 black-mesa lrmd: [27070]: info: Started. Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_startup: Registering Signal Handlers Jan 25 20:49:10 black-mesa corosync[27062]: [SERV ] Service engine loaded: corosync profile loading service Jan 25 20:49:10 black-mesa cib: [27069]: debug: crm_new_peer: Creating entry for node black-mesa.mit.edu/16779018 Jan 25 20:49:10 black-mesa attrd: [27071]: info: crm_cluster_connect: Connecting to OpenAIS Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_startup: Creating CIB and LRM objects Jan 25 20:49:10 black-mesa corosync[27062]: [SERV ] Service engine loaded: corosync cluster quorum service v0.1 Jan 25 20:49:10 black-mesa cib: [27069]: info: crm_new_peer: Node black-mesa.mit.edu now has id: 16779018 Jan 25 20:49:10 black-mesa attrd: [27071]: info: init_ais_connection: Creating connection to our AIS plugin Jan 25 20:49:10 black-mesa crmd: [27073]: info: G_main_add_SignalHandler: Added signal handler for signal 17 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] entering GATHER state from 15. Jan 25 20:49:10 black-mesa cib: [27069]: info: crm_new_peer: Node 16779018 is now known as black-mesa.mit.edu Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_CIB_START Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Creating commit token because I am the rep. Jan 25 20:49:10 black-mesa attrd: [27071]: info: init_ais_connection: AIS connection established Jan 25 20:49:10 black-mesa cib: [27069]: info: cib_init: Starting cib mainloop Jan 25 20:49:10 black-mesa crmd: [27073]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Saving state aru 0 high seq received 0 Jan 25 20:49:10 black-mesa cib: [27069]: info: ais_dispatch: Membership 80: quorum still lost Jan 25 20:49:10 black-mesa attrd: [27071]: info: get_ais_nodeid: Server details: id=16779018 uname=black-mesa.mit.edu Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Storing new sequence id for ring 50 Jan 25 20:49:10 black-mesa cib: [27069]: info: crm_update_peer: Node black-mesa.mit.edu: id=16779018 state=member (new) addr=r(0) ip(10.7.0.1) (new) votes=1 (new) born=0 seen=80 proc=00000000000000000000000000013312 (new) Jan 25 20:49:10 black-mesa attrd: [27071]: debug: crm_new_peer: Creating entry for node black-mesa.mit.edu/16779018 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] entering COMMIT state. Jan 25 20:49:10 black-mesa cib: [27069]: debug: Forking temp process write_cib_contents Jan 25 20:49:10 black-mesa attrd: [27071]: info: crm_new_peer: Node black-mesa.mit.edu now has id: 16779018 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] got commit token Jan 25 20:49:10 black-mesa crmd: [27073]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback Jan 25 20:49:10 black-mesa attrd: [27071]: info: crm_new_peer: Node 16779018 is now known as black-mesa.mit.edu Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] entering RECOVERY state. Jan 25 20:49:10 black-mesa crmd: [27073]: debug: cib_native_signon_raw: Connection to CIB successful Jan 25 20:49:10 black-mesa attrd: [27071]: info: main: Cluster connection active Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] position [0] member 10.7.0.1: Jan 25 20:49:10 black-mesa crmd: [27073]: info: do_cib_control: CIB connection established Jan 25 20:49:10 black-mesa cib: [27069]: debug: cib_common_callback_worker: Setting cib_refresh_notify callbacks for 27073 (a7fdc0f0-48f1-4f43-b45c-7bc3eaac964b): on Jan 25 20:49:10 black-mesa attrd: [27071]: info: main: Accepting attribute updates Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] previous ring seq 76 rep 10.7.0.1 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_HA_CONNECT Jan 25 20:49:10 black-mesa attrd: [27071]: info: main: Starting mainloop... Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] aru 0 high delivered 0 received flag 1 Jan 25 20:49:10 black-mesa crmd: [27073]: info: crm_cluster_connect: Connecting to OpenAIS Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Did not need to originate any messages in recovery. Jan 25 20:49:10 black-mesa crmd: [27073]: info: init_ais_connection: Creating connection to our AIS plugin Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] got commit token Jan 25 20:49:10 black-mesa crmd: [27073]: info: init_ais_connection: AIS connection established Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Sending initial ORF token Jan 25 20:49:10 black-mesa crmd: [27073]: info: get_ais_nodeid: Server details: id=16779018 uname=black-mesa.mit.edu Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: crm_new_peer: Creating entry for node black-mesa.mit.edu/16779018 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] install seq 0 aru 0 high seq received 0 Jan 25 20:49:10 black-mesa crmd: [27073]: info: crm_new_peer: Node black-mesa.mit.edu now has id: 16779018 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 Jan 25 20:49:10 black-mesa crmd: [27073]: info: crm_new_peer: Node 16779018 is now known as black-mesa.mit.edu Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] install seq 0 aru 0 high seq received 0 Jan 25 20:49:10 black-mesa crmd: [27073]: info: do_ha_control: Connected to the cluster Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_READCONFIG Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] install seq 0 aru 0 high seq received 0 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_LRM_CONNECT Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_lrm_control: Connecting to the LRM Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] install seq 0 aru 0 high seq received 0 Jan 25 20:49:10 black-mesa lrmd: [27070]: debug: on_msg_register:client crmd [27073] registered Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] retrans flag count 4 token aru 0 install seq 0 aru 0 0 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_lrm_control: LRM connection established Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] recovery to regular 1-0 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_CCM_CONNECT Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering to app 1 to 0 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_STARTED Jan 25 20:49:10 black-mesa corosync[27062]: [CLM ] CLM CONFIGURATION CHANGE Jan 25 20:49:10 black-mesa crmd: [27073]: info: do_started: Delaying start, CCM (0000000000100000) not connected Jan 25 20:49:10 black-mesa corosync[27062]: [CLM ] New Configuration: Jan 25 20:49:10 black-mesa crmd: [27073]: debug: register_fsa_input_adv: Stalling the FSA pending further input: cause=C_FSA_INTERNAL Jan 25 20:49:10 black-mesa corosync[27062]: [CLM ] Members Left: Jan 25 20:49:10 black-mesa crmd: [27073]: debug: s_crmd_fsa: Exiting the FSA: queue=0, fsa_actions=0x2, stalled=true Jan 25 20:49:10 black-mesa corosync[27062]: [CLM ] Members Joined: Jan 25 20:49:10 black-mesa crmd: [27073]: debug: fsa_dump_inputs: Added input: 0000000000000100 (R_CIB_CONNECTED) Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Evt conf change 1 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: fsa_dump_inputs: Added input: 0000000000000800 (R_LRM_CONNECTED) Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] m 0, j 0 l 0 Jan 25 20:49:10 black-mesa crmd: [27073]: info: crmd_init: Starting crmd's mainloop Jan 25 20:49:10 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_confchg_fn Jan 25 20:49:10 black-mesa crmd: [27073]: info: ais_dispatch: Membership 80: quorum still lost Jan 25 20:49:10 black-mesa corosync[27062]: [MSG ] [DEBUG]: msg_confchg_fn Jan 25 20:49:10 black-mesa crmd: [27073]: info: crm_update_peer: Node black-mesa.mit.edu: id=16779018 state=member (new) addr=r(0) ip(10.7.0.1) (new) votes=1 (new) born=0 seen=80 proc=00000000000000000000000000013312 (new) Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 80: memb=0, new=0, lost=0 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: post_cache_update: Updated cache after membership event 80. Jan 25 20:49:10 black-mesa corosync[27062]: [CLM ] CLM CONFIGURATION CHANGE Jan 25 20:49:10 black-mesa crmd: [27073]: debug: post_cache_update: post_cache_update added action A_ELECTION_CHECK to the FSA Jan 25 20:49:10 black-mesa corosync[27062]: [CLM ] New Configuration: Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_STARTED Jan 25 20:49:10 black-mesa corosync[27062]: [CLM ] ^Ir(0) ip(10.7.0.1) Jan 25 20:49:10 black-mesa crmd: [27073]: info: do_started: Delaying start, Config not read (0000000000000040) Jan 25 20:49:10 black-mesa corosync[27062]: [CLM ] Members Left: Jan 25 20:49:10 black-mesa crmd: [27073]: debug: register_fsa_input_adv: Stalling the FSA pending further input: cause=C_FSA_INTERNAL Jan 25 20:49:10 black-mesa corosync[27062]: [CLM ] Members Joined: Jan 25 20:49:10 black-mesa crmd: [27073]: debug: s_crmd_fsa: Exiting the FSA: queue=0, fsa_actions=0x200000002, stalled=true Jan 25 20:49:10 black-mesa corosync[27062]: [CLM ] ^Ir(0) ip(10.7.0.1) Jan 25 20:49:10 black-mesa crmd: [27073]: debug: config_query_callback: Call 3 : Parsing CIB options Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Evt conf change 0 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: cluster_option: Using default value '60s' for cluster option 'dc-deadtime' Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] m 1, j 1 l 0 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: cluster_option: Using default value '15min' for cluster option 'cluster-recheck-interval' Jan 25 20:49:10 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_confchg_fn Jan 25 20:49:10 black-mesa crmd: [27073]: debug: cluster_option: Using default value '2min' for cluster option 'election-timeout' Jan 25 20:49:10 black-mesa corosync[27062]: [MSG ] [DEBUG]: msg_confchg_fn Jan 25 20:49:10 black-mesa crmd: [27073]: debug: cluster_option: Using default value '20min' for cluster option 'shutdown-escalation' Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 80: memb=1, new=1, lost=0 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: cluster_option: Using default value '3min' for cluster option 'crmd-integration-timeout' Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: pcmk_peer_update: NEW: black-mesa.mit.edu 16779018 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: cluster_option: Using default value '30min' for cluster option 'crmd-finalization-timeout' Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] debug: pcmk_peer_update: Node 16779018 has address r(0) ip(10.7.0.1) Jan 25 20:49:10 black-mesa crmd: [27073]: info: config_query_callback: Checking for expired actions every 900000ms Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: pcmk_peer_update: MEMB: black-mesa.mit.edu 16779018 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_STARTED Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] debug: send_cluster_id: Local update: id=16779018, born=0, seq=80 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_started: Init server comms Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: update_member: Node black-mesa.mit.edu now has process list: 00000000000000000000000000013312 (78610) Jan 25 20:49:10 black-mesa crmd: [27073]: info: do_started: The local CRM is operational Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] This node is within the primary component and will provide service. Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_ELECTION_CHECK Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] entering OPERATIONAL state. Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_election_check: Ignore election check: we not in an election Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] A processor joined or left the membership and a new membership was formed. Jan 25 20:49:10 black-mesa crmd: [27073]: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_STARTING cause=C_FSA_INTERNAL origin=do_started ] Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_LOG Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering 0 to 1 Jan 25 20:49:10 black-mesa crmd: [27073]: info: do_state_transition: State transition S_STARTING -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_started ] Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 1 to pending delivery queue Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_INTEGRATE_TIMER_STOP Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] debug: pcmk_cluster_id_callback: Node update: black-mesa.mit.edu (1.0.6) Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_FINALIZE_TIMER_STOP Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] confchg entries 1 Jan 25 20:49:10 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_CL_JOIN_QUERY Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 16779018 Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Synchronization barrier completed Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Synchronization actions starting for (openais cluster membership service B.01.01) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering 1 to 2 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 2 to pending delivery queue Jan 25 20:49:10 black-mesa corosync[27062]: [CLM ] got nodejoin message 10.7.0.1 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 1 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering 2 to 3 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 3 to pending delivery queue Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] confchg entries 1 Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 16779018 Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Synchronization barrier completed Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Committing synchronization for (openais cluster membership service B.01.01) Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Synchronization actions starting for (dummy AMF service) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 2 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 3 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering 3 to 4 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 4 to pending delivery queue Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] confchg entries 1 Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 16779018 Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Synchronization barrier completed Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Committing synchronization for (dummy AMF service) Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Synchronization actions starting for (openais checkpoint service B.01.01) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering 4 to 5 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 5 to pending delivery queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 4 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 5 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering 5 to 6 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 6 to pending delivery queue Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] confchg entries 1 Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 16779018 Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Synchronization barrier completed Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Committing synchronization for (openais checkpoint service B.01.01) Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Synchronization actions starting for (openais event service B.01.01) Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Evt synchronize initialization Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] My node ID r(0) ip(10.7.0.1) Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Process Evt synchronization Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Send max event ID updates Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Process Evt synchronization Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Send open count updates Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] DONE Sending open counts Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 6 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering 6 to 7 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 7 to pending delivery queue Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Remote channel operation request Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] my node ID: 0x100070a Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Receive EVT_CONF_CHANGE_DONE from nodeid r(0) ip(10.7.0.1) members 1 checked in 1 Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Process Evt synchronization Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] DONE Sending retained events Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering 7 to 8 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 8 to pending delivery queue Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Remote channel operation request Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] my node ID: 0x100070a Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Receive EVT_CONF_DONE from nodeid r(0) ip(10.7.0.1) , members 1 checked in 1 Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Process Evt synchronization Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Recovery complete Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 7 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 8 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering 8 to 9 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 9 to pending delivery queue Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] confchg entries 1 Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 16779018 Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Synchronization barrier completed Jan 25 20:49:10 black-mesa corosync[27062]: [EVT ] Evt synchronize activation Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Committing synchronization for (openais event service B.01.01) Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Synchronization actions starting for (corosync cluster closed process group service v1.01) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering 9 to b Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq a to pending delivery queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq b to pending delivery queue Jan 25 20:49:10 black-mesa corosync[27062]: [CPG ] downlist left_list: 0 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 9 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering b to c Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq c to pending delivery queue Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] confchg entries 1 Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 16779018 Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Synchronization barrier completed Jan 25 20:49:10 black-mesa corosync[27062]: [SYNC ] Committing synchronization for (corosync cluster closed process group service v1.01) Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including b Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering c to d Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq d to pending delivery queue Jan 25 20:49:10 black-mesa corosync[27062]: [MAIN ] Completed service synchronization, ready to provide service. Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including c Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including d Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: pcmk_ipc: Recorded connection 0x651580 for cib/27069 Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: pcmk_ipc: Sending membership update 80 to cib Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] debug: process_ais_message: Msg[0] (dest=local:ais, from=black-mesa.mit.edu:cib.27069, remote=true, size=6): 27069 Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: pcmk_ipc: Recorded connection 0x652e60 for attrd/27071 Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] debug: process_ais_message: Msg[0] (dest=local:ais, from=black-mesa.mit.edu:attrd.27071, remote=true, size=6): 27071 Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: pcmk_ipc: Recorded connection 0x6532a0 for crmd/27073 Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: pcmk_ipc: Sending membership update 80 to crmd Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] debug: process_ais_message: Msg[0] (dest=local:ais, from=black-mesa.mit.edu:crmd.27073, remote=true, size=6): 27073 Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering d to e Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq e to pending delivery queue Jan 25 20:49:10 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including e Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: update_expected_votes: Expected quorum votes 1024 -> 2 Jan 25 20:49:10 black-mesa stonithd: [27068]: info: init_ais_connection: AIS connection established Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] info: pcmk_ipc: Recorded connection 0x651140 for stonithd/27068 Jan 25 20:49:10 black-mesa corosync[27062]: [pcmk ] debug: process_ais_message: Msg[0] (dest=local:ais, from=black-mesa.mit.edu:stonithd.27068, remote=true, size=6): 27068 Jan 25 20:49:10 black-mesa stonithd: [27068]: info: get_ais_nodeid: Server details: id=16779018 uname=black-mesa.mit.edu Jan 25 20:49:10 black-mesa stonithd: [27068]: info: crm_new_peer: Node black-mesa.mit.edu now has id: 16779018 Jan 25 20:49:10 black-mesa stonithd: [27068]: info: crm_new_peer: Node 16779018 is now known as black-mesa.mit.edu Jan 25 20:49:10 black-mesa stonithd: [27068]: debug: apichan=0x6159f0 Jan 25 20:49:10 black-mesa stonithd: [27068]: debug: callback_chan=0x617ba0 Jan 25 20:49:10 black-mesa stonithd: [27068]: notice: /usr/lib/heartbeat/stonithd start up successfully. Jan 25 20:49:10 black-mesa stonithd: [27068]: info: G_main_add_SignalHandler: Added signal handler for signal 17 Jan 25 20:49:10 black-mesa cib: [27078]: info: write_cib_contents: Archived previous version as /var/lib/heartbeat/crm/cib-18.raw Jan 25 20:49:10 black-mesa cib: [27078]: info: write_cib_contents: Wrote version 0.18.0 of the CIB to disk (digest: 4a890fa13b6218c46a0be63dd1ae5932) Jan 25 20:49:10 black-mesa cib: [27078]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.vzDnaW (digest: /var/lib/heartbeat/crm/cib.f4TvnW) Jan 25 20:49:10 black-mesa cib: [27069]: info: Managed write_cib_contents process 27078 exited with return code 0. Jan 25 20:49:11 black-mesa crmd: [27073]: debug: do_cl_join_query: Querying for a DC Jan 25 20:49:11 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_DC_TIMER_START Jan 25 20:49:11 black-mesa crmd: [27073]: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:60000ms), src=11 Jan 25 20:49:11 black-mesa crmd: [27073]: info: ais_dispatch: Membership 80: quorum still lost Jan 25 20:49:11 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:11 black-mesa corosync[27062]: [TOTEM ] Delivering e to f Jan 25 20:49:11 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq f to pending delivery queue Jan 25 20:49:11 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including f Jan 25 20:49:15 black-mesa attrd: [27071]: debug: cib_connect: CIB signon attempt 1 Jan 25 20:49:15 black-mesa attrd: [27071]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw Jan 25 20:49:15 black-mesa attrd: [27071]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback Jan 25 20:49:15 black-mesa attrd: [27071]: debug: cib_native_signon_raw: Connection to CIB successful Jan 25 20:49:15 black-mesa attrd: [27071]: info: cib_connect: Connected to the CIB after 1 signon attempts Jan 25 20:49:15 black-mesa attrd: [27071]: info: cib_connect: Sending full refresh Jan 25 20:49:15 black-mesa cib: [27069]: debug: cib_common_callback_worker: Setting cib_refresh_notify callbacks for 27071 (6b0e04d3-4787-4eaa-a447-2e908ed4b8c2): on Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] entering GATHER state from 11. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Creating commit token because I am the rep. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Saving state aru f high seq received f Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Storing new sequence id for ring 58 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] entering COMMIT state. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] got commit token Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] entering RECOVERY state. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] position [0] member 10.7.0.1: Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] previous ring seq 80 rep 10.7.0.1 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] aru f high delivered f received flag 1 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] position [1] member 10.7.0.2: Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] previous ring seq 84 rep 10.7.0.2 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] aru e high delivered e received flag 1 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Did not need to originate any messages in recovery. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] got commit token Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Sending initial ORF token Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] install seq 0 aru 0 high seq received 0 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] install seq 0 aru 0 high seq received 0 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] install seq 0 aru 0 high seq received 0 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] install seq 0 aru 0 high seq received 0 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] retrans flag count 4 token aru 0 install seq 0 aru 0 0 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] recovery to regular 1-0 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering to app 10 to f Jan 25 20:49:17 black-mesa corosync[27062]: [CLM ] CLM CONFIGURATION CHANGE Jan 25 20:49:17 black-mesa corosync[27062]: [CLM ] New Configuration: Jan 25 20:49:17 black-mesa corosync[27062]: [CLM ] ^Ir(0) ip(10.7.0.1) Jan 25 20:49:17 black-mesa corosync[27062]: [CLM ] Members Left: Jan 25 20:49:17 black-mesa corosync[27062]: [CLM ] Members Joined: Jan 25 20:49:17 black-mesa cib: [27069]: notice: ais_dispatch: Membership 88: quorum acquired Jan 25 20:49:17 black-mesa crmd: [27073]: notice: ais_dispatch: Membership 88: quorum acquired Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Evt conf change 1 Jan 25 20:49:17 black-mesa cib: [27069]: debug: crm_new_peer: Creating entry for node (null)/33556234 Jan 25 20:49:17 black-mesa crmd: [27073]: debug: crm_new_peer: Creating entry for node (null)/33556234 Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] m 1, j 0 l 0 Jan 25 20:49:17 black-mesa cib: [27069]: info: crm_new_peer: Node now has id: 33556234 Jan 25 20:49:17 black-mesa crmd: [27073]: info: crm_new_peer: Node now has id: 33556234 Jan 25 20:49:17 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_confchg_fn Jan 25 20:49:17 black-mesa cib: [27069]: info: crm_update_peer: Node (null): id=33556234 state=member (new) addr=r(0) ip(10.7.0.2) votes=0 born=0 seen=88 proc=00000000000000000000000000000000 Jan 25 20:49:17 black-mesa crmd: [27073]: info: crm_update_peer: Node (null): id=33556234 state=member (new) addr=r(0) ip(10.7.0.2) votes=0 born=0 seen=88 proc=00000000000000000000000000000000 Jan 25 20:49:17 black-mesa corosync[27062]: [MSG ] [DEBUG]: msg_confchg_fn Jan 25 20:49:17 black-mesa cib: [27069]: info: ais_dispatch: Membership 88: quorum retained Jan 25 20:49:17 black-mesa crmd: [27073]: debug: post_cache_update: Updated cache after membership event 88. Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 88: memb=1, new=0, lost=0 Jan 25 20:49:17 black-mesa cib: [27069]: info: crm_get_peer: Node 33556234 is now known as torchwood-institute.mit.edu Jan 25 20:49:17 black-mesa crmd: [27073]: debug: post_cache_update: post_cache_update added action A_ELECTION_CHECK to the FSA Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] info: pcmk_peer_update: memb: black-mesa.mit.edu 16779018 Jan 25 20:49:17 black-mesa cib: [27069]: info: crm_update_peer: Node torchwood-institute.mit.edu: id=33556234 state=member addr=r(0) ip(10.7.0.2) votes=1 (new) born=88 seen=88 proc=00000000000000000000000000013312 (new) Jan 25 20:49:17 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_ELECTION_CHECK Jan 25 20:49:17 black-mesa corosync[27062]: [CLM ] CLM CONFIGURATION CHANGE Jan 25 20:49:17 black-mesa crmd: [27073]: debug: do_election_check: Ignore election check: we not in an election Jan 25 20:49:17 black-mesa corosync[27062]: [CLM ] New Configuration: Jan 25 20:49:17 black-mesa crmd: [27073]: info: ais_dispatch: Membership 88: quorum retained Jan 25 20:49:17 black-mesa corosync[27062]: [CLM ] ^Ir(0) ip(10.7.0.1) Jan 25 20:49:17 black-mesa crmd: [27073]: info: crm_get_peer: Node 33556234 is now known as torchwood-institute.mit.edu Jan 25 20:49:17 black-mesa corosync[27062]: [CLM ] ^Ir(0) ip(10.7.0.2) Jan 25 20:49:17 black-mesa crmd: [27073]: info: crm_update_peer: Node torchwood-institute.mit.edu: id=33556234 state=member addr=r(0) ip(10.7.0.2) votes=1 (new) born=88 seen=88 proc=00000000000000000000000000013312 (new) Jan 25 20:49:17 black-mesa corosync[27062]: [CLM ] Members Left: Jan 25 20:49:17 black-mesa crmd: [27073]: debug: post_cache_update: Updated cache after membership event 88. Jan 25 20:49:17 black-mesa corosync[27062]: [CLM ] Members Joined: Jan 25 20:49:17 black-mesa crmd: [27073]: debug: post_cache_update: post_cache_update added action A_ELECTION_CHECK to the FSA Jan 25 20:49:17 black-mesa corosync[27062]: [CLM ] ^Ir(0) ip(10.7.0.2) Jan 25 20:49:17 black-mesa crmd: [27073]: debug: do_fsa_action: actions:trace: ^I// A_ELECTION_CHECK Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Evt conf change 0 Jan 25 20:49:17 black-mesa crmd: [27073]: debug: do_election_check: Ignore election check: we not in an election Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] m 2, j 1 l 0 Jan 25 20:49:17 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_confchg_fn Jan 25 20:49:17 black-mesa corosync[27062]: [MSG ] [DEBUG]: msg_confchg_fn Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 88: memb=2, new=1, lost=0 Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] info: update_member: Creating entry for node 33556234 born on 88 Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] info: update_member: Node 33556234/unknown is now: member Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] info: pcmk_peer_update: NEW: .pending. 33556234 Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] debug: pcmk_peer_update: Node 33556234 has address r(0) ip(10.7.0.2) Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] info: pcmk_peer_update: MEMB: black-mesa.mit.edu 16779018 Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] info: pcmk_peer_update: MEMB: .pending. 33556234 Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] debug: pcmk_peer_update: 1 nodes changed Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] info: send_member_notification: Sending membership update 88 to 2 children Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] debug: send_cluster_id: Local update: id=16779018, born=88, seq=88 Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] info: update_member: 0x64a870 Node 16779018 ((null)) born on: 88 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] This node is within the primary component and will provide service. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] entering OPERATIONAL state. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] A processor joined or left the membership and a new membership was formed. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 1 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 0 to 1 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 1 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] debug: pcmk_cluster_id_callback: Node update: torchwood-institute.mit.edu (1.0.6) Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] info: update_member: 0x63e060 Node 33556234 (torchwood-institute.mit.edu) born on: 88 Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] info: update_member: 0x63e060 Node 33556234 now known as torchwood-institute.mit.edu (was: (null)) Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] info: update_member: Node torchwood-institute.mit.edu now has process list: 00000000000000000000000000013312 (78610) Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] info: update_member: Node torchwood-institute.mit.edu now has 1 quorum votes (was 0) Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] info: send_member_notification: Sending membership update 88 to 2 children Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] confchg entries 2 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 33556234 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 0. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 33556234 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 1 to 2 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 2 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [pcmk ] debug: pcmk_cluster_id_callback: Node update: black-mesa.mit.edu (1.0.6) Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] confchg entries 2 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 16779018 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 33556234 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Synchronization barrier completed Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Synchronization actions starting for (openais cluster membership service B.01.01) Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 2 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 1 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 2 to 3 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 3 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [CLM ] got nodejoin message 10.7.0.1 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 3 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 4 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 3 to 4 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 4 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [CLM ] got nodejoin message 10.7.0.2 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 2 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 4 to 5 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 5 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] confchg entries 2 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 16779018 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 33556234 = 0. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 5 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 6 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 5 to 6 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 6 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] confchg entries 2 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 33556234 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 33556234 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Synchronization barrier completed Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Committing synchronization for (openais cluster membership service B.01.01) Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Synchronization actions starting for (dummy AMF service) Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 4 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 6 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 7 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 6 to 7 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 7 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] confchg entries 2 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 33556234 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 0. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 33556234 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 7 to 8 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 8 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] confchg entries 2 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 16779018 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 33556234 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Synchronization barrier completed Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Committing synchronization for (dummy AMF service) Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Synchronization actions starting for (openais checkpoint service B.01.01) Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 8 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 7 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 8 to 9 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 9 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 9 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq a Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 9 to a Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq a to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 8 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering a to b Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq b to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq b Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including a Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including b Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering b to c Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq c to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] confchg entries 2 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 16779018 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 33556234 = 0. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq c Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq d Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering c to d Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq d to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] confchg entries 2 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 33556234 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 33556234 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Synchronization barrier completed Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Committing synchronization for (openais checkpoint service B.01.01) Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Synchronization actions starting for (openais event service B.01.01) Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Evt synchronize initialization Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Process Evt synchronization Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Send max event ID updates Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Send set evt ID 0 to r(0) ip(10.7.0.1) Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq e Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering d to e Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq e to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Remote channel operation request Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] my node ID: 0x100070a Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Received Set event ID OP from nodeid 200070a to 0 for 200070a my addr r(0) ip(10.7.0.1) base 1 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including d Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering e to f Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq f to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Remote channel operation request Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] my node ID: 0x100070a Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Received Set event ID OP from nodeid 100070a to 0 for 100070a my addr r(0) ip(10.7.0.1) base 1 Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Process Evt synchronization Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Send open count updates Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] DONE Sending open counts Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq f Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 10 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering f to 10 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 10 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Remote channel operation request Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] my node ID: 0x100070a Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Receive EVT_CONF_CHANGE_DONE from nodeid r(0) ip(10.7.0.2) members 2 checked in 1 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including e Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 10 to 11 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 11 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Remote channel operation request Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] my node ID: 0x100070a Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Receive EVT_CONF_CHANGE_DONE from nodeid r(0) ip(10.7.0.1) members 2 checked in 2 Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] I am oldest in my transitional config Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Process Evt synchronization Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Send retained event updates Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 11 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 10 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 11 to 12 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 12 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Process Evt synchronization Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] DONE Sending retained events Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 12 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 11 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 12 to 13 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 13 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Remote channel operation request Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] my node ID: 0x100070a Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Receive EVT_CONF_DONE from nodeid r(0) ip(10.7.0.1) , members 2 checked in 1 Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Process Evt synchronization Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Wait for retained events Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 13 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 14 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 13 to 14 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 14 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Remote channel operation request Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] my node ID: 0x100070a Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Receive EVT_CONF_DONE from nodeid r(0) ip(10.7.0.2) , members 2 checked in 2 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 12 Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Process Evt synchronization Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Recovery complete Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 14 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 15 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 14 to 15 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 15 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] confchg entries 2 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 33556234 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 0. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 33556234 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 15 to 16 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 16 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] confchg entries 2 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 16779018 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 33556234 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Synchronization barrier completed Jan 25 20:49:17 black-mesa corosync[27062]: [EVT ] Evt synchronize activation Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Committing synchronization for (openais event service B.01.01) Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Synchronization actions starting for (corosync cluster closed process group service v1.01) Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 16 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 15 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 16 to 18 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 17 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 18 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [CPG ] downlist left_list: 0 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 17 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 18 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 19 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 18 to 19 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 19 to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 1a Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 19 to 1a Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 1a to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [CPG ] downlist left_list: 0 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 16 Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 1a to 1b Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 1b to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] confchg entries 2 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 16779018 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 33556234 = 0. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 1b Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 1c Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 1b to 1c Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 1c to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] confchg entries 2 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier Start Received From 33556234 Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 16779018 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Barrier completion status for nodeid 33556234 = 1. Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Synchronization barrier completed Jan 25 20:49:17 black-mesa corosync[27062]: [SYNC ] Committing synchronization for (corosync cluster closed process group service v1.01) Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 1a Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 1c to 1d Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 1d to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 1d Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 1e Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering 1d to 1e Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 1e to pending delivery queue Jan 25 20:49:17 black-mesa corosync[27062]: [MAIN ] Completed service synchronization, ready to provide service. Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 1c Jan 25 20:49:17 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 1e Jan 25 20:49:18 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 1f Jan 25 20:49:18 black-mesa corosync[27062]: [TOTEM ] Delivering 1e to 1f Jan 25 20:49:18 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 1f to pending delivery queue Jan 25 20:49:18 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 20 Jan 25 20:49:18 black-mesa corosync[27062]: [TOTEM ] Delivering 1f to 20 Jan 25 20:49:18 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 20 to pending delivery queue Jan 25 20:49:18 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 1f Jan 25 20:49:18 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 21 Jan 25 20:49:18 black-mesa corosync[27062]: [TOTEM ] Delivering 20 to 21 Jan 25 20:49:18 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 21 to pending delivery queue Jan 25 20:49:18 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 20 Jan 25 20:49:18 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 21 Jan 25 20:49:24 black-mesa corosync[27062]: [CONFDB] lib_init_fn: conn=0x653dd0 Jan 25 20:49:24 black-mesa corosync[27062]: [CONFDB] exit_fn for conn=0x653dd0 Jan 25 20:49:24 black-mesa corosync[27062]: [CPG ] lib_init_fn: conn=0x653dd0, cpd=0x6539b0 Jan 25 20:49:24 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_init_fn Jan 25 20:49:24 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:24 black-mesa corosync[27062]: [TOTEM ] Delivering 21 to 22 Jan 25 20:49:24 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 22 to pending delivery queue Jan 25 20:49:24 black-mesa corosync[27062]: [CPG ] got procjoin message from cluster node 16779018 Jan 25 20:49:24 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 22 Jan 25 20:49:24 black-mesa clvmd: Cluster LVM daemon started - connected to OpenAIS Jan 25 20:49:24 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 22 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 22 to 23 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 23 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 23 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_init_fn Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 23 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 23 to 24 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 24 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 24 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_exit_fn Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 24 Jan 25 20:49:25 black-mesa corosync[27062]: [CPG ] got mcast request on 0x653dd0 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 24 to 25 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 25 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 25 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 25 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 25 to 26 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 26 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 26 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_init_fn Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 26 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 26 to 27 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 27 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 27 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_exit_fn Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 27 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceUnlock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 27 to 28 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 28 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceUnlock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 28 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceClose Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 28 to 29 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 29 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceClose Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_resourcelock_release { name=V_xenvg-1 } [0] Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 29 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 28 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 29 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 29 to 2a Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 2a to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 2a Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_init_fn Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 2a Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 2a to 2b Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 2b to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 2b Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_exit_fn Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 2b Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceUnlock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 2b to 2c Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 2c to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceUnlock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 2c Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceClose Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 2c to 2d Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 2d to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceClose Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_resourcelock_release { name=V_black-mesa-1 } [0] Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 2d Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 2c Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceUnlock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 2d Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 2d to 2e Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 2e to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceUnlock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 2e Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceClose Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 2e to 2f Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 2f to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceClose Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_resourcelock_release { name=P_#global-2 } [0] Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 2f Jan 25 20:49:25 black-mesa corosync[27062]: [CPG ] got mcast request on 0x653dd0 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 2e Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 2f to 30 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 30 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 30 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 2f Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 30 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 30 to 31 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 31 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 31 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 31 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_init_fn Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 31 to 32 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 32 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 32 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_exit_fn Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 32 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 32 to 33 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 33 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 33 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_init_fn Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 33 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 33 to 34 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 34 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 34 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_exit_fn Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 34 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 34 to 35 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 35 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 35 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 35 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_init_fn Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 35 to 36 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 36 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 36 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_exit_fn Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 36 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 36 to 37 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 37 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 37 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_init_fn Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 37 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 37 to 38 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 38 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 38 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_exit_fn Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 38 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceUnlock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 38 to 39 Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 39 to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceUnlock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 39 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceClose Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 39 to 3a Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 3a to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceClose Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_resourcelock_release { name=V_xenvg-1 } [0] Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 3a Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 39 Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 3a Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 3a to 3b Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 3b to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 3b Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 3b Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_init_fn Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 3b to 3c Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 3c to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 3c Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_exit_fn Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 3c Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceUnlock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 3c to 3d Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 3d to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceUnlock Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 3d Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceClose Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering 3d to 3e Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 3e to pending delivery queue Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceClose Jan 25 20:49:25 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_resourcelock_release { name=V_black-mesa-1 } [0] Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 3e Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 3d Jan 25 20:49:25 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 3e Jan 25 20:49:27 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 3f Jan 25 20:49:27 black-mesa corosync[27062]: [TOTEM ] Delivering 3e to 3f Jan 25 20:49:27 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 3f to pending delivery queue Jan 25 20:49:27 black-mesa corosync[27062]: [CPG ] got procjoin message from cluster node 33556234 Jan 25 20:49:27 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 3f Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 40 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 3f to 40 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 40 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 40 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 41 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 40 to 41 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 41 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 41 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 42 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 41 to 42 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 42 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 42 Jan 25 20:49:28 black-mesa corosync[27062]: [CPG ] got mcast request on 0x653dd0 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 42 to 43 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 43 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 43 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 43 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 44 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 43 to 44 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 44 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 44 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 45 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 44 to 45 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 45 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 45 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 46 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 45 to 46 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 46 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceUnlock Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 47 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 46 to 47 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 47 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceClose Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_resourcelock_release { name=V_xenvg-1 } [0] Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 46 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 47 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 48 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 47 to 48 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 48 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 48 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 49 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 48 to 49 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 49 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 49 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 4a Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 49 to 4a Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 4a to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceUnlock Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 4b Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 4a to 4b Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 4b to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceClose Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_resourcelock_release { name=V_torchwood-institute-1 } [0] Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 4a Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 4b Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 4c Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 4b to 4c Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 4c to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceUnlock Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 4d Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 4c to 4d Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 4d to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceClose Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_resourcelock_release { name=P_#global-2 } [0] Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 4c Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 4e Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 4d to 4e Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 4e to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 4d Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 4e Jan 25 20:49:28 black-mesa corosync[27062]: [CPG ] got mcast request on 0x653dd0 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 4e to 4f Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 4f to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 4f Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 4f Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 50 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 4f to 50 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 50 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 50 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 51 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 50 to 51 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 51 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 51 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 52 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 51 to 52 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 52 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 52 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 53 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 52 to 53 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 53 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 53 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 54 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 53 to 54 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 54 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 54 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 55 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 54 to 55 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 55 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 55 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 56 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 55 to 56 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 56 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 56 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 57 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 56 to 57 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 57 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 57 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 58 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 57 to 58 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 58 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceUnlock Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 59 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 58 to 59 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 59 to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceClose Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_resourcelock_release { name=V_xenvg-1 } [0] Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 58 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 59 Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 5a Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 59 to 5a Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 5a to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 5a Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 5b Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 5a to 5b Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 5b to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 5b Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 5c Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 5b to 5c Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 5c to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceUnlock Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 5d Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering 5c to 5d Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 5d to pending delivery queue Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceClose Jan 25 20:49:28 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_resourcelock_release { name=V_torchwood-institute-1 } [0] Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 5c Jan 25 20:49:28 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 5d Jan 25 20:49:33 black-mesa corosync[27062]: [CPG ] got mcast request on 0x653dd0 Jan 25 20:49:33 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:33 black-mesa corosync[27062]: [TOTEM ] Delivering 5d to 5e Jan 25 20:49:33 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 5e to pending delivery queue Jan 25 20:49:33 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 5e Jan 25 20:49:33 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 5e Jan 25 20:49:33 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 5f Jan 25 20:49:33 black-mesa corosync[27062]: [TOTEM ] Delivering 5e to 5f Jan 25 20:49:33 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 5f to pending delivery queue Jan 25 20:49:33 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 5f Jan 25 20:49:42 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceOpen Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] Delivering 5f to 60 Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 60 to pending delivery queue Jan 25 20:49:42 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 60 Jan 25 20:49:42 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_init_fn Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 60 Jan 25 20:49:42 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceLock Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] Delivering 60 to 61 Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 61 to pending delivery queue Jan 25 20:49:42 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 61 Jan 25 20:49:42 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_exit_fn Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 61 Jan 25 20:49:42 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceOpen Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] Delivering 61 to 62 Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 62 to pending delivery queue Jan 25 20:49:42 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceOpen Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] Received ringid(10.7.0.1:88) seq 62 Jan 25 20:49:42 black-mesa corosync[27062]: [LCK ] [DEBUG]: lck_lib_init_fn Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] releasing messages up to and including 62 Jan 25 20:49:42 black-mesa corosync[27062]: [LCK ] LIB request: saLckResourceLock Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] mcasted message added to pending queue Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] Delivering 62 to 63 Jan 25 20:49:42 black-mesa corosync[27062]: [TOTEM ] Delivering MCAST message with seq 63 to pending delivery queue Jan 25 20:49:42 black-mesa corosync[27062]: [LCK ] EXEC request: saLckResourceLock