Jul 23 16:12:34 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-attrd [121631] (pcmk_cpg_membership) info: Group attrd event 3: node 2 pid 3296858 joined via cluster join Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-attrd [121631] (pcmk_cpg_membership) info: Group attrd event 3: an-a02n01 (node 1 pid 121631) is member Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (pcmk_cpg_membership) info: Group crmd event 3: node 2 pid 3296860 joined via cluster join Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (pcmk_cpg_membership) info: Group crmd event 3: an-a02n01 (node 1 pid 121633) is member Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (pcmk_cpg_membership) info: Group crmd event 3: an-a02n02 (node 2 pid 3296860) is member Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (crm_update_peer_proc) info: pcmk_cpg_membership: Node an-a02n02[2] - corosync-cpg is now online Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (peer_update_callback) info: Node an-a02n02 is now a peer | DC=true old=0x0000000 new=0x4000000 Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-fenced [121629] (pcmk_cpg_membership) info: Group stonith-ng event 3: node 2 pid 3296856 joined via cluster join Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-fenced [121629] (pcmk_cpg_membership) info: Group stonith-ng event 3: an-a02n01 (node 1 pid 121629) is member Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (pcmk_cpg_membership) info: Group cib event 3: node 2 pid 3296855 joined via cluster join Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (pcmk_cpg_membership) info: Group cib event 3: an-a02n01 (node 1 pid 121628) is member Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-attrd [121631] (crm_get_peer) info: Created entry c3ff630f-ae7a-49a5-af89-7aa8d4270273/0x559792685c00 for node an-a02n02/2 (2 total) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-attrd [121631] (crm_get_peer) info: Node 2 is now known as an-a02n02 Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-fenced [121629] (crm_get_peer) info: Created entry 301efa9f-ab21-4758-b392-9cb2155af993/0x564e3948de00 for node an-a02n02/2 (2 total) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-attrd [121631] (crm_get_peer) info: Node 2 has uuid 2 Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-attrd [121631] (pcmk_cpg_membership) info: Group attrd event 3: an-a02n02 (node 2 pid 3296858) is member Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-fenced [121629] (crm_get_peer) info: Node 2 is now known as an-a02n02 Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-attrd [121631] (crm_update_peer_proc) info: pcmk_cpg_membership: Node an-a02n02[2] - corosync-cpg is now online Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-attrd [121631] (update_peer_state_iter) notice: Node an-a02n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-fenced [121629] (crm_cs_flush) info: Sent 0 CPG messages (1 remaining): Try again (6) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-fenced [121629] (crm_get_peer) info: Node 2 has uuid 2 Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-fenced [121629] (pcmk_cpg_membership) info: Group stonith-ng event 3: an-a02n02 (node 2 pid 3296856) is member Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-fenced [121629] (crm_update_peer_proc) info: pcmk_cpg_membership: Node an-a02n02[2] - corosync-cpg is now online Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-fenced [121629] (update_peer_state_iter) notice: Node an-a02n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-attrd [121631] (crm_cs_flush) info: Sent 0 CPG messages (1 remaining): Try again (6) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (quorum_notification_cb) info: Quorum retained | membership=259 members=2 Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (update_peer_state_iter) notice: Node an-a02n02 state is now member | nodeid=2 previous=lost source=quorum_notification_cb Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (peer_update_callback) info: Cluster node an-a02n02 is now member (was lost) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) notice: State transition S_IDLE -> S_INTEGRATION | input=I_NODE_JOIN cause=C_FSA_INTERNAL origin=peer_update_callback Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_join_offer_one) info: Making join-3 offers to any unconfirmed nodes because an unknown node joined Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Making join-3 offers based on membership event 259 Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-3 offer to an-a02n02 Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Not making join-3 offer to already known node an-a02n01 (confirmed) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 100 aborted: Peer Halt | source=do_te_invoke:136 complete=true Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (crm_get_peer) info: Created entry b90d6436-ebda-4818-9e86-f7a67e348035/0x5619760a23b0 for node an-a02n02/2 (2 total) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (crm_get_peer) info: Node 2 is now known as an-a02n02 Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (crm_get_peer) info: Node 2 has uuid 2 Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (pcmk_cpg_membership) info: Group cib event 3: an-a02n02 (node 2 pid 3296855) is member Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (crm_update_peer_proc) info: pcmk_cpg_membership: Node an-a02n02[2] - corosync-cpg is now online Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (update_peer_state_iter) notice: Node an-a02n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/305) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/306) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/309) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/310) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.28 2 Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.29 (null) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=29 Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crmd=online, @crm-debug-origin=peer_update_callback Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/305, version=0.337.29) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/306, version=0.337.29) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/309, version=0.337.29) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.29 2 Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.30 (null) Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=30 Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=post_cache_update Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @in_ccm=true, @crm-debug-origin=post_cache_update Jul 23 16:13:11 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/310, version=0.337.30) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (pcmk__update_peer_expected) info: do_dc_join_filter_offer: Node an-a02n02[2] - expected state is now member (was down) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_INTEGRATION -> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_join_finalize) notice: Finalizing join-3 for 1 node (sync'ing CIB from an-a02n02) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_join_finalize) notice: Requested CIB version Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_sync operation for section 'all' to an-a02n02 (origin=local/crmd/311) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Digest matched on replace from an-a02n02: faa77275e1e8c408f4838d2d2dcdc91c Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Replacement 0.337.2 from an-a02n02 not applied to 0.337.30: current num_updates is greater than the replacement Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (finalize_sync_callback) warning: Could not sync CIB from an-a02n02 in join-3: Update was older than existing configuration Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (register_fsa_error_adv) info: Resetting the current action list Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (do_log) warning: Input I_ELECTION_DC received in state S_FINALIZE_JOIN from finalize_sync_callback Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_FINALIZE_JOIN -> S_INTEGRATION | input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=finalize_sync_callback Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Unset DC. Was an-a02n01 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-4 offer to an-a02n02 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-4 offer to an-a02n01 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_join_offer_all) info: Waiting on join-4 requests from 2 outstanding nodes Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_check_for_new_writer) notice: Detected another attribute writer (an-a02n02), starting new election Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (process_peer_sync_response) info: Processing sync-response from an-a02n02 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting #attrd-protocol[an-a02n02]: (unset) -> 3 | from an-a02n02 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (crm_cs_flush) info: Sent 3 CPG messages (0 remaining): OK (1) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Set DC to an-a02n01 (3.10.2) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-fenced [121629] (crm_cs_flush) info: Sent 2 CPG messages (0 remaining): OK (1) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (election_count_vote) info: election-attrd round 2 (owner node ID 2) pass: vote from an-a02n02 (Uptime) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (process_peer_sync_response) info: Processing sync-response from an-a02n02 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_INTEGRATION -> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (election_check) info: election-attrd won by local node Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_declare_winner) notice: Recorded local node as attribute writer (was unset) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 14 with 1 change for last-failure-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 15 with 1 change for fail-count-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Processed 2 private changes for #attrd-protocol, id=n/a, set=n/a Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 16 with 1 change for fail-count-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 17 with 1 change for last-failure-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 18 with 1 change for fail-count-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 19 with 1 change for last-failure-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 20 with 1 change for fail-count-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 21 with 1 change for last-failure-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Digest matched on replace from an-a02n01: 5c0ae7a81d266484cd53effd22475581 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Replaced 0.337.30 with 0.337.30 from an-a02n01 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: cib_perform_op: Local-only Change: 0.337.30 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: +~ /cib/status/node_state[@id='1']/lrm[@id='1'] moved to offset 1 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=post_cache_update Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_replace_notify) info: Local-only Replace: 0.337.30 from an-a02n01 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=an-a02n01/crmd/313, version=0.337.30) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/314) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/315) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/14) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/15) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/16) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-fenced [121629] (xml_patch_version_check) notice: Versions did not change in patch 0.337.30 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/17) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/18) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_replaced_cb) notice: Updating all attributes after cib_refresh_notify event Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Write out of 'last-failure-apc_snmp_node2_an-pdu03#start_0' delayed: update 14 in progress Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Write out of 'fail-count-apc_snmp_node2_an-pdu03#start_0' delayed: update 15 in progress Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Processed 2 private changes for #attrd-protocol, id=n/a, set=n/a Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Write out of 'fail-count-apc_snmp_node1_an-pdu03#start_0' delayed: update 16 in progress Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Write out of 'last-failure-apc_snmp_node1_an-pdu03#start_0' delayed: update 17 in progress Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Write out of 'fail-count-apc_snmp_node2_an-pdu04#start_0' delayed: update 18 in progress Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Write out of 'last-failure-apc_snmp_node2_an-pdu04#start_0' delayed: update 19 in progress Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Write out of 'fail-count-apc_snmp_node1_an-pdu04#start_0' delayed: update 20 in progress Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/19) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Write out of 'last-failure-apc_snmp_node1_an-pdu04#start_0' delayed: update 21 in progress Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/20) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/21) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/314, version=0.337.30) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/315, version=0.337.30) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_delay_metadata_1[4128871] exited with status 0 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/14, version=0.337.30) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 14 result for last-failure-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=1627003019 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/15, version=0.337.30) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 15 result for fail-count-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=INFINITY Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/16, version=0.337.30) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 16 result for fail-count-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=INFINITY Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/17, version=0.337.30) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 17 result for last-failure-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=1627003020 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/18, version=0.337.30) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 18 result for fail-count-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=INFINITY Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/19, version=0.337.30) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 19 result for last-failure-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=1627003034 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/20, version=0.337.30) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 20 result for fail-count-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=INFINITY Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/21, version=0.337.30) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 21 result for last-failure-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=1627003035 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-24.raw Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.337.0 of the CIB to disk (digest: 1636edea2c3c05c32ce58a90fcd16dcd) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.OJOoxc (digest: /var/lib/pacemaker/cib/cib.uryeTa) Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_delay_metadata_1[4128872] exited with status 0 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4128874] exited with status 0 Jul 23 16:13:12 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4128911] exited with status 0 Jul 23 16:13:13 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4128917] exited with status 0 Jul 23 16:13:13 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4128935] exited with status 0 Jul 23 16:13:13 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4128937] exited with status 0 Jul 23 16:13:13 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4128949] exited with status 0 Jul 23 16:13:13 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4129089] exited with status 0 Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4129227] exited with status 0 Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_FINALIZE_JOIN -> S_ELECTION | input=I_ELECTION cause=C_FSA_INTERNAL origin=do_cib_replaced Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Unset DC. Was an-a02n01 Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/318) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (do_log) info: Input I_JOIN_RESULT received in state S_ELECTION from route_message Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/319) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (do_log) info: Input I_JOIN_RESULT received in state S_ELECTION from route_message Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (election_check) info: election-DC won by local node Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (do_log) info: Input I_ELECTION_DC received in state S_ELECTION from election_win_cb Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) notice: State transition S_ELECTION -> S_INTEGRATION | input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=election_win_cb Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/318, version=0.337.30) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.30 2 Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.31 (null) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=31 Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_cib_replaced Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_cib_replaced, @join=member, @expected=member Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/319, version=0.337.31) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_takeover) info: Taking over DC status for this partition Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_master operation for section 'all': OK (rc=0, origin=local/crmd/320, version=0.337.31) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/321) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/323) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/325) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/327) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-5 offer to an-a02n02 Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-5 offer to an-a02n01 Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_join_offer_all) info: Waiting on join-5 requests from 2 outstanding nodes Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Set DC to an-a02n01 (3.10.2) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=an-a02n01/crmd/321, version=0.337.31) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/323, version=0.337.31) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/325, version=0.337.31) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/327, version=0.337.31) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_INTEGRATION -> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Digest matched on replace from an-a02n01: 37a1008549014c5b03f3859ebd7e67d9 Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Replaced 0.337.31 with 0.337.31 from an-a02n01 Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=an-a02n01/crmd/330, version=0.337.31) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/331) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/332) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-25.raw Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/331, version=0.337.31) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.337.0 of the CIB to disk (digest: 513fd971e2a46a594a89277e208e4395) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.7UP1Ni (digest: /var/lib/pacemaker/cib/cib.hAzBMk) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_delay_metadata_1[4129388] exited with status 0 Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/332, version=0.337.31) Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_delay_metadata_1[4129392] exited with status 0 Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4129395] exited with status 0 Jul 23 16:13:14 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4129406] exited with status 0 Jul 23 16:13:15 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4129412] exited with status 0 Jul 23 16:13:15 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4129422] exited with status 0 Jul 23 16:13:15 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4129425] exited with status 0 Jul 23 16:13:15 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4129431] exited with status 0 Jul 23 16:13:15 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4129433] exited with status 0 Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4129434] exited with status 0 Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-controld [121633] (controld_delete_node_state) info: Deleting resource history for node an-a02n02 (via CIB call 333) | xpath=//node_state[@uname='an-a02n02']/lrm Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='an-a02n02']/lrm to all (origin=local/crmd/333) Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/334) Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-controld [121633] (controld_delete_node_state) info: Deleting resource history for node an-a02n01 (via CIB call 335) | xpath=//node_state[@uname='an-a02n01']/lrm Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.31 2 Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.32 (null) Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2'] Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=32 Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n02']/lrm: OK (rc=0, origin=an-a02n01/crmd/333, version=0.337.32) Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.32 2 Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.33 (null) Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=33 Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_lrm_query_internal Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/334, version=0.337.33) Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='an-a02n01']/lrm to all (origin=local/crmd/335) Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.33 2 Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.34 (null) Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/lrm[@id='1'] Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=34 Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n01']/lrm: OK (rc=0, origin=an-a02n01/crmd/335, version=0.337.34) Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_delay_metadata_1[4129435] exited with status 0 Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_delay_metadata_1[4129436] exited with status 0 Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4129437] exited with status 0 Jul 23 16:13:16 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4129450] exited with status 0 Jul 23 16:13:17 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4129454] exited with status 0 Jul 23 16:13:17 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4129457] exited with status 0 Jul 23 16:13:17 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4129459] exited with status 0 Jul 23 16:13:17 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4129466] exited with status 0 Jul 23 16:13:17 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4129468] exited with status 0 Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4129476] exited with status 0 Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/336) Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE | input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 100 aborted: Peer Cancelled | source=do_te_invoke:129 complete=true Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/339) Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/340) Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/341) Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.34 2 Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.35 (null) Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=35 Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_lrm_query_internal Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/336, version=0.337.35) Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/339, version=0.337.35) Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.35 2 Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.36 (null) Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=36 Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/340, version=0.337.36) Jul 23 16:13:18 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=an-a02n01/crmd/341, version=0.337.36) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 100 aborted: History refresh | source=process_resource_updates:268 complete=true Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 22 18:16:47 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu03 on an-a02n01 at Jul 22 18:16:47 2021 | rc=1 id=apc_snmp_node2_an-pdu03_last_failure_0 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu03 has failed INFINITY times on an-a02n01 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu03 has failed INFINITY times on an-a02n01 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu03 on an-a02n02 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu04 on an-a02n02 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node2_an-pdu03 on an-a02n02 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node2_an-pdu04 on an-a02n02 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start apc_snmp_node1_an-pdu03 ( an-a02n02 ) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start apc_snmp_node1_an-pdu04 ( an-a02n02 ) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start apc_snmp_node2_an-pdu03 ( an-a02n02 ) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start apc_snmp_node2_an-pdu04 ( an-a02n02 ) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node2 (Started an-a02n01) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node2 (Started an-a02n01) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n01) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n01) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 101, saving inputs in /var/lib/pacemaker/pengine/pe-input-122.bz2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (do_te_invoke) info: Processing graph 101 (ref=pe_calc-dc-1627081999-234) derived from /var/lib/pacemaker/pengine/pe-input-122.bz2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation apc_snmp_node1_an-pdu03_monitor_0 on an-a02n02 | action 11 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation apc_snmp_node1_an-pdu04_monitor_0 on an-a02n02 | action 12 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation apc_snmp_node2_an-pdu03_monitor_0 on an-a02n02 | action 13 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation apc_snmp_node2_an-pdu04_monitor_0 on an-a02n02 | action 14 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation ipmilan_node2_monitor_0 on an-a02n02 | action 15 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation delay_node2_monitor_0 on an-a02n02 | action 16 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation ipmilan_node1_monitor_0 on an-a02n02 | action 17 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation delay_node1_monitor_0 on an-a02n02 | action 18 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation srv13-psql_monitor_0 on an-a02n02 | action 19 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation srv14-file-processor_monitor_0 on an-a02n02 | action 20 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation srv15-tickets_monitor_0 on an-a02n02 | action 21 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation srv16-machine-interface_monitor_0 on an-a02n02 | action 22 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation srv17-order-processing_monitor_0 on an-a02n02 | action 23 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation srv18-inventory_monitor_0 on an-a02n02 | action 24 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation srv19-uls_monitor_0 on an-a02n02 | action 25 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation srv20-fileprocessor_monitor_0 on an-a02n02 | action 26 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation srv21-tickets_monitor_0 on an-a02n02 | action 27 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation srv22-machineinterface_monitor_0 on an-a02n02 | action 28 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation srv23-orderprocessing_monitor_0 on an-a02n02 | action 29 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation srv24-inventory_monitor_0 on an-a02n02 | action 30 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.36 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.37 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=37 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/11, version=0.337.37) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.37 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.38 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=38 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/12, version=0.337.38) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.38 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.39 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=39 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @transition-magic=0:7;11:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=5, @rc-code=7, @op-status=0, @exec-time=1 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/13, version=0.337.39) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 11 (apc_snmp_node1_an-pdu03_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=5 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation apc_snmp_node1_an-pdu03_start_0 on an-a02n02 | action 31 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.39 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.40 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=40 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_0']: @transition-magic=0:7;12:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=9, @rc-code=7, @op-status=0 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/14, version=0.337.40) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 12 (apc_snmp_node1_an-pdu04_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=9 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation apc_snmp_node1_an-pdu04_start_0 on an-a02n02 | action 33 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.40 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.41 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=41 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/15, version=0.337.41) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.41 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.42 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=42 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_0']: @transition-magic=0:7;13:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=13, @rc-code=7, @op-status=0 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/16, version=0.337.42) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 13 (apc_snmp_node2_an-pdu03_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=13 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation apc_snmp_node2_an-pdu03_start_0 on an-a02n02 | action 35 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.42 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.43 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=43 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/17, version=0.337.43) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.43 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.44 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=44 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_0']: @transition-magic=0:7;14:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=17, @rc-code=7, @op-status=0 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/18, version=0.337.44) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 14 (apc_snmp_node2_an-pdu04_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=17 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation apc_snmp_node2_an-pdu04_start_0 on an-a02n02 | action 37 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.44 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.45 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=45 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/19, version=0.337.45) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.45 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.46 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=46 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='ipmilan_node2']/lrm_rsc_op[@id='ipmilan_node2_last_0']: @transition-magic=0:7;15:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=21, @rc-code=7, @op-status=0 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/20, version=0.337.46) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 15 (ipmilan_node2_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=21 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.46 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.47 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=47 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/23, version=0.337.49) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.49 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.50 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=50 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='ipmilan_node1']/lrm_rsc_op[@id='ipmilan_node1_last_0']: @transition-magic=0:7;17:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=29, @rc-code=7, @op-status=0 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/24, version=0.337.50) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 17 (ipmilan_node1_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=29 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.50 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.51 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=51 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/28, version=0.337.54) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.54 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.55 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=55 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/29, version=0.337.55) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.55 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.56 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=56 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/30, version=0.337.56) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.56 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.57 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=57 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/31, version=0.337.57) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.57 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.58 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=58 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/32, version=0.337.58) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.58 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.59 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=59 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/33, version=0.337.59) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.59 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.60 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=60 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/34, version=0.337.60) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.60 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.61 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=61 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/35, version=0.337.61) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.61 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.62 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=62 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/36, version=0.337.62) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.62 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.63 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=63 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/37, version=0.337.63) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.63 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.64 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=64 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/38, version=0.337.64) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.64 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.65 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=65 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @operation_key=apc_snmp_node1_an-pdu03_start_0, @operation=start, @transition-key=31:101:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;31:101:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/39, version=0.337.65) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.65 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.66 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=66 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_0']: @operation_key=apc_snmp_node1_an-pdu04_start_0, @operation=start, @transition-key=33:101:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;33:101:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/40, version=0.337.66) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.66 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.67 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=67 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_0']: @operation_key=apc_snmp_node2_an-pdu03_start_0, @operation=start, @transition-key=35:101:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;35:101:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/41, version=0.337.67) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.67 2 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.68 (null) Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=68 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_0']: @operation_key=apc_snmp_node2_an-pdu04_start_0, @operation=start, @transition-key=37:101:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;37:101:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1 Jul 23 16:13:19 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/42, version=0.337.68) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.68 2 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.69 (null) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=69 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='srv18-inventory']/lrm_rsc_op[@id='srv18-inventory_last_0']: @transition-magic=0:7;24:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=57, @rc-code=7, @op-status=0, @exec-time=377 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/43, version=0.337.69) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 24 (srv18-inventory_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=57 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.69 2 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.70 (null) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=70 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='srv16-machine-interface']/lrm_rsc_op[@id='srv16-machine-interface_last_0']: @transition-magic=0:7;22:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=49, @rc-code=7, @op-status=0, @exec-time=380 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/44, version=0.337.70) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 22 (srv16-machine-interface_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=49 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.70 2 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.71 (null) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=71 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='srv17-order-processing']/lrm_rsc_op[@id='srv17-order-processing_last_0']: @transition-magic=0:7;23:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=53, @rc-code=7, @op-status=0, @exec-time=381 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/45, version=0.337.71) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 23 (srv17-order-processing_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=53 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.71 2 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.72 (null) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=72 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='srv24-inventory']/lrm_rsc_op[@id='srv24-inventory_last_0']: @transition-magic=0:7;30:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=81, @rc-code=7, @op-status=0, @exec-time=379 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/46, version=0.337.72) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 30 (srv24-inventory_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=81 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.72 2 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.73 (null) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=73 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='srv22-machineinterface']/lrm_rsc_op[@id='srv22-machineinterface_last_0']: @transition-magic=0:7;28:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=73, @rc-code=7, @op-status=0, @exec-time=389 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/47, version=0.337.73) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 28 (srv22-machineinterface_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=73 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.73 2 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.74 (null) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=74 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='srv20-fileprocessor']/lrm_rsc_op[@id='srv20-fileprocessor_last_0']: @transition-magic=0:7;26:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=65, @rc-code=7, @op-status=0, @exec-time=391 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/48, version=0.337.74) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 26 (srv20-fileprocessor_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=65 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.74 2 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.75 (null) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=75 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='srv19-uls']/lrm_rsc_op[@id='srv19-uls_last_0']: @transition-magic=0:7;25:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=61, @rc-code=7, @op-status=0, @exec-time=393 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/49, version=0.337.75) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 25 (srv19-uls_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=61 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.75 2 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.76 (null) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=76 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='srv14-file-processor']/lrm_rsc_op[@id='srv14-file-processor_last_0']: @transition-magic=0:7;20:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=41, @rc-code=7, @op-status=0, @exec-time=398 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/50, version=0.337.76) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 20 (srv14-file-processor_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=41 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.76 2 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.77 (null) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=77 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='srv21-tickets']/lrm_rsc_op[@id='srv21-tickets_last_0']: @transition-magic=0:7;27:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=69, @rc-code=7, @op-status=0, @exec-time=396 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/51, version=0.337.77) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 27 (srv21-tickets_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=69 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.77 2 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.78 (null) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=78 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='srv13-psql']/lrm_rsc_op[@id='srv13-psql_last_0']: @transition-magic=0:7;19:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=37, @rc-code=7, @op-status=0, @exec-time=402 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/52, version=0.337.78) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 19 (srv13-psql_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=37 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.78 2 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.79 (null) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=79 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='srv15-tickets']/lrm_rsc_op[@id='srv15-tickets_last_0']: @transition-magic=0:7;21:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=45, @rc-code=7, @op-status=0, @exec-time=401 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/53, version=0.337.79) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 21 (srv15-tickets_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=45 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.79 2 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.80 (null) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=80 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='srv23-orderprocessing']/lrm_rsc_op[@id='srv23-orderprocessing_last_0']: @transition-magic=0:7;29:101:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=77, @rc-code=7, @op-status=0, @exec-time=396 Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/54, version=0.337.80) Jul 23 16:13:20 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 29 (srv23-orderprocessing_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=77 Jul 23 16:13:25 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_ping) info: Reporting our current digest to an-a02n01: e4cfea920e17146a260e162cc41143a4 for 0.337.80 (0x5619761712c0 0) Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.80 2 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.81 (null) Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=81 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_0']: @transition-magic=0:1;33:101:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=83, @rc-code=1, @op-status=0, @exec-time=12332 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']: Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/23, version=0.337.82) Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 23 result for fail-count-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=INFINITY Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=INFINITY Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) notice: Transition 101 aborted by transient_attributes.2 'create': Transient attribute change | cib=0.337.82 source=abort_unless_down:313 path=/cib/status/node_state[@id='2'] complete=false Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/24) Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.82 2 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.83 (null) Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=83 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/24, version=0.337.83) Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 24 result for last-failure-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=1627003035 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=1627082012 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 101 aborted by status-2-last-failure-apc_snmp_node1_an-pdu04.start_0 doing create last-failure-apc_snmp_node1_an-pdu04#start_0=1627082012: Transient attribute change | cib=0.337.83 source=abort_unless_down:313 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=false Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.83 2 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.84 (null) Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=84 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @transition-magic=0:1;31:101:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=82, @rc-code=1, @op-status=0, @exec-time=12345 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']: Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/25, version=0.337.85) Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 25 result for fail-count-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=INFINITY Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=INFINITY Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 101 aborted by status-2-fail-count-apc_snmp_node1_an-pdu03.start_0 doing create fail-count-apc_snmp_node1_an-pdu03#start_0=INFINITY: Transient attribute change | cib=0.337.85 source=abort_unless_down:313 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=false Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.85 2 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.86 (null) Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=86 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/26, version=0.337.86) Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 101 aborted by status-2-last-failure-apc_snmp_node1_an-pdu03.start_0 doing create last-failure-apc_snmp_node1_an-pdu03#start_0=1627082012: Transient attribute change | cib=0.337.86 source=abort_unless_down:313 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=false Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 26 result for last-failure-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=1627003020 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=1627082012 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.86 2 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.87 (null) Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=87 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_0']: @transition-magic=0:1;37:101:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=85, @rc-code=1, @op-status=0, @exec-time=12344 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']: Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/27, version=0.337.88) Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 27 result for fail-count-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=INFINITY Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=INFINITY Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 101 aborted by status-2-fail-count-apc_snmp_node2_an-pdu04.start_0 doing create fail-count-apc_snmp_node2_an-pdu04#start_0=INFINITY: Transient attribute change | cib=0.337.88 source=abort_unless_down:313 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=false Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.88 2 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.89 (null) Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=89 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/28, version=0.337.89) Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 28 result for last-failure-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=1627003034 Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 101 aborted by status-2-last-failure-apc_snmp_node2_an-pdu04.start_0 doing create last-failure-apc_snmp_node2_an-pdu04#start_0=1627082012: Transient attribute change | cib=0.337.89 source=abort_unless_down:313 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=false Jul 23 16:13:32 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=1627082012 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.89 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.90 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=90 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_0']: @transition-magic=0:1;35:101:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=84, @rc-code=1, @op-status=0, @exec-time=13351 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']: Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/29, version=0.337.91) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 29 result for fail-count-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=INFINITY Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=INFINITY Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.91 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.92 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=92 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/30, version=0.337.92) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 30 result for last-failure-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=1627003019 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=1627082013 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 101 aborted by status-2-fail-count-apc_snmp_node2_an-pdu03.start_0 doing create fail-count-apc_snmp_node2_an-pdu03#start_0=INFINITY: Transient attribute change | cib=0.337.91 source=abort_unless_down:313 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=true Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 101 aborted by status-2-last-failure-apc_snmp_node2_an-pdu03.start_0 doing create last-failure-apc_snmp_node2_an-pdu03#start_0=1627082013: Transient attribute change | cib=0.337.92 source=abort_unless_down:313 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=true Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 22 18:16:47 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu03 on an-a02n01 at Jul 22 18:16:47 2021 | rc=1 id=apc_snmp_node2_an-pdu03_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu03_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu03_last_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu03 has failed INFINITY times on an-a02n01 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu03 has failed INFINITY times on an-a02n01 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu03 has failed INFINITY times on an-a02n02 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu03 cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu04 cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu04 cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node2_an-pdu03 on an-a02n02 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for ipmilan_node1 on an-a02n02 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Stop apc_snmp_node1_an-pdu03 ( an-a02n02 ) due to node availability Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Stop apc_snmp_node1_an-pdu04 ( an-a02n02 ) due to node availability Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Recover apc_snmp_node2_an-pdu03 ( an-a02n02 ) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Stop apc_snmp_node2_an-pdu04 ( an-a02n02 ) due to node availability Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node2 (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node2 (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Move ipmilan_node1 ( an-a02n01 -> an-a02n02 ) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 102, saving inputs in /var/lib/pacemaker/pengine/pe-input-123.bz2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (handle_response) info: pe_calc calculation pe_calc-dc-1627082013-259 is obsolete Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 22 18:16:47 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu03 on an-a02n01 at Jul 22 18:16:47 2021 | rc=1 id=apc_snmp_node2_an-pdu03_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu03_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu03_last_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu03 has failed INFINITY times on an-a02n01 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu03 has failed INFINITY times on an-a02n01 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu03 has failed INFINITY times on an-a02n02 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu03 has failed INFINITY times on an-a02n02 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu03 cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu04 cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu03 cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu04 cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for delay_node2 on an-a02n02 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for delay_node1 on an-a02n02 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Stop apc_snmp_node1_an-pdu03 ( an-a02n02 ) due to node availability Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Stop apc_snmp_node1_an-pdu04 ( an-a02n02 ) due to node availability Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Stop apc_snmp_node2_an-pdu03 ( an-a02n02 ) due to node availability Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Stop apc_snmp_node2_an-pdu04 ( an-a02n02 ) due to node availability Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node2 (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Move delay_node2 ( an-a02n01 -> an-a02n02 ) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Move delay_node1 ( an-a02n01 -> an-a02n02 ) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 103, saving inputs in /var/lib/pacemaker/pengine/pe-input-124.bz2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (do_te_invoke) info: Processing graph 103 (ref=pe_calc-dc-1627082013-260) derived from /var/lib/pacemaker/pengine/pe-input-124.bz2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation apc_snmp_node1_an-pdu03_stop_0 on an-a02n02 | action 11 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation apc_snmp_node1_an-pdu04_stop_0 on an-a02n02 | action 12 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation apc_snmp_node2_an-pdu03_stop_0 on an-a02n02 | action 13 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation apc_snmp_node2_an-pdu04_stop_0 on an-a02n02 | action 14 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation delay_node2_stop_0 locally on an-a02n01 | action 17 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of stop operation for delay_node2 on an-a02n01 | transition_key=17:103:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=delay_node2_stop_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:delay_node2 action:stop call_id:119 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation delay_node1_stop_0 locally on an-a02n01 | action 22 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/347) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-execd [121630] (log_finished) info: delay_node2 stop (call 119) exited with status 0 (execution time 0ms, queue time 0ms) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of stop operation for delay_node1 on an-a02n01 | transition_key=22:103:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=delay_node1_stop_0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:delay_node1 action:stop call_id:121 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) info: Result of monitor operation for delay_node2 on an-a02n01: Cancelled | call=117 key=delay_node2_monitor_60000 confirmed=true Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.92 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.93 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=93 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='delay_node2']/lrm_rsc_op[@id='delay_node2_last_0']: @operation_key=delay_node2_stop_0, @operation=stop, @crm-debug-origin=do_update_resource, @transition-key=17:103:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;17:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change= Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-execd [121630] (log_finished) info: delay_node1 stop (call 121) exited with status 0 (execution time 0ms, queue time 0ms) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) notice: Result of stop operation for delay_node2 on an-a02n01: ok | rc=0 call=119 key=delay_node2_stop_0 confirmed=true cib-update=349 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) info: Result of monitor operation for delay_node1 on an-a02n01: Cancelled | call=116 key=delay_node1_monitor_60000 confirmed=true Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/347, version=0.337.93) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) notice: Result of stop operation for delay_node1 on an-a02n01: ok | rc=0 call=121 key=delay_node1_stop_0 confirmed=true cib-update=350 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/348) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/349) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/350) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.93 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.94 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=94 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @operation_key=apc_snmp_node1_an-pdu03_stop_0, @operation=stop, @transition-key=11:103:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;11:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=16 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/59, version=0.337.94) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.94 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.95 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=95 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='delay_node1']/lrm_rsc_op[@id='delay_node1_last_0']: @operation_key=delay_node1_stop_0, @operation=stop, @crm-debug-origin=do_update_resource, @transition-key=22:103:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;22:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change= Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/348, version=0.337.95) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.95 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.96 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=96 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='delay_node2']/lrm_rsc_op[@id='delay_node2_last_0']: @transition-magic=0:0;17:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=119, @rc-code=0, @op-status=0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/349, version=0.337.96) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 103 action 17 (delay_node2_stop_0 on an-a02n01) confirmed: ok | rc=0 call-id=119 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation delay_node2_start_0 on an-a02n02 | action 18 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.96 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.97 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=97 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='delay_node1']/lrm_rsc_op[@id='delay_node1_last_0']: @transition-magic=0:0;22:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=121, @rc-code=0, @op-status=0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/350, version=0.337.97) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 103 action 22 (delay_node1_stop_0 on an-a02n01) confirmed: ok | rc=0 call-id=121 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation delay_node1_start_0 on an-a02n02 | action 23 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.97 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.98 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=98 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_0']: @operation_key=apc_snmp_node1_an-pdu04_stop_0, @operation=stop, @transition-key=12:103:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;12:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=16 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/60, version=0.337.98) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.98 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.99 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=99 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @transition-magic=0:0;11:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=86, @rc-code=0, @op-status=0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/61, version=0.337.99) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 103 action 11 (apc_snmp_node1_an-pdu03_stop_0 on an-a02n02) confirmed: ok | rc=0 call-id=86 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.99 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.100 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=100 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_0']: @transition-magic=0:0;12:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=87, @rc-code=0, @op-status=0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/62, version=0.337.100) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 103 action 12 (apc_snmp_node1_an-pdu04_stop_0 on an-a02n02) confirmed: ok | rc=0 call-id=87 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.100 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.101 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=101 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_0']: @operation_key=apc_snmp_node2_an-pdu03_stop_0, @operation=stop, @transition-key=13:103:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;13:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=16 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/63, version=0.337.101) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.101 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.102 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=102 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_0']: @operation_key=apc_snmp_node2_an-pdu04_stop_0, @operation=stop, @transition-key=14:103:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;14:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=16 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/64, version=0.337.102) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.102 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.103 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=103 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_0']: @transition-magic=0:0;13:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=88, @rc-code=0, @op-status=0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/65, version=0.337.103) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 103 action 13 (apc_snmp_node2_an-pdu03_stop_0 on an-a02n02) confirmed: ok | rc=0 call-id=88 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.103 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.104 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=104 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_0']: @transition-magic=0:0;14:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=89, @rc-code=0, @op-status=0 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/66, version=0.337.104) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 103 action 14 (apc_snmp_node2_an-pdu04_stop_0 on an-a02n02) confirmed: ok | rc=0 call-id=89 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.104 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.105 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=105 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='delay_node2']/lrm_rsc_op[@id='delay_node2_last_0']: @operation_key=delay_node2_start_0, @operation=start, @transition-key=18:103:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;18:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1627082013 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/67, version=0.337.105) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.105 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.106 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=106 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='delay_node1']/lrm_rsc_op[@id='delay_node1_last_0']: @operation_key=delay_node1_start_0, @operation=start, @transition-key=23:103:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;23:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1627082013 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/68, version=0.337.106) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.106 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.107 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=107 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='delay_node2']/lrm_rsc_op[@id='delay_node2_last_0']: @transition-magic=0:0;18:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=90, @rc-code=0, @op-status=0, @exec-time=9 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/69, version=0.337.107) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 103 action 18 (delay_node2_start_0 on an-a02n02) confirmed: ok | rc=0 call-id=90 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation delay_node2_monitor_60000 on an-a02n02 | action 19 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.107 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.108 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=108 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='delay_node1']/lrm_rsc_op[@id='delay_node1_last_0']: @transition-magic=0:0;23:103:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=91, @rc-code=0, @op-status=0, @exec-time=10 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/70, version=0.337.108) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 103 action 23 (delay_node1_start_0 on an-a02n02) confirmed: ok | rc=0 call-id=91 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation delay_node1_monitor_60000 on an-a02n02 | action 24 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.108 2 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.337.109 (null) Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=109 Jul 23 16:13:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='delay_node2']: S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd Jul 23 16:13:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_ping) info: Reporting our current digest to an-a02n01: fbde84a1ab201d656041be5ee9534f46 for 0.337.112 (0x5619761712c0 0) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.337.112 2 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.0 6a24af66df3d9f825cc2681222f8f5d6 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=338, @num_updates=0 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/configuration/resources/primitive[@id='apc_snmp_node1_an-pdu03']/instance_attributes[@id='apc_snmp_node1_an-pdu03-instance_attributes']/nvpair[@id='apc_snmp_node1_an-pdu03-instance_attributes-ip']: @value=10.201.2.3 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_replace_notify) info: Replaced: 0.337.112 -> 0.338.0 from an-a02n02 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_replace operation for section configuration: OK (rc=0, origin=an-a02n02/cibadmin/2, version=0.338.0) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 103 aborted by apc_snmp_node1_an-pdu03-instance_attributes-ip doing modify ip=10.201.2.3: Configuration change | cib=0.338.0 source=te_update_diff_v2:474 path=/cib/configuration/resources/primitive[@id='apc_snmp_node1_an-pdu03']/instance_attributes[@id='apc_snmp_node1_an-pdu03-instance_attributes']/nvpair[@id='apc_snmp_node1_an-pdu03-instance_attributes-ip'] complete=true Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_replaced_cb) notice: Updating all attributes after cib_refresh_notify event Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 31 with 2 changes for last-failure-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 32 with 2 changes for fail-count-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Processed 2 private changes for #attrd-protocol, id=n/a, set=n/a Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 33 with 2 changes for fail-count-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 34 with 2 changes for last-failure-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 35 with 2 changes for fail-count-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 36 with 2 changes for last-failure-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 37 with 2 changes for fail-count-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 38 with 2 changes for last-failure-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_ELECTION | input=I_ELECTION cause=C_FSA_INTERNAL origin=do_cib_replaced Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Unset DC. Was an-a02n01 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/31) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/353) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/354) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-controld [121633] (election_check) info: election-DC won by local node Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-controld [121633] (do_log) info: Input I_ELECTION_DC received in state S_ELECTION from election_win_cb Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) notice: State transition S_ELECTION -> S_INTEGRATION | input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=election_win_cb Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-fenced [121629] (update_cib_stonith_devices_v2) info: Updating device list from CIB: modify nvpair[@id='apc_snmp_node1_an-pdu03-instance_attributes-ip'] Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-fenced [121629] (cib_devices_update) info: Updating devices to version 0.338.0 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/32) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-26.raw Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/33) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/34) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/35) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/36) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/31, version=0.338.0) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 31 result for last-failure-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=1627003019 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=1627082013 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.338.0 of the CIB to disk (digest: caa9c9d11084bba7f1ad0b3668a8e153) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.0AuvQm (digest: /var/lib/pacemaker/cib/cib.CdApcl) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_device_register) notice: Added 'apc_snmp_node1_an-pdu03' to device list (8 active devices) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/353, version=0.338.0) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.0 2 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.1 (null) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=1 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_cib_replaced Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_cib_replaced Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/354, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/32, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 32 result for fail-count-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=INFINITY Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=INFINITY Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/33, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 33 result for fail-count-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=INFINITY Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=INFINITY Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/34, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 34 result for last-failure-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=1627003020 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=1627082012 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/35, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 35 result for fail-count-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=INFINITY Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=INFINITY Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/36, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 36 result for last-failure-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=1627003034 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=1627082012 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/37) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/38) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/37, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 37 result for fail-count-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=INFINITY Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=INFINITY Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/38, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 38 result for last-failure-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=1627003035 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=1627082012 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_takeover) info: Taking over DC status for this partition Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_master operation for section 'all': OK (rc=0, origin=local/crmd/356, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/357) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/359) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/361) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-6 offer to an-a02n02 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-6 offer to an-a02n01 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_join_offer_all) info: Waiting on join-6 requests from 2 outstanding nodes Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=an-a02n01/crmd/357, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/359, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/361, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/363) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Set DC to an-a02n01 (3.10.2) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/363, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_INTEGRATION -> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Digest matched on replace from an-a02n01: a2d0c684b380b002faa4969d747b5985 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Replaced 0.338.1 with 0.338.1 from an-a02n01 Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=an-a02n01/crmd/366, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/367) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/368) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-27.raw Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/367, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.338.0 of the CIB to disk (digest: caa9c9d11084bba7f1ad0b3668a8e153) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.LWA0ou (digest: /var/lib/pacemaker/cib/cib.V5Ncly) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/368, version=0.338.1) Jul 23 16:22:27 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157918] exited with status 0 Jul 23 16:22:28 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157920] exited with status 0 Jul 23 16:22:28 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157921] exited with status 0 Jul 23 16:22:28 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4157922] exited with status 0 Jul 23 16:22:28 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4157924] exited with status 0 Jul 23 16:22:28 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157926] exited with status 0 Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157927] exited with status 0 Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157928] exited with status 0 Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-controld [121633] (controld_delete_node_state) info: Deleting resource history for node an-a02n02 (via CIB call 369) | xpath=//node_state[@uname='an-a02n02']/lrm Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='an-a02n02']/lrm to all (origin=local/crmd/369) Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-controld [121633] (controld_delete_node_state) info: Deleting resource history for node an-a02n01 (via CIB call 371) | xpath=//node_state[@uname='an-a02n01']/lrm Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/370) Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='an-a02n01']/lrm to all (origin=local/crmd/371) Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.1 2 Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.2 (null) Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2'] Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=2 Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n02']/lrm: OK (rc=0, origin=an-a02n01/crmd/369, version=0.338.2) Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.2 2 Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.3 (null) Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=3 Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_lrm_query_internal Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/370, version=0.338.3) Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.3 2 Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.4 (null) Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/lrm[@id='1'] Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=4 Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n01']/lrm: OK (rc=0, origin=an-a02n01/crmd/371, version=0.338.4) Jul 23 16:22:29 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157929] exited with status 0 Jul 23 16:22:30 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157930] exited with status 0 Jul 23 16:22:30 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157931] exited with status 0 Jul 23 16:22:30 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4157932] exited with status 0 Jul 23 16:22:30 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4157934] exited with status 0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157936] exited with status 0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157937] exited with status 0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157938] exited with status 0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/372) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE | input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 103 aborted: Peer Cancelled | source=do_te_invoke:129 complete=true Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.4 2 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.5 (null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=5 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_lrm_query_internal Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/372, version=0.338.5) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 103 aborted: History refresh | source=process_resource_updates:268 complete=true Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/375) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/376) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/377) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/375, version=0.338.5) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.5 2 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.6 (null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=6 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/376, version=0.338.6) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=an-a02n01/crmd/377, version=0.338.6) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_digest_cmp) info: Parameters to 0ms-interval start action for apc_snmp_node1_an-pdu03 on an-a02n01 changed: hash was 3fd89635ef735220dd13344790fbf853 vs. now b6d660e5416057f7575eac6369636a4f (reload:3.10.2) 0:1;41:0:0:789131c0-5164-4a52-9822-30c5026c3827 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe__clear_failcount) notice: Clearing failure of apc_snmp_node1_an-pdu03 on an-a02n01 because resource parameters have changed | apc_snmp_node1_an-pdu03_clear_failcount_0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 22 18:16:47 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu03 on an-a02n01 at Jul 22 18:16:47 2021 | rc=1 id=apc_snmp_node2_an-pdu03_last_failure_0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_digest_cmp) info: Parameters to 0ms-interval start action for apc_snmp_node1_an-pdu03 on an-a02n02 changed: hash was 3fd89635ef735220dd13344790fbf853 vs. now b6d660e5416057f7575eac6369636a4f (reload:3.10.2) 0:1;31:101:0:789131c0-5164-4a52-9822-30c5026c3827 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe__clear_failcount) notice: Clearing failure of apc_snmp_node1_an-pdu03 on an-a02n02 because resource parameters have changed | apc_snmp_node1_an-pdu03_clear_failcount_0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu03_last_failure_0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu03 has failed INFINITY times on an-a02n01 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu03 has failed INFINITY times on an-a02n02 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu04 cannot run anywhere Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu03 cannot run anywhere Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu04 cannot run anywhere Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu03 on an-a02n01 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for ipmilan_node2 on an-a02n02 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start apc_snmp_node1_an-pdu03 ( an-a02n01 ) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu04 (Stopped) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu03 (Stopped) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Stopped) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Move ipmilan_node2 ( an-a02n01 -> an-a02n02 ) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node2 (Started an-a02n02) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n01) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n02) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 104, saving inputs in /var/lib/pacemaker/pengine/pe-input-125.bz2 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (do_te_invoke) info: Processing graph 104 (ref=pe_calc-dc-1627082551-278) derived from /var/lib/pacemaker/pengine/pe-input-125.bz2 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation apc_snmp_node1_an-pdu03_start_0 locally on an-a02n01 | action 13 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of start operation for apc_snmp_node1_an-pdu03 on an-a02n01 | transition_key=13:104:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=apc_snmp_node1_an-pdu03_start_0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:apc_snmp_node1_an-pdu03 action:start call_id:122 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (te_crm_command) info: Executing crm-event (12) without waiting: clear_failcount on an-a02n02 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/380) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (te_crm_command) info: Executing crm-event (1) locally without waiting: clear_failcount on an-a02n01 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation ipmilan_node2_stop_0 locally on an-a02n01 | action 15 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of stop operation for ipmilan_node2 on an-a02n01 | transition_key=15:104:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=ipmilan_node2_stop_0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:ipmilan_node2 action:stop call_id:124 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) info: Result of monitor operation for ipmilan_node2 on an-a02n01: Cancelled | call=113 key=ipmilan_node2_monitor_60000 confirmed=true Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.6 2 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.7 (null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=7 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @operation_key=apc_snmp_node1_an-pdu03_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=13:104:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;13:104:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-cod Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/380, version=0.338.7) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (update_attrd_clear_failures) info: Asking pacemaker-attrd to clear failure of all operations for apc_snmp_node1_an-pdu03 on cluster node an-a02n01 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/381) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section /cib/status/node_state[@uname='an-a02n01']/lrm/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_failure_0'] to all (origin=local/crmd/382) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.7 2 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.8 (null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=8 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='ipmilan_node2']/lrm_rsc_op[@id='ipmilan_node2_last_0']: @operation_key=ipmilan_node2_stop_0, @operation=stop, @crm-debug-origin=do_update_resource, @transition-key=15:104:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;15:104:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-c Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/381, version=0.338.8) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]: INFINITY -> (unset) | from an-a02n01 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 40 with 2 changes for fail-count-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]: 1627003020 -> (unset) | from an-a02n01 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 41 with 2 changes for last-failure-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.8 2 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.9 (null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_failure_0'] Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=9 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-execd [121630] (log_finished) info: ipmilan_node2 stop (call 124) exited with status 0 (execution time 1ms, queue time 0ms) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) notice: Result of stop operation for ipmilan_node2 on an-a02n01: ok | rc=0 call=124 key=ipmilan_node2_stop_0 confirmed=true cib-update=383 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]: INFINITY -> (unset) | from an-a02n02 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Write out of 'fail-count-apc_snmp_node1_an-pdu03#start_0' delayed: update 40 in progress Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]: 1627082012 -> (unset) | from an-a02n02 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Write out of 'last-failure-apc_snmp_node1_an-pdu03#start_0' delayed: update 41 in progress Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section /cib/status/node_state[@uname='an-a02n01']/lrm/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_failure_0']: OK (rc=0, origin=an-a02n01/crmd/382, version=0.338.9) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) notice: Transition 104 aborted by deletion of lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_failure_0']: Resource operation removal | cib=0.338.9 source=process_op_deletion:358 path=/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_failure_0'] complete=false Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/383) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/40) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/41) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.9 2 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.10 (null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=10 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='ipmilan_node2']/lrm_rsc_op[@id='ipmilan_node2_last_0']: @transition-magic=0:0;15:104:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=124, @rc-code=0, @op-status=0, @exec-time=1 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/383, version=0.338.10) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 104 action 15 (ipmilan_node2_stop_0 on an-a02n01) confirmed: ok | rc=0 call-id=124 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.10 2 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.11 (null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-fail-count-apc_snmp_node1_an-pdu03.start_0'] Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=11 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/40, version=0.338.11) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 40 result for fail-count-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 104 aborted by deletion of nvpair[@id='status-1-fail-count-apc_snmp_node1_an-pdu03.start_0']: Transient attribute change | cib=0.338.11 source=abort_unless_down:327 path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-fail-count-apc_snmp_node1_an-pdu03.start_0'] complete=false Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=INFINITY Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 42 with 2 changes for fail-count-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.11 2 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.12 (null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-last-failure-apc_snmp_node1_an-pdu03.start_0'] Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=12 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/41, version=0.338.12) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 41 result for last-failure-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=1627082012 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 104 aborted by deletion of nvpair[@id='status-1-last-failure-apc_snmp_node1_an-pdu03.start_0']: Transient attribute change | cib=0.338.12 source=abort_unless_down:327 path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-last-failure-apc_snmp_node1_an-pdu03.start_0'] complete=false Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 43 with 2 changes for last-failure-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.12 2 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.13 (null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_failure_0'] Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=13 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section /cib/status/node_state[@uname='an-a02n02']/lrm/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_failure_0']: OK (rc=0, origin=an-a02n02/crmd/77, version=0.338.13) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 104 aborted by deletion of lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_failure_0']: Resource operation removal | cib=0.338.13 source=process_op_deletion:358 path=/cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_failure_0'] complete=false Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/42) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/43) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.13 2 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.14 (null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-fail-count-apc_snmp_node1_an-pdu03.start_0'] Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=14 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/42, version=0.338.14) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 42 result for fail-count-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 104 aborted by deletion of nvpair[@id='status-2-fail-count-apc_snmp_node1_an-pdu03.start_0']: Transient attribute change | cib=0.338.14 source=abort_unless_down:327 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-fail-count-apc_snmp_node1_an-pdu03.start_0'] complete=false Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.14 2 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.15 (null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-last-failure-apc_snmp_node1_an-pdu03.start_0'] Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=15 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/43, version=0.338.15) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 43 result for last-failure-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 104 aborted by deletion of nvpair[@id='status-2-last-failure-apc_snmp_node1_an-pdu03.start_0']: Transient attribute change | cib=0.338.15 source=abort_unless_down:327 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-last-failure-apc_snmp_node1_an-pdu03.start_0'] complete=false Jul 23 16:22:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-execd [121630] (log_finished) info: apc_snmp_node1_an-pdu03 start (call 122) exited with status 0 (execution time 379ms, queue time 0ms) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_apc_snmp_metadata_1[4157943] exited with status 0 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) notice: Result of start operation for apc_snmp_node1_an-pdu03 on an-a02n01: ok | rc=0 call=122 key=apc_snmp_node1_an-pdu03_start_0 confirmed=true cib-update=384 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/384) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.15 2 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.16 (null) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=16 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @transition-magic=0:0;13:104:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=122, @rc-code=0, @op-status=0, @exec-time=379 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/384, version=0.338.16) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 104 action 13 (apc_snmp_node1_an-pdu03_start_0 on an-a02n01) confirmed: ok | rc=0 call-id=122 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-controld [121633] (run_graph) notice: Transition 104 (Complete=4, Pending=0, Fired=0, Skipped=2, Incomplete=3, Source=/var/lib/pacemaker/pengine/pe-input-125.bz2): Stopped Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu03 on an-a02n01 at Jul 22 18:16:47 2021 | rc=1 id=apc_snmp_node2_an-pdu03_last_failure_0 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu03_last_failure_0 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu03 has failed INFINITY times on an-a02n01 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu03 has failed INFINITY times on an-a02n02 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu04 cannot run anywhere Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu03 cannot run anywhere Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu04 cannot run anywhere Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu03 on an-a02n01 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for ipmilan_node2 on an-a02n02 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu03 (Started an-a02n01) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu04 (Stopped) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu03 (Stopped) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Stopped) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start ipmilan_node2 ( an-a02n02 ) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node2 (Started an-a02n02) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n01) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n02) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 105, saving inputs in /var/lib/pacemaker/pengine/pe-input-126.bz2 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-controld [121633] (do_te_invoke) info: Processing graph 105 (ref=pe_calc-dc-1627082552-283) derived from /var/lib/pacemaker/pengine/pe-input-126.bz2 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation apc_snmp_node1_an-pdu03_monitor_60000 locally on an-a02n01 | action 12 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of monitor operation for apc_snmp_node1_an-pdu03 on an-a02n01 | transition_key=12:105:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=apc_snmp_node1_an-pdu03_monitor_60000 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation ipmilan_node2_start_0 on an-a02n02 | action 13 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/386) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.16 2 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.338.17 (null) Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=17 Jul 23 16:22:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']: S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.338.22 2 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.0 (null) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=339, @num_updates=0 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/configuration/resources/primitive[@id='apc_snmp_node2_an-pdu03']/instance_attributes[@id='apc_snmp_node2_an-pdu03-instance_attributes']/nvpair[@id='apc_snmp_node2_an-pdu03-instance_attributes-ip']: @value=10.201.2.3 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_replace_notify) info: Replaced: 0.338.22 -> 0.339.0 from an-a02n02 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_replace operation for section configuration: OK (rc=0, origin=an-a02n02/cibadmin/2, version=0.339.0) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_replaced_cb) notice: Updating all attributes after cib_refresh_notify event Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 105 aborted by apc_snmp_node2_an-pdu03-instance_attributes-ip doing modify ip=10.201.2.3: Configuration change | cib=0.339.0 source=te_update_diff_v2:474 path=/cib/configuration/resources/primitive[@id='apc_snmp_node2_an-pdu03']/instance_attributes[@id='apc_snmp_node2_an-pdu03-instance_attributes']/nvpair[@id='apc_snmp_node2_an-pdu03-instance_attributes-ip'] complete=true Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 44 with 2 changes for last-failure-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 45 with 2 changes for fail-count-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Processed 2 private changes for #attrd-protocol, id=n/a, set=n/a Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/44) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 46 with 2 changes for fail-count-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/45) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 47 with 2 changes for last-failure-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/46) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 48 with 2 changes for fail-count-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 49 with 2 changes for last-failure-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 50 with 2 changes for fail-count-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_ELECTION | input=I_ELECTION cause=C_FSA_INTERNAL origin=do_cib_replaced Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Unset DC. Was an-a02n01 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 51 with 2 changes for last-failure-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/44, version=0.339.0) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-fenced [121629] (update_cib_stonith_devices_v2) info: Updating device list from CIB: modify nvpair[@id='apc_snmp_node2_an-pdu03-instance_attributes-ip'] Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-fenced [121629] (cib_devices_update) info: Updating devices to version 0.339.0 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 44 result for last-failure-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=1627003019 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=1627082013 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/390) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/391) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-controld [121633] (election_check) info: election-DC won by local node Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-controld [121633] (do_log) info: Input I_ELECTION_DC received in state S_ELECTION from election_win_cb Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) notice: State transition S_ELECTION -> S_INTEGRATION | input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=election_win_cb Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/47) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/48) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/49) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/50) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/51) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/45, version=0.339.0) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 45 result for fail-count-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=INFINITY Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=INFINITY Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_device_register) notice: Added 'apc_snmp_node2_an-pdu03' to device list (8 active devices) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.0 2 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.1 (null) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=1 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/46, version=0.339.1) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 46 result for fail-count-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/390, version=0.339.1) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.1 2 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.2 (null) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=2 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_cib_replaced Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_cib_replaced Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/391, version=0.339.2) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.2 2 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.3 (null) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=3 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/47, version=0.339.3) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 47 result for last-failure-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/48, version=0.339.3) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 48 result for fail-count-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=INFINITY Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=INFINITY Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/49, version=0.339.3) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 49 result for last-failure-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=1627003034 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=1627082012 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/50, version=0.339.3) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 50 result for fail-count-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=INFINITY Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=INFINITY Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/51, version=0.339.3) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 51 result for last-failure-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=1627003035 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=1627082012 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_takeover) info: Taking over DC status for this partition Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-28.raw Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.339.0 of the CIB to disk (digest: 49133dea2a2bc8449f565905153ef22f) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_master operation for section 'all': OK (rc=0, origin=local/crmd/393, version=0.339.3) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.aAM8uO (digest: /var/lib/pacemaker/cib/cib.DTcgIN) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/394) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/396) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/398) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/400) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-7 offer to an-a02n02 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-7 offer to an-a02n01 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_join_offer_all) info: Waiting on join-7 requests from 2 outstanding nodes Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=an-a02n01/crmd/394, version=0.339.3) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/396, version=0.339.3) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/398, version=0.339.3) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Set DC to an-a02n01 (3.10.2) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/400, version=0.339.3) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_INTEGRATION -> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Digest matched on replace from an-a02n01: ed782acbb5ff659690d78c049f31bffb Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Replaced 0.339.3 with 0.339.3 from an-a02n01 Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=an-a02n01/crmd/403, version=0.339.3) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/404) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/405) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-29.raw Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/404, version=0.339.3) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.339.0 of the CIB to disk (digest: 49133dea2a2bc8449f565905153ef22f) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.87jfCW (digest: /var/lib/pacemaker/cib/cib.1I6N1V) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/405, version=0.339.3) Jul 23 16:22:33 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_apc_snmp_metadata_1[4157950] exited with status 0 Jul 23 16:22:34 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157953] exited with status 0 Jul 23 16:22:34 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157954] exited with status 0 Jul 23 16:22:34 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157955] exited with status 0 Jul 23 16:22:34 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4157956] exited with status 0 Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157958] exited with status 0 Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157959] exited with status 0 Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157960] exited with status 0 Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-controld [121633] (controld_delete_node_state) info: Deleting resource history for node an-a02n02 (via CIB call 406) | xpath=//node_state[@uname='an-a02n02']/lrm Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='an-a02n02']/lrm to all (origin=local/crmd/406) Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-controld [121633] (controld_delete_node_state) info: Deleting resource history for node an-a02n01 (via CIB call 408) | xpath=//node_state[@uname='an-a02n01']/lrm Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/407) Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='an-a02n01']/lrm to all (origin=local/crmd/408) Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.3 2 Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.4 (null) Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2'] Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=4 Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n02']/lrm: OK (rc=0, origin=an-a02n01/crmd/406, version=0.339.4) Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.4 2 Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.5 (null) Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=5 Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_lrm_query_internal Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/407, version=0.339.5) Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.5 2 Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.6 (null) Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/lrm[@id='1'] Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=6 Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n01']/lrm: OK (rc=0, origin=an-a02n01/crmd/408, version=0.339.6) Jul 23 16:22:35 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_apc_snmp_metadata_1[4157961] exited with status 0 Jul 23 16:22:36 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157963] exited with status 0 Jul 23 16:22:36 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157964] exited with status 0 Jul 23 16:22:36 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157965] exited with status 0 Jul 23 16:22:36 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4157966] exited with status 0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157968] exited with status 0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157969] exited with status 0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157970] exited with status 0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/409) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE | input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 105 aborted: Peer Cancelled | source=do_te_invoke:129 complete=true Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.6 2 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.7 (null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=7 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_lrm_query_internal Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/409, version=0.339.7) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 105 aborted: History refresh | source=process_resource_updates:268 complete=true Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/412) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/413) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/414) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/412, version=0.339.7) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.7 2 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.8 (null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=8 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/413, version=0.339.8) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=an-a02n01/crmd/414, version=0.339.8) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_digest_cmp) info: Parameters to 0ms-interval start action for apc_snmp_node2_an-pdu03 on an-a02n01 changed: hash was b65fbd52b0bbfad83728e2328fb84a09 vs. now e7b5280893307b461d430ebe126191cd (reload:3.10.2) 0:1;45:0:0:789131c0-5164-4a52-9822-30c5026c3827 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe__clear_failcount) notice: Clearing failure of apc_snmp_node2_an-pdu03 on an-a02n01 because resource parameters have changed | apc_snmp_node2_an-pdu03_clear_failcount_0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu03 on an-a02n01 at Jul 22 18:16:47 2021 | rc=1 id=apc_snmp_node2_an-pdu03_last_failure_0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_digest_cmp) info: Parameters to 0ms-interval start action for apc_snmp_node2_an-pdu03 on an-a02n02 changed: hash was b65fbd52b0bbfad83728e2328fb84a09 vs. now e7b5280893307b461d430ebe126191cd (reload:3.10.2) 0:1;35:101:0:789131c0-5164-4a52-9822-30c5026c3827 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe__clear_failcount) notice: Clearing failure of apc_snmp_node2_an-pdu03 on an-a02n02 because resource parameters have changed | apc_snmp_node2_an-pdu03_clear_failcount_0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu03_last_failure_0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu04 cannot run anywhere Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu04 cannot run anywhere Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node2_an-pdu03 on an-a02n02 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for delay_node2 on an-a02n01 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu03 (Started an-a02n01) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu04 (Stopped) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start apc_snmp_node2_an-pdu03 ( an-a02n02 ) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Stopped) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node2 (Started an-a02n02) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Move delay_node2 ( an-a02n02 -> an-a02n01 ) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n01) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n02) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 106, saving inputs in /var/lib/pacemaker/pengine/pe-input-127.bz2 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (do_te_invoke) info: Processing graph 106 (ref=pe_calc-dc-1627082557-294) derived from /var/lib/pacemaker/pengine/pe-input-127.bz2 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation apc_snmp_node2_an-pdu03_start_0 on an-a02n02 | action 16 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (te_crm_command) info: Executing crm-event (13) without waiting: clear_failcount on an-a02n02 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (te_crm_command) info: Executing crm-event (7) locally without waiting: clear_failcount on an-a02n01 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation delay_node2_stop_0 on an-a02n02 | action 20 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (update_attrd_clear_failures) info: Asking pacemaker-attrd to clear failure of all operations for apc_snmp_node2_an-pdu03 on cluster node an-a02n01 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n01]: 1627003019 -> (unset) | from an-a02n01 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section /cib/status/node_state[@uname='an-a02n01']/lrm/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_failure_0'] to all (origin=local/crmd/417) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 53 with 2 changes for last-failure-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/53) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n01]: INFINITY -> (unset) | from an-a02n01 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 54 with 2 changes for fail-count-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/54) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n02]: 1627082013 -> (unset) | from an-a02n02 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Write out of 'last-failure-apc_snmp_node2_an-pdu03#start_0' delayed: update 53 in progress Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n02]: INFINITY -> (unset) | from an-a02n02 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Write out of 'fail-count-apc_snmp_node2_an-pdu03#start_0' delayed: update 54 in progress Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.8 2 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.9 (null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_failure_0'] Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=9 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section /cib/status/node_state[@uname='an-a02n01']/lrm/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_failure_0']: OK (rc=0, origin=an-a02n01/crmd/417, version=0.339.9) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) notice: Transition 106 aborted by deletion of lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_failure_0']: Resource operation removal | cib=0.339.9 source=process_op_deletion:358 path=/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_failure_0'] complete=false Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.9 2 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.10 (null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-last-failure-apc_snmp_node2_an-pdu03.start_0'] Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=10 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/53, version=0.339.10) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 53 result for last-failure-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 106 aborted by deletion of nvpair[@id='status-1-last-failure-apc_snmp_node2_an-pdu03.start_0']: Transient attribute change | cib=0.339.10 source=abort_unless_down:327 path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-last-failure-apc_snmp_node2_an-pdu03.start_0'] complete=false Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=1627082013 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 55 with 2 changes for last-failure-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.10 2 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.11 (null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-fail-count-apc_snmp_node2_an-pdu03.start_0'] Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=11 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/54, version=0.339.11) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 54 result for fail-count-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=INFINITY Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 106 aborted by deletion of nvpair[@id='status-1-fail-count-apc_snmp_node2_an-pdu03.start_0']: Transient attribute change | cib=0.339.11 source=abort_unless_down:327 path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-fail-count-apc_snmp_node2_an-pdu03.start_0'] complete=false Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 56 with 2 changes for fail-count-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.11 2 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.12 (null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=12 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_0']: @operation_key=apc_snmp_node2_an-pdu03_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=16:106:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;16:106:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-cod Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/84, version=0.339.12) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.12 2 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.13 (null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_failure_0'] Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=13 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section /cib/status/node_state[@uname='an-a02n02']/lrm/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_failure_0']: OK (rc=0, origin=an-a02n02/crmd/85, version=0.339.13) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 106 aborted by deletion of lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_failure_0']: Resource operation removal | cib=0.339.13 source=process_op_deletion:358 path=/cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_failure_0'] complete=false Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/55) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/56) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.13 2 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.14 (null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-last-failure-apc_snmp_node2_an-pdu03.start_0'] Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=14 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/55, version=0.339.14) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 55 result for last-failure-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 106 aborted by deletion of nvpair[@id='status-2-last-failure-apc_snmp_node2_an-pdu03.start_0']: Transient attribute change | cib=0.339.14 source=abort_unless_down:327 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-last-failure-apc_snmp_node2_an-pdu03.start_0'] complete=false Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.14 2 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.15 (null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-fail-count-apc_snmp_node2_an-pdu03.start_0'] Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=15 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/56, version=0.339.15) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 56 result for fail-count-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 106 aborted by deletion of nvpair[@id='status-2-fail-count-apc_snmp_node2_an-pdu03.start_0']: Transient attribute change | cib=0.339.15 source=abort_unless_down:327 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-fail-count-apc_snmp_node2_an-pdu03.start_0'] complete=false Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.15 2 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.16 (null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=16 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='delay_node2']/lrm_rsc_op[@id='delay_node2_last_0']: @operation_key=delay_node2_stop_0, @operation=stop, @crm-debug-origin=do_update_resource, @transition-key=20:106:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;20:106:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change= Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/86, version=0.339.16) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.16 2 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.17 (null) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=17 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='delay_node2']/lrm_rsc_op[@id='delay_node2_last_0']: @transition-magic=0:0;20:106:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=98, @rc-code=0, @op-status=0 Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/87, version=0.339.17) Jul 23 16:22:37 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 106 action 20 (delay_node2_stop_0 on an-a02n02) confirmed: ok | rc=0 call-id=98 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.17 2 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.18 (null) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=18 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_0']: @transition-magic=0:0;16:106:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=96, @rc-code=0, @op-status=0, @exec-time=328 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/88, version=0.339.18) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 106 action 16 (apc_snmp_node2_an-pdu03_start_0 on an-a02n02) confirmed: ok | rc=0 call-id=96 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-controld [121633] (run_graph) notice: Transition 106 (Complete=4, Pending=0, Fired=0, Skipped=2, Incomplete=3, Source=/var/lib/pacemaker/pengine/pe-input-127.bz2): Stopped Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu04 cannot run anywhere Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu04 cannot run anywhere Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node2_an-pdu03 on an-a02n02 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for delay_node2 on an-a02n01 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu03 (Started an-a02n01) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu04 (Stopped) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu03 (Started an-a02n02) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Stopped) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node2 (Started an-a02n02) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start delay_node2 ( an-a02n01 ) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n01) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n02) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 107, saving inputs in /var/lib/pacemaker/pengine/pe-input-128.bz2 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-controld [121633] (do_te_invoke) info: Processing graph 107 (ref=pe_calc-dc-1627082558-299) derived from /var/lib/pacemaker/pengine/pe-input-128.bz2 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation apc_snmp_node2_an-pdu03_monitor_60000 on an-a02n02 | action 15 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation delay_node2_start_0 locally on an-a02n01 | action 18 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of start operation for delay_node2 on an-a02n01 | transition_key=18:107:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=delay_node2_start_0 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:delay_node2 action:start call_id:126 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/419) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.18 2 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.19 (null) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=19 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='delay_node2']/lrm_rsc_op[@id='delay_node2_last_0']: @operation_key=delay_node2_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=18:107:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;18:107:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-chang Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/419, version=0.339.19) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.19 2 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.339.20 (null) Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=20 Jul 23 16:22:38 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']: S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.339.24 2 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.0 (null) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=340, @num_updates=0 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/configuration/resources/primitive[@id='apc_snmp_node1_an-pdu04']/instance_attributes[@id='apc_snmp_node1_an-pdu04-instance_attributes']/nvpair[@id='apc_snmp_node1_an-pdu04-instance_attributes-ip']: @value=10.201.2.4 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_replace_notify) info: Replaced: 0.339.24 -> 0.340.0 from an-a02n02 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_replace operation for section configuration: OK (rc=0, origin=an-a02n02/cibadmin/2, version=0.340.0) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_replaced_cb) notice: Updating all attributes after cib_refresh_notify event Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 107 aborted by apc_snmp_node1_an-pdu04-instance_attributes-ip doing modify ip=10.201.2.4: Configuration change | cib=0.340.0 source=te_update_diff_v2:474 path=/cib/configuration/resources/primitive[@id='apc_snmp_node1_an-pdu04']/instance_attributes[@id='apc_snmp_node1_an-pdu04-instance_attributes']/nvpair[@id='apc_snmp_node1_an-pdu04-instance_attributes-ip'] complete=true Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 57 with 2 changes for last-failure-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 58 with 2 changes for fail-count-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Processed 2 private changes for #attrd-protocol, id=n/a, set=n/a Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 59 with 2 changes for fail-count-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 60 with 2 changes for last-failure-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 61 with 2 changes for fail-count-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 62 with 2 changes for last-failure-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 63 with 2 changes for fail-count-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 64 with 2 changes for last-failure-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_ELECTION | input=I_ELECTION cause=C_FSA_INTERNAL origin=do_cib_replaced Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Unset DC. Was an-a02n01 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-fenced [121629] (update_cib_stonith_devices_v2) info: Updating device list from CIB: modify nvpair[@id='apc_snmp_node1_an-pdu04-instance_attributes-ip'] Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-fenced [121629] (cib_devices_update) info: Updating devices to version 0.340.0 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (election_check) info: election-DC won by local node Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (do_log) info: Input I_ELECTION_DC received in state S_ELECTION from election_win_cb Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) notice: State transition S_ELECTION -> S_INTEGRATION | input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=election_win_cb Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/425) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/426) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/57) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/58) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/59) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/60) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/61) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_device_register) notice: Added 'apc_snmp_node1_an-pdu04' to device list (8 active devices) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-30.raw Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.340.0 of the CIB to disk (digest: 779459e6f05c9b21dbb3bcd14e315945) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.5PEVCG (digest: /var/lib/pacemaker/cib/cib.FHgXwI) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/425, version=0.340.0) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.0 2 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.1 (null) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=1 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_cib_replaced Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_cib_replaced Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/426, version=0.340.1) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.1 2 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.2 (null) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=2 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/57, version=0.340.2) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 57 result for last-failure-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.2 2 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.3 (null) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=3 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/58, version=0.340.3) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_takeover) info: Taking over DC status for this partition Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 58 result for fail-count-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.3 2 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.4 (null) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=4 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/59, version=0.340.4) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 59 result for fail-count-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.4 2 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.5 (null) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=5 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/60, version=0.340.5) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 60 result for last-failure-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/61, version=0.340.5) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 61 result for fail-count-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=INFINITY Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=INFINITY Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_master operation for section 'all': OK (rc=0, origin=local/crmd/428, version=0.340.5) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/429) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/62) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/63) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/64) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/431) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=an-a02n01/crmd/429, version=0.340.5) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/62, version=0.340.5) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 62 result for last-failure-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=1627003034 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=1627082012 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/63, version=0.340.5) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 63 result for fail-count-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=INFINITY Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=INFINITY Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/64, version=0.340.5) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 64 result for last-failure-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=1627003035 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=1627082012 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/431, version=0.340.5) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/433) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-8 offer to an-a02n02 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-8 offer to an-a02n01 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_join_offer_all) info: Waiting on join-8 requests from 2 outstanding nodes Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/433, version=0.340.5) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/435) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/435, version=0.340.5) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Set DC to an-a02n01 (3.10.2) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_INTEGRATION -> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Digest matched on replace from an-a02n01: 563dcf7cb8af63f6a58a32a598202099 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Replaced 0.340.5 with 0.340.5 from an-a02n01 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=an-a02n01/crmd/438, version=0.340.5) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/439) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/440) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/439, version=0.340.5) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/440, version=0.340.5) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_delay_metadata_1[4157975] exited with status 0 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-31.raw Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.340.0 of the CIB to disk (digest: 779459e6f05c9b21dbb3bcd14e315945) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.FO7rMP (digest: /var/lib/pacemaker/cib/cib.AB0dMP) Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_apc_snmp_metadata_1[4157977] exited with status 0 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157979] exited with status 0 Jul 23 16:22:40 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157980] exited with status 0 Jul 23 16:22:41 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157981] exited with status 0 Jul 23 16:22:41 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4157982] exited with status 0 Jul 23 16:22:41 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157984] exited with status 0 Jul 23 16:22:41 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157985] exited with status 0 Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157986] exited with status 0 Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-controld [121633] (controld_delete_node_state) info: Deleting resource history for node an-a02n02 (via CIB call 441) | xpath=//node_state[@uname='an-a02n02']/lrm Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='an-a02n02']/lrm to all (origin=local/crmd/441) Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-controld [121633] (controld_delete_node_state) info: Deleting resource history for node an-a02n01 (via CIB call 443) | xpath=//node_state[@uname='an-a02n01']/lrm Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/442) Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='an-a02n01']/lrm to all (origin=local/crmd/443) Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.5 2 Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.6 (null) Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2'] Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=6 Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n02']/lrm: OK (rc=0, origin=an-a02n01/crmd/441, version=0.340.6) Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.6 2 Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.7 (null) Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=7 Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_lrm_query_internal Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/442, version=0.340.7) Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.7 2 Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.8 (null) Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/lrm[@id='1'] Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=8 Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n01']/lrm: OK (rc=0, origin=an-a02n01/crmd/443, version=0.340.8) Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_delay_metadata_1[4157987] exited with status 0 Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_apc_snmp_metadata_1[4157988] exited with status 0 Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157990] exited with status 0 Jul 23 16:22:42 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157991] exited with status 0 Jul 23 16:22:43 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157992] exited with status 0 Jul 23 16:22:43 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4157993] exited with status 0 Jul 23 16:22:43 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157995] exited with status 0 Jul 23 16:22:43 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157996] exited with status 0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4157997] exited with status 0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/444) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE | input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 107 aborted: Peer Cancelled | source=do_te_invoke:129 complete=true Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.8 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.9 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=9 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_lrm_query_internal Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/444, version=0.340.9) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/447) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/448) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/449) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 107 aborted: History refresh | source=process_resource_updates:268 complete=true Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/447, version=0.340.9) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.9 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.10 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=10 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/448, version=0.340.10) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=an-a02n01/crmd/449, version=0.340.10) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_digest_cmp) info: Parameters to 0ms-interval start action for apc_snmp_node1_an-pdu04 on an-a02n01 changed: hash was dcfe282eb9c3d7f7189c538ed5e16421 vs. now 953d17264336f1467fe1409a4ee17bd5 (reload:3.10.2) 0:1;11:2:0:789131c0-5164-4a52-9822-30c5026c3827 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe__clear_failcount) notice: Clearing failure of apc_snmp_node1_an-pdu04 on an-a02n01 because resource parameters have changed | apc_snmp_node1_an-pdu04_clear_failcount_0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_digest_cmp) info: Parameters to 0ms-interval start action for apc_snmp_node1_an-pdu04 on an-a02n02 changed: hash was dcfe282eb9c3d7f7189c538ed5e16421 vs. now 953d17264336f1467fe1409a4ee17bd5 (reload:3.10.2) 0:1;33:101:0:789131c0-5164-4a52-9822-30c5026c3827 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe__clear_failcount) notice: Clearing failure of apc_snmp_node1_an-pdu04 on an-a02n02 because resource parameters have changed | apc_snmp_node1_an-pdu04_clear_failcount_0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu04 cannot run anywhere Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu04 on an-a02n02 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for ipmilan_node2 on an-a02n01 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for ipmilan_node1 on an-a02n02 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu03 (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start apc_snmp_node1_an-pdu04 ( an-a02n02 ) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu03 (Started an-a02n02) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Stopped) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Move ipmilan_node2 ( an-a02n02 -> an-a02n01 ) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node2 (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Move ipmilan_node1 ( an-a02n01 -> an-a02n02 ) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n02) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 108, saving inputs in /var/lib/pacemaker/pengine/pe-input-129.bz2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (do_te_invoke) info: Processing graph 108 (ref=pe_calc-dc-1627082564-310) derived from /var/lib/pacemaker/pengine/pe-input-129.bz2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation apc_snmp_node1_an-pdu04_start_0 on an-a02n02 | action 17 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (te_crm_command) info: Executing crm-event (12) without waiting: clear_failcount on an-a02n02 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (te_crm_command) info: Executing crm-event (2) locally without waiting: clear_failcount on an-a02n01 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation ipmilan_node2_stop_0 on an-a02n02 | action 21 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation ipmilan_node1_stop_0 locally on an-a02n01 | action 26 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of stop operation for ipmilan_node1 on an-a02n01 | transition_key=26:108:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=ipmilan_node1_stop_0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:ipmilan_node1 action:stop call_id:129 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) info: Result of monitor operation for ipmilan_node1 on an-a02n01: Cancelled | call=109 key=ipmilan_node1_monitor_60000 confirmed=true Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/452) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-execd [121630] (log_finished) info: ipmilan_node1 stop (call 129) exited with status 0 (execution time 0ms, queue time 0ms) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) notice: Result of stop operation for ipmilan_node1 on an-a02n01: ok | rc=0 call=129 key=ipmilan_node1_stop_0 confirmed=true cib-update=453 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/453) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (update_attrd_clear_failures) info: Asking pacemaker-attrd to clear failure of all operations for apc_snmp_node1_an-pdu04 on cluster node an-a02n01 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section /cib/status/node_state[@uname='an-a02n01']/lrm/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_failure_0'] to all (origin=local/crmd/454) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.10 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.11 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=11 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='ipmilan_node1']/lrm_rsc_op[@id='ipmilan_node1_last_0']: @operation_key=ipmilan_node1_stop_0, @operation=stop, @crm-debug-origin=do_update_resource, @transition-key=26:108:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;26:108:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-c Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n01]: INFINITY -> (unset) | from an-a02n01 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/452, version=0.340.11) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 66 with 2 changes for fail-count-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n01]: 1627003035 -> (unset) | from an-a02n01 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.11 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.12 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=12 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 67 with 2 changes for last-failure-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='ipmilan_node1']/lrm_rsc_op[@id='ipmilan_node1_last_0']: @transition-magic=0:0;26:108:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=129, @rc-code=0, @op-status=0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/453, version=0.340.12) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 108 action 26 (ipmilan_node1_stop_0 on an-a02n01) confirmed: ok | rc=0 call-id=129 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation ipmilan_node1_start_0 on an-a02n02 | action 27 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n02]: INFINITY -> (unset) | from an-a02n02 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Write out of 'fail-count-apc_snmp_node1_an-pdu04#start_0' delayed: update 66 in progress Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n02]: 1627082012 -> (unset) | from an-a02n02 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Write out of 'last-failure-apc_snmp_node1_an-pdu04#start_0' delayed: update 67 in progress Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.12 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.13 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_failure_0'] Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=13 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section /cib/status/node_state[@uname='an-a02n01']/lrm/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_failure_0']: OK (rc=0, origin=an-a02n01/crmd/454, version=0.340.13) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) notice: Transition 108 aborted by deletion of lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_failure_0']: Resource operation removal | cib=0.340.13 source=process_op_deletion:358 path=/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_failure_0'] complete=false Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.13 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.14 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=14 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_0']: @operation_key=apc_snmp_node1_an-pdu04_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=17:108:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;17:108:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-cod Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/93, version=0.340.14) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/66) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/67) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.14 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.15 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-fail-count-apc_snmp_node1_an-pdu04.start_0'] Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=15 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/66, version=0.340.15) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 66 result for fail-count-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=INFINITY Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 108 aborted by deletion of nvpair[@id='status-1-fail-count-apc_snmp_node1_an-pdu04.start_0']: Transient attribute change | cib=0.340.15 source=abort_unless_down:327 path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-fail-count-apc_snmp_node1_an-pdu04.start_0'] complete=false Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 68 with 2 changes for fail-count-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.15 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.16 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-last-failure-apc_snmp_node1_an-pdu04.start_0'] Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=16 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/67, version=0.340.16) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 108 aborted by deletion of nvpair[@id='status-1-last-failure-apc_snmp_node1_an-pdu04.start_0']: Transient attribute change | cib=0.340.16 source=abort_unless_down:327 path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-last-failure-apc_snmp_node1_an-pdu04.start_0'] complete=false Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 67 result for last-failure-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=1627082012 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 69 with 2 changes for last-failure-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.16 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.17 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_failure_0'] Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=17 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section /cib/status/node_state[@uname='an-a02n02']/lrm/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_failure_0']: OK (rc=0, origin=an-a02n02/crmd/94, version=0.340.17) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 108 aborted by deletion of lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_failure_0']: Resource operation removal | cib=0.340.17 source=process_op_deletion:358 path=/cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_failure_0'] complete=false Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.17 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.18 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=18 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='ipmilan_node2']/lrm_rsc_op[@id='ipmilan_node2_last_0']: @operation_key=ipmilan_node2_stop_0, @operation=stop, @crm-debug-origin=do_update_resource, @transition-key=21:108:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;21:108:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-c Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/95, version=0.340.18) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.18 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.19 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=19 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='ipmilan_node1']/lrm_rsc_op[@id='ipmilan_node1_last_0']: @operation_key=ipmilan_node1_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=27:108:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;27:108:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/96, version=0.340.19) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.19 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.20 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=20 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='ipmilan_node2']/lrm_rsc_op[@id='ipmilan_node2_last_0']: @transition-magic=0:0;21:108:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=102, @rc-code=0, @op-status=0, @exec-time=1 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/97, version=0.340.20) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 108 action 21 (ipmilan_node2_stop_0 on an-a02n02) confirmed: ok | rc=0 call-id=102 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/68) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/69) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.20 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.21 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-fail-count-apc_snmp_node1_an-pdu04.start_0'] Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=21 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/68, version=0.340.21) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 68 result for fail-count-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=(null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 108 aborted by deletion of nvpair[@id='status-2-fail-count-apc_snmp_node1_an-pdu04.start_0']: Transient attribute change | cib=0.340.21 source=abort_unless_down:327 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-fail-count-apc_snmp_node1_an-pdu04.start_0'] complete=false Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.21 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.22 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-last-failure-apc_snmp_node1_an-pdu04.start_0'] Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=22 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/69, version=0.340.22) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 69 result for last-failure-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=(null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 108 aborted by deletion of nvpair[@id='status-2-last-failure-apc_snmp_node1_an-pdu04.start_0']: Transient attribute change | cib=0.340.22 source=abort_unless_down:327 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-last-failure-apc_snmp_node1_an-pdu04.start_0'] complete=false Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.22 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.23 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=23 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='ipmilan_node1']/lrm_rsc_op[@id='ipmilan_node1_last_0']: @transition-magic=0:0;27:108:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=103, @rc-code=0, @op-status=0, @exec-time=284 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/98, version=0.340.23) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 108 action 27 (ipmilan_node1_start_0 on an-a02n02) confirmed: ok | rc=0 call-id=103 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.23 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.24 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=24 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_0']: @transition-magic=0:0;17:108:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=100, @rc-code=0, @op-status=0, @exec-time=376 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/99, version=0.340.24) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 108 action 17 (apc_snmp_node1_an-pdu04_start_0 on an-a02n02) confirmed: ok | rc=0 call-id=100 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (run_graph) notice: Transition 108 (Complete=6, Pending=0, Fired=0, Skipped=3, Incomplete=4, Source=/var/lib/pacemaker/pengine/pe-input-129.bz2): Stopped Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n01 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node2_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu04 cannot run anywhere Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu04 on an-a02n02 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for ipmilan_node2 on an-a02n01 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for ipmilan_node1 on an-a02n02 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu03 (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu04 (Started an-a02n02) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu03 (Started an-a02n02) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Stopped) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start ipmilan_node2 ( an-a02n01 ) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node2 (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n02) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n02) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 109, saving inputs in /var/lib/pacemaker/pengine/pe-input-130.bz2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (do_te_invoke) info: Processing graph 109 (ref=pe_calc-dc-1627082564-317) derived from /var/lib/pacemaker/pengine/pe-input-130.bz2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation apc_snmp_node1_an-pdu04_monitor_60000 on an-a02n02 | action 15 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation ipmilan_node2_start_0 locally on an-a02n01 | action 18 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of start operation for ipmilan_node2 on an-a02n01 | transition_key=18:109:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=ipmilan_node2_start_0 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:ipmilan_node2 action:start call_id:130 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation ipmilan_node1_monitor_60000 on an-a02n02 | action 24 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/456) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.24 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.25 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=25 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='ipmilan_node2']/lrm_rsc_op[@id='ipmilan_node2_last_0']: @operation_key=ipmilan_node2_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=18:109:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;18:109:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/456, version=0.340.25) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.25 2 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.340.26 (null) Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=26 Jul 23 16:22:44 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']: S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.340.32 2 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.341.0 (null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=341, @num_updates=0 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/configuration/resources/primitive[@id='apc_snmp_node2_an-pdu04']/instance_attributes[@id='apc_snmp_node2_an-pdu04-instance_attributes']/nvpair[@id='apc_snmp_node2_an-pdu04-instance_attributes-ip']: @value=10.201.2.4 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_replace_notify) info: Replaced: 0.340.32 -> 0.341.0 from an-a02n02 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_replace operation for section configuration: OK (rc=0, origin=an-a02n02/cibadmin/2, version=0.341.0) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_replaced_cb) notice: Updating all attributes after cib_refresh_notify event Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 70 with 2 changes for last-failure-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 71 with 2 changes for fail-count-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Processed 2 private changes for #attrd-protocol, id=n/a, set=n/a Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/70) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 72 with 2 changes for fail-count-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 109 aborted by apc_snmp_node2_an-pdu04-instance_attributes-ip doing modify ip=10.201.2.4: Configuration change | cib=0.341.0 source=te_update_diff_v2:474 path=/cib/configuration/resources/primitive[@id='apc_snmp_node2_an-pdu04']/instance_attributes[@id='apc_snmp_node2_an-pdu04-instance_attributes']/nvpair[@id='apc_snmp_node2_an-pdu04-instance_attributes-ip'] complete=true Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 73 with 2 changes for last-failure-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/71) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 74 with 2 changes for fail-count-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/72) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 75 with 2 changes for last-failure-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 76 with 2 changes for fail-count-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/73) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 77 with 2 changes for last-failure-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/74) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/75) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_ELECTION | input=I_ELECTION cause=C_FSA_INTERNAL origin=do_cib_replaced Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Unset DC. Was an-a02n01 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.341.0 2 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.341.1 (null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=1 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-fenced [121629] (update_cib_stonith_devices_v2) info: Updating device list from CIB: modify nvpair[@id='apc_snmp_node2_an-pdu04-instance_attributes-ip'] Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-fenced [121629] (cib_devices_update) info: Updating devices to version 0.341.0 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/70, version=0.341.1) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 70 result for last-failure-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.341.1 2 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.341.2 (null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=2 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/71, version=0.341.2) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 71 result for fail-count-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.341.2 2 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.341.3 (null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=3 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (election_check) info: election-DC won by local node Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (do_log) info: Input I_ELECTION_DC received in state S_ELECTION from election_win_cb Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) notice: State transition S_ELECTION -> S_INTEGRATION | input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=election_win_cb Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/72, version=0.341.3) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 72 result for fail-count-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.341.3 2 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.341.4 (null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=4 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/73, version=0.341.4) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 73 result for last-failure-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/74, version=0.341.4) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 74 result for fail-count-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=INFINITY Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=INFINITY Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/75, version=0.341.4) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 75 result for last-failure-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=1627003034 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=1627082012 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/462) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/463) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_device_register) notice: Added 'apc_snmp_node2_an-pdu04' to device list (8 active devices) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/76) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/77) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/462, version=0.341.4) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.341.4 2 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.341.5 (null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=5 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_cib_replaced Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_cib_replaced Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/463, version=0.341.5) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.341.5 2 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.341.6 (null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=6 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/76, version=0.341.6) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 76 result for fail-count-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=(null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.341.6 2 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.341.7 (null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=7 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/77, version=0.341.7) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 77 result for last-failure-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=(null) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_takeover) info: Taking over DC status for this partition Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-32.raw Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.341.0 of the CIB to disk (digest: fdcb7a1f3a0d9dd650bdfae41a9f478b) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.1W4n42 (digest: /var/lib/pacemaker/cib/cib.9zkawZ) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_master operation for section 'all': OK (rc=0, origin=local/crmd/465, version=0.341.7) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/466) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/468) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/470) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/472) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-9 offer to an-a02n02 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-9 offer to an-a02n01 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_join_offer_all) info: Waiting on join-9 requests from 2 outstanding nodes Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=an-a02n01/crmd/466, version=0.341.7) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/468, version=0.341.7) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/470, version=0.341.7) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Set DC to an-a02n01 (3.10.2) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/472, version=0.341.7) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_INTEGRATION -> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Digest matched on replace from an-a02n01: 0226e2d0923cf1c7328bdd2bb69480ba Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Replaced 0.341.7 with 0.341.7 from an-a02n01 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=an-a02n01/crmd/475, version=0.341.7) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/476) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/477) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-33.raw Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/476, version=0.341.7) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.341.0 of the CIB to disk (digest: fdcb7a1f3a0d9dd650bdfae41a9f478b) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.wDOXTd (digest: /var/lib/pacemaker/cib/cib.5aCr2e) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/477, version=0.341.7) Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_delay_metadata_1[4158613] exited with status 0 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_apc_snmp_metadata_1[4158623] exited with status 0 Jul 23 16:22:46 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158663] exited with status 0 Jul 23 16:22:47 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158678] exited with status 0 Jul 23 16:22:47 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158688] exited with status 0 Jul 23 16:22:47 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4158887] exited with status 0 Jul 23 16:22:47 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158898] exited with status 0 Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158908] exited with status 0 Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158909] exited with status 0 Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-controld [121633] (controld_delete_node_state) info: Deleting resource history for node an-a02n02 (via CIB call 478) | xpath=//node_state[@uname='an-a02n02']/lrm Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='an-a02n02']/lrm to all (origin=local/crmd/478) Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-controld [121633] (controld_delete_node_state) info: Deleting resource history for node an-a02n01 (via CIB call 480) | xpath=//node_state[@uname='an-a02n01']/lrm Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/479) Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='an-a02n01']/lrm to all (origin=local/crmd/480) Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.341.7 2 Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.341.8 (null) Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2'] Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=8 Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n02']/lrm: OK (rc=0, origin=an-a02n01/crmd/478, version=0.341.8) Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.341.8 2 Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.341.9 (null) Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=9 Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_lrm_query_internal Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/479, version=0.341.9) Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.341.9 2 Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.341.10 (null) Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/lrm[@id='1'] Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=10 Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n01']/lrm: OK (rc=0, origin=an-a02n01/crmd/480, version=0.341.10) Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_delay_metadata_1[4158910] exited with status 0 Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_apc_snmp_metadata_1[4158911] exited with status 0 Jul 23 16:22:48 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158913] exited with status 0 Jul 23 16:22:49 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158914] exited with status 0 Jul 23 16:22:49 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158917] exited with status 0 Jul 23 16:22:49 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4158919] exited with status 0 Jul 23 16:22:49 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158921] exited with status 0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.341.10 2 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.342.0 (null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=342, @num_updates=0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/configuration/fencing-topology/fencing-level[@id='fl-an-a02n01-2']: @devices=apc_snmp_node1_an-pdu04 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_replace_notify) info: Replaced: 0.341.10 -> 0.342.0 from an-a02n02 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_replace operation for section configuration: OK (rc=0, origin=an-a02n02/cibadmin/2, version=0.342.0) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_replaced_cb) notice: Updating all attributes after cib_refresh_notify event Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 79 with 2 changes for last-failure-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_level_remove) info: Removed level 2 from fencing topology for an-a02n01 (2 active levels remaining) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 80 with 2 changes for fail-count-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Processed 2 private changes for #attrd-protocol, id=n/a, set=n/a Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_level_register) info: Target an-a02n01 has 3 active fencing levels Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 81 with 2 changes for fail-count-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 82 with 2 changes for last-failure-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 83 with 2 changes for fail-count-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 84 with 2 changes for last-failure-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 85 with 2 changes for fail-count-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 86 with 2 changes for last-failure-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/79) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/80) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/81) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/82) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/83) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/84) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.342.0 2 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.342.1 (null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=1 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/79, version=0.342.1) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 79 result for last-failure-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.342.1 2 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.342.2 (null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=2 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-34.raw Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/80, version=0.342.2) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 80 result for fail-count-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.342.2 2 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.342.3 (null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=3 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/81, version=0.342.3) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 81 result for fail-count-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.342.3 2 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.342.4 (null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=4 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/82, version=0.342.4) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 82 result for last-failure-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/83, version=0.342.4) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 83 result for fail-count-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=INFINITY Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=INFINITY Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/84, version=0.342.4) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 84 result for last-failure-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=1627003034 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=1627082012 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/85) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/86) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.342.0 of the CIB to disk (digest: bb364a28474464cfeca75e0f497d0f7e) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.342.4 2 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.342.5 (null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=5 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/85, version=0.342.5) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.Pp7RLB (digest: /var/lib/pacemaker/cib/cib.furjPz) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 85 result for fail-count-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=(null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.342.5 2 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.342.6 (null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=6 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/86, version=0.342.6) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 86 result for last-failure-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=(null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.342.6 2 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.343.0 (null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/configuration/resources/primitive[@id='apc_snmp_node1_an-pdu03'] Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=343, @num_updates=0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //primitive[@id='apc_snmp_node1_an-pdu03']: OK (rc=0, origin=an-a02n02/cibadmin/2, version=0.343.0) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-35.raw Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.343.0 of the CIB to disk (digest: 7a8ef0f45eadab4fdb5aede58f233e19) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.uSfL8J (digest: /var/lib/pacemaker/cib/cib.8ZRYbK) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158924] exited with status 0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158941] exited with status 0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/481) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE | input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 109 aborted: Peer Cancelled | source=do_te_invoke:129 complete=true Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_ELECTION | input=I_ELECTION cause=C_FSA_INTERNAL origin=do_cib_replaced Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Unset DC. Was an-a02n01 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.343.0 2 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.343.1 (null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=1 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_lrm_query_internal Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (election_check) info: election-DC won by local node Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (do_log) info: Input I_ELECTION_DC received in state S_ELECTION from election_win_cb Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) notice: State transition S_ELECTION -> S_INTEGRATION | input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=election_win_cb Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/481, version=0.343.1) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/484) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/485) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/488) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/489) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/490) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/484, version=0.343.1) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.343.1 2 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.343.2 (null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=2 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_cib_replaced Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_cib_replaced Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/485, version=0.343.2) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/488, version=0.343.2) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.343.2 2 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.343.3 (null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=3 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/489, version=0.343.3) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=an-a02n01/crmd/490, version=0.343.3) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_takeover) info: Taking over DC status for this partition Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_master operation for section 'all': OK (rc=0, origin=local/crmd/491, version=0.343.3) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/492) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/494) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/496) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/498) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-10 offer to an-a02n02 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-10 offer to an-a02n01 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_join_offer_all) info: Waiting on join-10 requests from 2 outstanding nodes Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Set DC to an-a02n01 (3.10.2) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=an-a02n01/crmd/492, version=0.343.3) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/494, version=0.343.3) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/496, version=0.343.3) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/498, version=0.343.3) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_INTEGRATION -> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Digest matched on replace from an-a02n01: cb8789aaa7c654b5cdbe44da82772e05 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Replaced 0.343.3 with 0.343.3 from an-a02n01 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=an-a02n01/crmd/501, version=0.343.3) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/502) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/503) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-36.raw Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/502, version=0.343.3) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.343.0 of the CIB to disk (digest: 7a8ef0f45eadab4fdb5aede58f233e19) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.0ptvJV (digest: /var/lib/pacemaker/cib/cib.EFJHEX) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/503, version=0.343.3) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_delay_metadata_1[4158942] exited with status 0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.343.3 2 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.344.0 (null) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=344, @num_updates=0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/configuration/resources: Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=an-a02n02/cibadmin/2, version=0.344.0) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-fenced [121629] (update_cib_stonith_devices_v2) info: Updating device list from CIB: create resources Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-fenced [121629] (cib_devices_update) info: Updating devices to version 0.344.0 Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_device_register) info: Overwriting existing entry for apc_snmp_node1_an-pdu03 from CIB Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_device_register) notice: Added 'apc_snmp_node1_an-pdu03' to device list (8 active devices) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-37.raw Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.344.0 of the CIB to disk (digest: 0bb3754bf209f5091faf20d077fd54ea) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.XkjLF5 (digest: /var/lib/pacemaker/cib/cib.WBBzJ6) Jul 23 16:22:50 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_apc_snmp_metadata_1[4158944] exited with status 0 Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158947] exited with status 0 Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.344.0 2 Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.0 (null) Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=345 Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/configuration/resources/primitive[@id='apc_snmp_node1_an-pdu04']: Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=an-a02n02/cibadmin/2, version=0.345.0) Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-fenced [121629] (update_cib_stonith_devices_v2) info: Updating device list from CIB: create primitive[@id='apc_snmp_node1_an-pdu04'] Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-fenced [121629] (cib_devices_update) info: Updating devices to version 0.345.0 Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-fenced [121629] (cib_device_update) info: Device apc_snmp_node1_an-pdu04 has been disabled Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-38.raw Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.345.0 of the CIB to disk (digest: 2fd8a84ca2e0bff215c4b12561f5f467) Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.gw3mEo (digest: /var/lib/pacemaker/cib/cib.OL6ttp) Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158948] exited with status 0 Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158950] exited with status 0 Jul 23 16:22:51 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4158963] exited with status 0 Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158965] exited with status 0 Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158967] exited with status 0 Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158975] exited with status 0 Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-controld [121633] (controld_delete_node_state) info: Deleting resource history for node an-a02n02 (via CIB call 504) | xpath=//node_state[@uname='an-a02n02']/lrm Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='an-a02n02']/lrm to all (origin=local/crmd/504) Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/505) Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-controld [121633] (controld_delete_node_state) info: Deleting resource history for node an-a02n01 (via CIB call 506) | xpath=//node_state[@uname='an-a02n01']/lrm Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.0 2 Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.1 (null) Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2'] Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=1 Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n02']/lrm: OK (rc=0, origin=an-a02n01/crmd/504, version=0.345.1) Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.1 2 Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.2 (null) Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=2 Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_lrm_query_internal Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/505, version=0.345.2) Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='an-a02n01']/lrm to all (origin=local/crmd/506) Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.2 2 Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.3 (null) Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/lrm[@id='1'] Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=3 Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n01']/lrm: OK (rc=0, origin=an-a02n01/crmd/506, version=0.345.3) Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_delay_metadata_1[4158976] exited with status 0 Jul 23 16:22:52 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_apc_snmp_metadata_1[4158977] exited with status 0 Jul 23 16:22:53 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158979] exited with status 0 Jul 23 16:22:53 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158980] exited with status 0 Jul 23 16:22:53 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158981] exited with status 0 Jul 23 16:22:53 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4158982] exited with status 0 Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158984] exited with status 0 Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4158999] exited with status 0 Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4159001] exited with status 0 Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/507) Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE | input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 109 aborted: Peer Cancelled | source=do_te_invoke:129 complete=true Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.3 2 Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.4 (null) Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=4 Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_lrm_query_internal Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/507, version=0.345.4) Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 109 aborted: History refresh | source=process_resource_updates:268 complete=true Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/510) Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/511) Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/512) Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/510, version=0.345.4) Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.4 2 Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.5 (null) Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=5 Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/511, version=0.345.5) Jul 23 16:22:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=an-a02n01/crmd/512, version=0.345.5) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_digest_cmp) info: Parameters to 0ms-interval start action for apc_snmp_node2_an-pdu04 on an-a02n01 changed: hash was 09508b690d4ef39a75f68a8014ee9848 vs. now 25e19c24d758eebc99cf83d00d559588 (reload:3.10.2) 0:1;15:2:0:789131c0-5164-4a52-9822-30c5026c3827 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe__clear_failcount) notice: Clearing failure of apc_snmp_node2_an-pdu04 on an-a02n01 because resource parameters have changed | apc_snmp_node2_an-pdu04_clear_failcount_0 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 22 18:17:00 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_digest_cmp) info: Parameters to 0ms-interval start action for apc_snmp_node2_an-pdu04 on an-a02n02 changed: hash was 09508b690d4ef39a75f68a8014ee9848 vs. now 25e19c24d758eebc99cf83d00d559588 (reload:3.10.2) 0:1;37:101:0:789131c0-5164-4a52-9822-30c5026c3827 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe__clear_failcount) notice: Clearing failure of apc_snmp_node2_an-pdu04 on an-a02n02 because resource parameters have changed | apc_snmp_node2_an-pdu04_clear_failcount_0 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:13:19 2021 | rc=1 id=apc_snmp_node2_an-pdu04_last_failure_0 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_digest_cmp) info: Parameters to 0ms-interval start action for apc_snmp_node1_an-pdu03 on an-a02n01 changed: hash was b6d660e5416057f7575eac6369636a4f vs. now 3fd89635ef735220dd13344790fbf853 (reload:3.10.2) 0:0;13:104:0:789131c0-5164-4a52-9822-30c5026c3827 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_action_definition) info: params:reload Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_digest_cmp) info: Parameters to 60000ms-interval monitor action for apc_snmp_node1_an-pdu03 on an-a02n01 changed: hash was 5e7bff0e01ad15b6456cdc9f08b6b6ff vs. now df3699fd58da6b53bb2ebd50f24d2a95 (reschedule:3.10.2) 0:0;12:105:0:789131c0-5164-4a52-9822-30c5026c3827 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_action_definition) info: params:reload Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu04 cannot run anywhere Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node2_an-pdu04 on an-a02n01 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for delay_node2 on an-a02n02 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for delay_node1 on an-a02n01 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu03 on an-a02n02 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Stop apc_snmp_node1_an-pdu04 ( an-a02n02 ) due to node availability Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu03 (Started an-a02n02) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start apc_snmp_node2_an-pdu04 ( an-a02n01 ) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node2 (Started an-a02n01) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Move delay_node2 ( an-a02n01 -> an-a02n02 ) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n02) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Move delay_node1 ( an-a02n02 -> an-a02n01 ) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Move apc_snmp_node1_an-pdu03 ( an-a02n01 -> an-a02n02 ) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 110, saving inputs in /var/lib/pacemaker/pengine/pe-input-131.bz2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (do_te_invoke) info: Processing graph 110 (ref=pe_calc-dc-1627082575-336) derived from /var/lib/pacemaker/pengine/pe-input-131.bz2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation apc_snmp_node1_an-pdu04_stop_0 on an-a02n02 | action 17 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation apc_snmp_node2_an-pdu04_start_0 locally on an-a02n01 | action 20 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of start operation for apc_snmp_node2_an-pdu04 on an-a02n01 | transition_key=20:110:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=apc_snmp_node2_an-pdu04_start_0 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (te_crm_command) info: Executing crm-event (15) without waiting: clear_failcount on an-a02n02 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:apc_snmp_node2_an-pdu04 action:start call_id:132 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (te_crm_command) info: Executing crm-event (8) locally without waiting: clear_failcount on an-a02n01 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/516) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation delay_node2_stop_0 locally on an-a02n01 | action 24 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of stop operation for delay_node2 on an-a02n01 | transition_key=24:110:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=delay_node2_stop_0 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:delay_node2 action:stop call_id:134 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation delay_node1_stop_0 on an-a02n02 | action 29 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation apc_snmp_node1_an-pdu03_stop_0 locally on an-a02n01 | action 44 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.5 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.6 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=6 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_0']: @operation_key=apc_snmp_node2_an-pdu04_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=20:110:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;20:110:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-cod Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/516, version=0.345.6) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/517) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.6 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.7 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=7 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='delay_node2']/lrm_rsc_op[@id='delay_node2_last_0']: @operation_key=delay_node2_stop_0, @operation=stop, @crm-debug-origin=do_update_resource, @transition-key=24:110:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;24:110:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change= Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/517, version=0.345.7) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.7 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.8 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=8 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_0']: @operation_key=apc_snmp_node1_an-pdu04_stop_0, @operation=stop, @crm-debug-origin=do_update_resource, @transition-key=17:110:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;17:110:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code= Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n02]: INFINITY -> (unset) | from an-a02n02 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 88 with 2 changes for fail-count-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n02]: 1627082012 -> (unset) | from an-a02n02 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/108, version=0.345.8) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 89 with 2 changes for last-failure-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.8 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.9 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=9 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_0']: @transition-magic=0:0;17:110:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=107, @rc-code=0, @op-status=0 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/109, version=0.345.9) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/88) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/89) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.9 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.10 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-fail-count-apc_snmp_node2_an-pdu04.start_0'] Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=10 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/88, version=0.345.10) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 88 result for fail-count-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=INFINITY Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=(null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.10 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.11 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-last-failure-apc_snmp_node2_an-pdu04.start_0'] Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=11 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/89, version=0.345.11) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 89 result for last-failure-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=1627003034 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=(null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-execd [121630] (log_finished) info: delay_node2 stop (call 134) exited with status 0 (execution time 2ms, queue time 0ms) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of stop operation for apc_snmp_node1_an-pdu03 on an-a02n01 | transition_key=44:110:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=apc_snmp_node1_an-pdu03_stop_0 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:apc_snmp_node1_an-pdu03 action:stop call_id:136 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.11 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.12 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_failure_0'] Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=12 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-execd [121630] (log_finished) info: apc_snmp_node1_an-pdu03 stop (call 136) exited with status 0 (execution time 0ms, queue time 0ms) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 110 action 17 (apc_snmp_node1_an-pdu04_stop_0 on an-a02n02) confirmed: ok | rc=0 call-id=107 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) notice: Transition 110 aborted by deletion of nvpair[@id='status-2-fail-count-apc_snmp_node2_an-pdu04.start_0']: Transient attribute change | cib=0.345.10 source=abort_unless_down:327 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-fail-count-apc_snmp_node2_an-pdu04.start_0'] complete=false Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 110 aborted by deletion of nvpair[@id='status-2-last-failure-apc_snmp_node2_an-pdu04.start_0']: Transient attribute change | cib=0.345.11 source=abort_unless_down:327 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-last-failure-apc_snmp_node2_an-pdu04.start_0'] complete=false Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) info: Result of monitor operation for delay_node2 on an-a02n01: Cancelled | call=127 key=delay_node2_monitor_60000 confirmed=true Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) notice: Result of stop operation for delay_node2 on an-a02n01: ok | rc=0 call=134 key=delay_node2_stop_0 confirmed=true cib-update=519 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) info: Result of monitor operation for apc_snmp_node1_an-pdu03 on an-a02n01: Cancelled | call=125 key=apc_snmp_node1_an-pdu03_monitor_60000 confirmed=true Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) notice: Result of stop operation for apc_snmp_node1_an-pdu03 on an-a02n01: ok | rc=0 call=136 key=apc_snmp_node1_an-pdu03_stop_0 confirmed=true cib-update=520 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (update_attrd_clear_failures) info: Asking pacemaker-attrd to clear failure of all operations for apc_snmp_node2_an-pdu04 on cluster node an-a02n01 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n01]: INFINITY -> (unset) | from an-a02n01 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 90 with 2 changes for fail-count-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section /cib/status/node_state[@uname='an-a02n02']/lrm/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_failure_0']: OK (rc=0, origin=an-a02n02/crmd/110, version=0.345.12) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n01]: 1627003034 -> (unset) | from an-a02n01 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 91 with 2 changes for last-failure-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 110 aborted by deletion of lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_failure_0']: Resource operation removal | cib=0.345.12 source=process_op_deletion:358 path=/cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_failure_0'] complete=false Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.12 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.13 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=13 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='delay_node1']/lrm_rsc_op[@id='delay_node1_last_0']: @operation_key=delay_node1_stop_0, @operation=stop, @crm-debug-origin=do_update_resource, @transition-key=29:110:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;29:110:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change= Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/111, version=0.345.13) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.13 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.14 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=14 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='delay_node1']/lrm_rsc_op[@id='delay_node1_last_0']: @transition-magic=0:0;29:110:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=109, @rc-code=0, @op-status=0 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/112, version=0.345.14) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 110 action 29 (delay_node1_stop_0 on an-a02n02) confirmed: ok | rc=0 call-id=109 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/518) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/519) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/520) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section /cib/status/node_state[@uname='an-a02n01']/lrm/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_failure_0'] to all (origin=local/crmd/521) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/90) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/91) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.14 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.15 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=15 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @operation_key=apc_snmp_node1_an-pdu03_stop_0, @operation=stop, @crm-debug-origin=do_update_resource, @transition-key=44:110:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;44:110:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code= Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/518, version=0.345.15) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.15 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.16 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=16 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='delay_node2']/lrm_rsc_op[@id='delay_node2_last_0']: @transition-magic=0:0;24:110:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=134, @rc-code=0, @op-status=0, @exec-time=2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/519, version=0.345.16) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 110 action 24 (delay_node2_stop_0 on an-a02n01) confirmed: ok | rc=0 call-id=134 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.16 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.17 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=17 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @transition-magic=0:0;44:110:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=136, @rc-code=0, @op-status=0 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/520, version=0.345.17) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 110 action 44 (apc_snmp_node1_an-pdu03_stop_0 on an-a02n01) confirmed: ok | rc=0 call-id=136 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.17 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.18 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_failure_0'] Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=18 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section /cib/status/node_state[@uname='an-a02n01']/lrm/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_failure_0']: OK (rc=0, origin=an-a02n01/crmd/521, version=0.345.18) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 110 aborted by deletion of lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_failure_0']: Resource operation removal | cib=0.345.18 source=process_op_deletion:358 path=/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_failure_0'] complete=false Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.18 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.19 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-fail-count-apc_snmp_node2_an-pdu04.start_0'] Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=19 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/90, version=0.345.19) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 90 result for fail-count-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=(null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 110 aborted by deletion of nvpair[@id='status-1-fail-count-apc_snmp_node2_an-pdu04.start_0']: Transient attribute change | cib=0.345.19 source=abort_unless_down:327 path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-fail-count-apc_snmp_node2_an-pdu04.start_0'] complete=false Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.19 2 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.20 (null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-last-failure-apc_snmp_node2_an-pdu04.start_0'] Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=20 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/91, version=0.345.20) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 91 result for last-failure-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=(null) Jul 23 16:22:55 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 110 aborted by deletion of nvpair[@id='status-1-last-failure-apc_snmp_node2_an-pdu04.start_0']: Transient attribute change | cib=0.345.20 source=abort_unless_down:327 path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-last-failure-apc_snmp_node2_an-pdu04.start_0'] complete=false Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-execd [121630] (log_finished) info: apc_snmp_node2_an-pdu04 start (call 132) exited with status 0 (execution time 331ms, queue time 0ms) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_apc_snmp_metadata_1[4159019] exited with status 0 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) notice: Result of start operation for apc_snmp_node2_an-pdu04 on an-a02n01: ok | rc=0 call=132 key=apc_snmp_node2_an-pdu04_start_0 confirmed=true cib-update=522 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/522) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.20 2 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.21 (null) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=21 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu04_last_0']: @transition-magic=0:0;20:110:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=132, @rc-code=0, @op-status=0, @exec-time=331 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/522, version=0.345.21) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 110 action 20 (apc_snmp_node2_an-pdu04_start_0 on an-a02n01) confirmed: ok | rc=0 call-id=132 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-controld [121633] (run_graph) notice: Transition 110 (Complete=7, Pending=0, Fired=0, Skipped=4, Incomplete=7, Source=/var/lib/pacemaker/pengine/pe-input-131.bz2): Stopped Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu04 cannot run anywhere Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node2_an-pdu04 on an-a02n01 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for delay_node2 on an-a02n02 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for delay_node1 on an-a02n01 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu03 on an-a02n02 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu04 (Stopped) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu03 (Started an-a02n02) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Started an-a02n01) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node2 (Started an-a02n01) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start delay_node2 ( an-a02n02 ) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n02) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start delay_node1 ( an-a02n01 ) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start apc_snmp_node1_an-pdu03 ( an-a02n02 ) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 111, saving inputs in /var/lib/pacemaker/pengine/pe-input-132.bz2 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-controld [121633] (do_te_invoke) info: Processing graph 111 (ref=pe_calc-dc-1627082576-344) derived from /var/lib/pacemaker/pengine/pe-input-132.bz2 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation apc_snmp_node2_an-pdu04_monitor_60000 locally on an-a02n01 | action 14 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of monitor operation for apc_snmp_node2_an-pdu04 on an-a02n01 | transition_key=14:111:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=apc_snmp_node2_an-pdu04_monitor_60000 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation delay_node2_start_0 on an-a02n02 | action 17 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation delay_node1_start_0 locally on an-a02n01 | action 21 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/524) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of start operation for delay_node1 on an-a02n01 | transition_key=21:111:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=delay_node1_start_0 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/525) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:delay_node1 action:start call_id:138 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation apc_snmp_node1_an-pdu03_start_0 on an-a02n02 | action 35 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.21 2 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.22 (null) Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=22 Jul 23 16:22:56 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu04']: Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/92, version=0.345.34) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 92 result for fail-count-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=INFINITY Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.34 2 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.35 (null) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=35 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/93, version=0.345.35) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 93 result for last-failure-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=1627082589 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 111 aborted by status-2-fail-count-apc_snmp_node1_an-pdu03.start_0 doing create fail-count-apc_snmp_node1_an-pdu03#start_0=INFINITY: Transient attribute change | cib=0.345.34 source=abort_unless_down:313 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=true Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 111 aborted by status-2-last-failure-apc_snmp_node1_an-pdu03.start_0 doing create last-failure-apc_snmp_node1_an-pdu03#start_0=1627082589: Transient attribute change | cib=0.345.35 source=abort_unless_down:313 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=true Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:22:56 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:22:56 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_0 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu04 cannot run anywhere Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu03 on an-a02n02 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu04 (Stopped) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu03 (Started an-a02n02) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node2 (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node2 (Started an-a02n02) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n02) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Recover apc_snmp_node1_an-pdu03 ( an-a02n02 ) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 112, saving inputs in /var/lib/pacemaker/pengine/pe-input-133.bz2 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-controld [121633] (handle_response) info: pe_calc calculation pe_calc-dc-1627082589-351 is obsolete Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:22:56 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:22:56 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_0 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu03 has failed INFINITY times on an-a02n02 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu04 cannot run anywhere Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu03 on an-a02n01 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu04 (Stopped) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu03 (Started an-a02n02) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node2 (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node2 (Started an-a02n02) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n02) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Recover apc_snmp_node1_an-pdu03 ( an-a02n02 -> an-a02n01 ) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 113, saving inputs in /var/lib/pacemaker/pengine/pe-input-134.bz2 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-controld [121633] (do_te_invoke) info: Processing graph 113 (ref=pe_calc-dc-1627082589-352) derived from /var/lib/pacemaker/pengine/pe-input-134.bz2 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation apc_snmp_node1_an-pdu03_stop_0 on an-a02n02 | action 11 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.35 2 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.36 (null) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=36 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @operation_key=apc_snmp_node1_an-pdu03_stop_0, @operation=stop, @transition-key=11:113:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;11:113:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=16 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/119, version=0.345.36) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.36 2 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.37 (null) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=37 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @transition-magic=0:0;11:113:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=113, @rc-code=0, @op-status=0 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/120, version=0.345.37) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 113 action 11 (apc_snmp_node1_an-pdu03_stop_0 on an-a02n02) confirmed: ok | rc=0 call-id=113 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation apc_snmp_node1_an-pdu03_start_0 locally on an-a02n01 | action 38 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of start operation for apc_snmp_node1_an-pdu03 on an-a02n01 | transition_key=38:113:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=apc_snmp_node1_an-pdu03_start_0 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/533) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:apc_snmp_node1_an-pdu03 action:start call_id:140 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.37 2 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.38 (null) Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=38 Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @operation_key=apc_snmp_node1_an-pdu03_start_0, @operation=start, @transition-key=38:113:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;38:113:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change= Jul 23 16:23:09 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/533, version=0.345.38) Jul 23 16:23:14 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_ping) info: Reporting our current digest to an-a02n01: c47a6b4e95d7431909917c2bdffe04f4 for 0.345.38 (0x56197640dcd0 0) Jul 23 16:23:15 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_1[4159153] error output [ 2021-07-23 16:23:15,635 ERROR: Returned 1: Timeout: No Response from 10.201.2.1:161 ] Jul 23 16:23:15 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_1[4159153] error output [ ] Jul 23 16:23:15 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_1[4159153] error output [ ] Jul 23 16:23:15 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_1[4159153] error output [ 2021-07-23 16:23:15,635 ERROR: Please use '-h' for usage ] Jul 23 16:23:15 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_1[4159153] error output [ ] Jul 23 16:23:15 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4159153] stderr: [ 2021-07-23 16:23:15,635 ERROR: Returned 1: Timeout: No Response from 10.201.2.1:161 ] Jul 23 16:23:15 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4159153] stderr: [ ] Jul 23 16:23:15 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4159153] stderr: [ ] Jul 23 16:23:15 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4159153] stderr: [ 2021-07-23 16:23:15,635 ERROR: Please use '-h' for usage ] Jul 23 16:23:15 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4159153] stderr: [ ] Jul 23 16:23:15 an-a02n01.alteeve.com pacemaker-fenced [121629] (internal_stonith_action_execute) info: Attempt 2 to execute fence_apc_snmp (monitor). remaining timeout is 14 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]: INFINITY -> (unset) | from an-a02n01 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 94 with 2 changes for fail-count-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_peer_update) notice: Setting last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]: 1627082589 -> (unset) | from an-a02n01 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 95 with 2 changes for last-failure-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/94) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/95) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.38 2 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.39 (null) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-fail-count-apc_snmp_node1_an-pdu03.start_0'] Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=39 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/94, version=0.345.39) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 94 result for fail-count-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) notice: Transition 113 aborted by deletion of nvpair[@id='status-2-fail-count-apc_snmp_node1_an-pdu03.start_0']: Transient attribute change | cib=0.345.39 source=abort_unless_down:327 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-fail-count-apc_snmp_node1_an-pdu03.start_0'] complete=false Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.39 2 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.40 (null) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-last-failure-apc_snmp_node1_an-pdu03.start_0'] Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=40 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/95, version=0.345.40) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 95 result for last-failure-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 113 aborted by deletion of nvpair[@id='status-2-last-failure-apc_snmp_node1_an-pdu03.start_0']: Transient attribute change | cib=0.345.40 source=abort_unless_down:327 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-last-failure-apc_snmp_node1_an-pdu03.start_0'] complete=false Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.40 2 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.41 (null) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03'] Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=41 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n02']/lrm/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']: OK (rc=0, origin=an-a02n02/crmd/121, version=0.345.40) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.40 2 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.345.41 (null) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03'] Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=41 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n02']/lrm/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']: OK (rc=0, origin=an-a02n02/crmd/122, version=0.345.41) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 113 aborted by deletion of lrm_resource[@id='apc_snmp_node1_an-pdu03']: Resource state removal | cib=0.345.41 source=abort_unless_down:327 path=/cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03'] complete=false Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.345.41 2 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.346.0 (null) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=346, @num_updates=0 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']/nvpair[@id='cib-bootstrap-options-last-lrm-refresh']: @value=1627082596 Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n02/crmd/124, version=0.346.0) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 113 aborted by cib-bootstrap-options-last-lrm-refresh doing modify last-lrm-refresh=1627082596: Configuration change | cib=0.346.0 source=te_update_diff_v2:474 path=/cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']/nvpair[@id='cib-bootstrap-options-last-lrm-refresh'] complete=false Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-39.raw Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.346.0 of the CIB to disk (digest: eae429533562bb1e20d7929bf35529d1) Jul 23 16:23:16 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.Oz803D (digest: /var/lib/pacemaker/cib/cib.t2pcAF) Jul 23 16:23:21 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_ping) info: Reporting our current digest to an-a02n01: c2c07cd045b9d490d650e6b5d05df5c3 for 0.346.0 (0x5619760ad450 0) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_2[4159224] error output [ 2021-07-23 16:23:22,793 ERROR: Returned 1: Timeout: No Response from 10.201.2.1:161 ] Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_2[4159224] error output [ ] Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_2[4159224] error output [ ] Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_2[4159224] error output [ 2021-07-23 16:23:22,793 ERROR: Please use '-h' for usage ] Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_2[4159224] error output [ ] Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4159224] stderr: [ 2021-07-23 16:23:22,793 ERROR: Returned 1: Timeout: No Response from 10.201.2.1:161 ] Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4159224] stderr: [ ] Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4159224] stderr: [ ] Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4159224] stderr: [ 2021-07-23 16:23:22,793 ERROR: Please use '-h' for usage ] Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4159224] stderr: [ ] Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-fenced [121629] (update_remaining_timeout) info: Attempted to execute agent fence_apc_snmp (monitor) the maximum number of times (2) allowed Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_operation) notice: Operation 'monitor' [4159224] using apc_snmp_node1_an-pdu03 returned -201 (Generic Pacemaker error) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-execd [121630] (log_finished) info: apc_snmp_node1_an-pdu03 start (call 140) exited with status 1 (execution time 13319ms, queue time 0ms) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) notice: Result of start operation for apc_snmp_node1_an-pdu03 on an-a02n01: error | rc=1 call=140 key=apc_snmp_node1_an-pdu03_start_0 confirmed=true cib-update=535 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/535) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.346.0 2 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.346.1 (null) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=1 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @transition-magic=0:1;38:113:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=140, @rc-code=1, @op-status=0, @exec-time=13319 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']: Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/96, version=0.346.2) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 96 result for fail-count-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=INFINITY Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/97) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.346.2 2 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.346.3 (null) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=3 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/97, version=0.346.3) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 97 result for last-failure-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=1627082602 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 113 aborted by status-1-fail-count-apc_snmp_node1_an-pdu03.start_0 doing create fail-count-apc_snmp_node1_an-pdu03#start_0=INFINITY: Transient attribute change | cib=0.346.2 source=abort_unless_down:313 path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1'] complete=true Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 113 aborted by status-1-last-failure-apc_snmp_node1_an-pdu03.start_0 doing create last-failure-apc_snmp_node1_an-pdu03#start_0=1627082602: Transient attribute change | cib=0.346.3 source=abort_unless_down:313 path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1'] complete=true Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 23 16:23:09 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 23 16:23:09 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_0 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu04 cannot run anywhere Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu03 on an-a02n02 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu04 (Stopped) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu03 (Started an-a02n02) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node2 (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node2 (Started an-a02n02) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n02) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Recover apc_snmp_node1_an-pdu03 ( an-a02n01 -> an-a02n02 ) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 114, saving inputs in /var/lib/pacemaker/pengine/pe-input-135.bz2 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-controld [121633] (handle_response) info: pe_calc calculation pe_calc-dc-1627082602-355 is obsolete Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 23 16:23:09 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 23 16:23:09 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_0 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu03 has failed INFINITY times on an-a02n01 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu04 cannot run anywhere Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu03 on an-a02n02 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu04 (Stopped) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu03 (Started an-a02n02) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node2 (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node2 (Started an-a02n02) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n02) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Recover apc_snmp_node1_an-pdu03 ( an-a02n01 -> an-a02n02 ) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 115, saving inputs in /var/lib/pacemaker/pengine/pe-input-136.bz2 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-controld [121633] (do_te_invoke) info: Processing graph 115 (ref=pe_calc-dc-1627082602-356) derived from /var/lib/pacemaker/pengine/pe-input-136.bz2 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation apc_snmp_node1_an-pdu03_monitor_0 on an-a02n02 | action 14 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation apc_snmp_node1_an-pdu03_stop_0 locally on an-a02n01 | action 2 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of stop operation for apc_snmp_node1_an-pdu03 on an-a02n01 | transition_key=2:115:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=apc_snmp_node1_an-pdu03_stop_0 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/539) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:apc_snmp_node1_an-pdu03 action:stop call_id:141 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-execd [121630] (log_finished) info: apc_snmp_node1_an-pdu03 stop (call 141) exited with status 0 (execution time 0ms, queue time 0ms) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.346.3 2 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) notice: Result of stop operation for apc_snmp_node1_an-pdu03 on an-a02n01: ok | rc=0 call=141 key=apc_snmp_node1_an-pdu03_stop_0 confirmed=true cib-update=540 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.346.4 (null) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=4 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @operation_key=apc_snmp_node1_an-pdu03_stop_0, @operation=stop, @transition-key=2:115:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;2:115:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1627 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/539, version=0.346.4) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/540) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.346.4 2 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.346.5 (null) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=5 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @transition-magic=0:0;2:115:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=141, @rc-code=0, @op-status=0 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/540, version=0.346.5) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 115 action 2 (apc_snmp_node1_an-pdu03_stop_0 on an-a02n01) confirmed: ok | rc=0 call-id=141 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.346.5 2 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.346.6 (null) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=6 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/126, version=0.346.6) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.346.6 2 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.346.7 (null) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=7 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @transition-magic=0:7;14:115:7:789131c0-5164-4a52-9822-30c5026c3827, @call-id=118, @rc-code=7, @op-status=0 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/127, version=0.346.7) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 115 action 14 (apc_snmp_node1_an-pdu03_monitor_0 on an-a02n02) confirmed: not running | rc=7 call-id=118 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation apc_snmp_node1_an-pdu03_start_0 on an-a02n02 | action 39 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.346.7 2 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.346.8 (null) Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=8 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @operation_key=apc_snmp_node1_an-pdu03_start_0, @operation=start, @transition-key=39:115:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;39:115:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1 Jul 23 16:23:22 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/128, version=0.346.8) Jul 23 16:23:27 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_ping) info: Reporting our current digest to an-a02n01: 05ad8f8cf3fe23da74845387b2d1f9fd for 0.346.8 (0x5619760ad450 0) Jul 23 16:23:28 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_apply_diff operation for section 'all' to all (origin=local/cibadmin/2) Jul 23 16:23:28 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.346.8 2 Jul 23 16:23:28 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.347.0 a28da2a03bddb39f0d3e971cd6640c86 Jul 23 16:23:28 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/configuration/resources/primitive[@id='apc_snmp_node1_an-pdu04']/meta_attributes[@id='apc_snmp_node1_an-pdu04-meta_attributes']/nvpair[@id='apc_snmp_node1_an-pdu04-meta_attributes-target-role'] Jul 23 16:23:28 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=347, @num_updates=0 Jul 23 16:23:28 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=an-a02n01/cibadmin/2, version=0.347.0) Jul 23 16:23:28 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) notice: Transition 115 aborted by deletion of nvpair[@id='apc_snmp_node1_an-pdu04-meta_attributes-target-role']: Configuration change | cib=0.347.0 source=te_update_diff_v2:474 path=/cib/configuration/resources/primitive[@id='apc_snmp_node1_an-pdu04']/meta_attributes[@id='apc_snmp_node1_an-pdu04-meta_attributes']/nvpair[@id='apc_snmp_node1_an-pdu04-meta_attributes-target-role'] complete=false Jul 23 16:23:28 an-a02n01.alteeve.com pacemaker-fenced [121629] (update_cib_stonith_devices_v2) info: Updating device list from CIB: (meta) attribute deleted from resource Jul 23 16:23:28 an-a02n01.alteeve.com pacemaker-fenced [121629] (cib_devices_update) info: Updating devices to version 0.347.0 Jul 23 16:23:28 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_device_register) notice: Added 'apc_snmp_node1_an-pdu04' to device list (8 active devices) Jul 23 16:23:28 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-40.raw Jul 23 16:23:28 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.347.0 of the CIB to disk (digest: 71dd588bb0d8988144ae131c7ba8d738) Jul 23 16:23:28 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.cyn1Ry (digest: /var/lib/pacemaker/cib/cib.WqI9JA) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_replace operation for section configuration to all (origin=local/cibadmin/2) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.347.0 2 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.348.0 (null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/configuration/fencing-topology/fencing-level[@id='fl-an-a02n01-2'] Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=348 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_replace_notify) info: Local-only Replace: 0.348.0 from an-a02n01 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_replace operation for section configuration: OK (rc=0, origin=an-a02n01/cibadmin/2, version=0.348.0) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_replaced_cb) notice: Updating all attributes after cib_refresh_notify event Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by deletion of fencing-level[@id='fl-an-a02n01-2']: Configuration change | cib=0.348.0 source=te_update_diff_v2:474 path=/cib/configuration/fencing-topology/fencing-level[@id='fl-an-a02n01-2'] complete=false Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 98 with 2 changes for last-failure-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 99 with 2 changes for fail-count-apc_snmp_node2_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Processed 2 private changes for #attrd-protocol, id=n/a, set=n/a Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 100 with 2 changes for fail-count-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 101 with 2 changes for last-failure-apc_snmp_node1_an-pdu03#start_0 (id n/a, set n/a) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 102 with 2 changes for fail-count-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 103 with 2 changes for last-failure-apc_snmp_node2_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 104 with 2 changes for fail-count-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_TRANSITION_ENGINE -> S_ELECTION | input=I_ELECTION cause=C_FSA_INTERNAL origin=do_cib_replaced Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Unset DC. Was an-a02n01 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (write_attribute) info: Sent CIB request 105 with 2 changes for last-failure-apc_snmp_node1_an-pdu04#start_0 (id n/a, set n/a) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/543) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/544) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-fenced [121629] (update_fencing_topology) info: Re-initializing fencing topology after delete operation 0.348.0 for /cib/configuration/fencing-topology/fencing-level[@id='fl-an-a02n01-2'] Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/98) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_level_remove) info: No fencing topology found for an-a02n02 (0 active entries) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/99) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/100) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/101) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_level_register) info: Target an-a02n02 has 1 active fencing level Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/102) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_level_register) info: Target an-a02n02 has 2 active fencing levels Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_level_register) info: Target an-a02n02 has 3 active fencing levels Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_level_remove) info: No fencing topology found for an-a02n01 (1 active entry) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_level_register) info: Target an-a02n01 has 1 active fencing level Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_level_register) info: Target an-a02n01 has 2 active fencing levels Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-controld [121633] (election_check) info: election-DC won by local node Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-controld [121633] (do_log) info: Input I_ELECTION_DC received in state S_ELECTION from election_win_cb Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) notice: State transition S_ELECTION -> S_INTEGRATION | input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=election_win_cb Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/543, version=0.348.0) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.348.0 2 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.348.1 (null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=1 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_cib_replaced Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_cib_replaced Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/544, version=0.348.1) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.348.1 2 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.348.2 (null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=2 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/98, version=0.348.2) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 98 result for last-failure-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.348.2 2 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.348.3 (null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=3 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/99, version=0.348.3) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 99 result for fail-count-apc_snmp_node2_an-pdu03#start_0: OK | rc=0 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n01]=(null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.348.3 2 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.348.4 (null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=4 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/100, version=0.348.4) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 100 result for fail-count-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=INFINITY Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.348.4 2 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.348.5 (null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=5 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/101, version=0.348.5) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 101 result for last-failure-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=1627082602 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=(null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.348.5 2 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.348.6 (null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=6 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/102, version=0.348.6) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 102 result for fail-count-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=(null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/103) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/104) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/105) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.348.6 2 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.348.7 (null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=7 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/103, version=0.348.7) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 103 result for last-failure-apc_snmp_node2_an-pdu04#start_0: OK | rc=0 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node2_an-pdu04#start_0[an-a02n02]=(null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.348.7 2 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.348.8 (null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=8 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/104, version=0.348.8) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 104 result for fail-count-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=(null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.348.8 2 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.348.9 (null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=9 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/105, version=0.348.9) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 105 result for last-failure-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=(null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-41.raw Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_takeover) info: Taking over DC status for this partition Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.348.0 of the CIB to disk (digest: 91f943c1dac5b98e6d799f6aeef67d12) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.SF2t32 (digest: /var/lib/pacemaker/cib/cib.RU21Y2) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_master operation for section 'all': OK (rc=0, origin=local/crmd/545, version=0.348.9) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/546) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=an-a02n01/crmd/546, version=0.348.9) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/548) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/550) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-11 offer to an-a02n02 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-controld [121633] (join_make_offer) info: Sending join-11 offer to an-a02n01 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-controld [121633] (do_dc_join_offer_all) info: Waiting on join-11 requests from 2 outstanding nodes Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-controld [121633] (update_dc) info: Set DC to an-a02n01 (3.10.2) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/548, version=0.348.9) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/550, version=0.348.9) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/552) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_INTEGRATION -> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=an-a02n01/crmd/552, version=0.348.9) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Digest matched on replace from an-a02n01: ad47c887bc4d7e3e1ab7a73466ae4e44 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_replace) info: Replaced 0.348.9 with 0.348.9 from an-a02n01 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=an-a02n01/crmd/555, version=0.348.9) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/556) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/557) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/556, version=0.348.9) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/557, version=0.348.9) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_delay_metadata_1[4160057] exited with status 0 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-42.raw Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.348.0 of the CIB to disk (digest: 91f943c1dac5b98e6d799f6aeef67d12) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.zeq7Nf (digest: /var/lib/pacemaker/cib/cib.wz8AWg) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //primitive[@id='apc_snmp_node1_an-pdu04'] to all (origin=local/cibadmin/2) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.348.9 2 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.349.0 (null) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/configuration/resources/primitive[@id='apc_snmp_node1_an-pdu04'] Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=349, @num_updates=0 Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //primitive[@id='apc_snmp_node1_an-pdu04']: OK (rc=0, origin=an-a02n01/cibadmin/2, version=0.349.0) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_device_remove) info: Removed 'apc_snmp_node1_an-pdu04' from device list (7 active devices) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-43.raw Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.349.0 of the CIB to disk (digest: 93282e109411f51fa6f057dc01739e27) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.49BT2x (digest: /var/lib/pacemaker/cib/cib.Eeuzxy) Jul 23 16:23:31 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4160060] exited with status 0 Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_apply_diff operation for section 'all' to all (origin=local/cibadmin/2) Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.349.0 2 Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.350.0 (null) Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=350 Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/configuration/resources: Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=an-a02n01/cibadmin/2, version=0.350.0) Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-fenced [121629] (update_cib_stonith_devices_v2) info: Updating device list from CIB: create resources Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-fenced [121629] (cib_devices_update) info: Updating devices to version 0.350.0 Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-44.raw Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_device_register) notice: Added 'apc_snmp_node1_an-pdu04' to device list (8 active devices) Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.350.0 of the CIB to disk (digest: 881f2d2253092b9bddefbde756f44702) Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.0yTI8s (digest: /var/lib/pacemaker/cib/cib.0QW8Xt) Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4160066] exited with status 0 Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_apply_diff operation for section 'all' to all (origin=local/cibadmin/2) Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.350.0 2 Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.0 (null) Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=351 Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/configuration/resources/primitive[@id='apc_snmp_node2_an-pdu03']: Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=an-a02n01/cibadmin/2, version=0.351.0) Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-fenced [121629] (update_cib_stonith_devices_v2) info: Updating device list from CIB: create primitive[@id='apc_snmp_node2_an-pdu03'] Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-fenced [121629] (cib_devices_update) info: Updating devices to version 0.351.0 Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-fenced [121629] (cib_device_update) info: Device apc_snmp_node2_an-pdu03 has been disabled Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-45.raw Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.351.0 of the CIB to disk (digest: 1c7d1d25634db84aa9743aceef9de1a9) Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.yHKqcA (digest: /var/lib/pacemaker/cib/cib.SuVKiz) Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4160078] exited with status 0 Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4160104] exited with status 0 Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4160106] exited with status 0 Jul 23 16:23:32 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_apc_snmp_metadata_1[4160107] exited with status 0 Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4160109] exited with status 0 Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4160110] exited with status 0 Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by deletion of primitive[@id='apc_snmp_node1_an-pdu04']: Configuration change | cib=0.349.0 source=te_update_diff_v2:474 path=/cib/configuration/resources/primitive[@id='apc_snmp_node1_an-pdu04'] complete=false Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by primitive.apc_snmp_node1_an-pdu04 'create': Configuration change | cib=0.350.0 source=te_update_diff_v2:474 path=/cib/configuration/resources complete=false Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by meta_attributes.apc_snmp_node2_an-pdu03-meta_attributes 'create': Configuration change | cib=0.351.0 source=te_update_diff_v2:474 path=/cib/configuration/resources/primitive[@id='apc_snmp_node2_an-pdu03'] complete=false Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-controld [121633] (controld_delete_node_state) info: Deleting resource history for node an-a02n02 (via CIB call 558) | xpath=//node_state[@uname='an-a02n02']/lrm Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='an-a02n02']/lrm to all (origin=local/crmd/558) Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/559) Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-controld [121633] (controld_delete_node_state) info: Deleting resource history for node an-a02n01 (via CIB call 560) | xpath=//node_state[@uname='an-a02n01']/lrm Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.0 2 Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.1 (null) Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2'] Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=1 Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n02']/lrm: OK (rc=0, origin=an-a02n01/crmd/558, version=0.351.1) Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.1 2 Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.2 (null) Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=2 Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_lrm_query_internal Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/559, version=0.351.2) Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='an-a02n01']/lrm to all (origin=local/crmd/560) Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.2 2 Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.3 (null) Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/lrm[@id='1'] Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=3 Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='an-a02n01']/lrm: OK (rc=0, origin=an-a02n01/crmd/560, version=0.351.3) Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_delay_metadata_1[4160112] exited with status 0 Jul 23 16:23:33 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4160113] exited with status 0 Jul 23 16:23:34 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4160116] exited with status 0 Jul 23 16:23:34 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4160117] exited with status 0 Jul 23 16:23:34 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4160118] exited with status 0 Jul 23 16:23:34 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4160120] exited with status 0 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_apc_snmp_metadata_1[4160136] exited with status 0 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4160138] exited with status 0 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: server_meta-data_0[4160139] exited with status 0 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/561) Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by deletion of lrm[@id='2']: Resource state removal | cib=0.351.1 source=abort_unless_down:327 path=/cib/status/node_state[@id='2']/lrm[@id='2'] complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv14-file-processor_monitor_0 'create' on an-a02n02: Old event | magic=0:7;20:101:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 20 (srv14-file-processor_monitor_0 on an-a02n02): arrived really late | rc=7 target-rc=7 call-id=41 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation delay_node1_stop_0 'create' on an-a02n02: Old event | magic=0:0;29:110:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 110 action 29 (delay_node1_stop_0 on an-a02n02): arrived really late | rc=0 target-rc=0 call-id=109 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation delay_node2_start_0 'create' on an-a02n02: Old event | magic=0:0;17:111:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 111 action 17 (delay_node2_start_0 on an-a02n02): arrived really late | rc=0 target-rc=0 call-id=110 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation delay_node2_monitor_60000 'create' on an-a02n02: Change in recurring result | magic=0:0;18:111:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:408 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 111 action 18 (delay_node2_monitor_60000 on an-a02n02): arrived after initial scheduling | rc=0 target-rc=0 call-id=112 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation apc_snmp_node1_an-pdu04_stop_0 'create' on an-a02n02: Old event | magic=0:0;17:110:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 110 action 17 (apc_snmp_node1_an-pdu04_stop_0 on an-a02n02): arrived really late | rc=0 target-rc=0 call-id=107 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv22-machineinterface_monitor_0 'create' on an-a02n02: Old event | magic=0:7;28:101:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 28 (srv22-machineinterface_monitor_0 on an-a02n02): arrived really late | rc=7 target-rc=7 call-id=73 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv13-psql_monitor_0 'create' on an-a02n02: Old event | magic=0:7;19:101:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 19 (srv13-psql_monitor_0 on an-a02n02): arrived really late | rc=7 target-rc=7 call-id=37 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv24-inventory_monitor_0 'create' on an-a02n02: Old event | magic=0:7;30:101:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 30 (srv24-inventory_monitor_0 on an-a02n02): arrived really late | rc=7 target-rc=7 call-id=81 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv15-tickets_monitor_0 'create' on an-a02n02: Old event | magic=0:7;21:101:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 21 (srv15-tickets_monitor_0 on an-a02n02): arrived really late | rc=7 target-rc=7 call-id=45 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv18-inventory_monitor_0 'create' on an-a02n02: Old event | magic=0:7;24:101:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 24 (srv18-inventory_monitor_0 on an-a02n02): arrived really late | rc=7 target-rc=7 call-id=57 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv21-tickets_monitor_0 'create' on an-a02n02: Old event | magic=0:7;27:101:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 27 (srv21-tickets_monitor_0 on an-a02n02): arrived really late | rc=7 target-rc=7 call-id=69 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv17-order-processing_monitor_0 'create' on an-a02n02: Old event | magic=0:7;23:101:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 23 (srv17-order-processing_monitor_0 on an-a02n02): arrived really late | rc=7 target-rc=7 call-id=53 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation ipmilan_node1_start_0 'create' on an-a02n02: Old event | magic=0:0;27:108:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 108 action 27 (ipmilan_node1_start_0 on an-a02n02): arrived really late | rc=0 target-rc=0 call-id=103 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation ipmilan_node1_monitor_60000 'create' on an-a02n02: Change in recurring result | magic=0:0;24:109:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:408 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 109 action 24 (ipmilan_node1_monitor_60000 on an-a02n02): arrived after initial scheduling | rc=0 target-rc=0 call-id=105 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation ipmilan_node2_stop_0 'create' on an-a02n02: Old event | magic=0:0;21:108:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 108 action 21 (ipmilan_node2_stop_0 on an-a02n02): arrived really late | rc=0 target-rc=0 call-id=102 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv19-uls_monitor_0 'create' on an-a02n02: Old event | magic=0:7;25:101:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 25 (srv19-uls_monitor_0 on an-a02n02): arrived really late | rc=7 target-rc=7 call-id=61 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation apc_snmp_node2_an-pdu03_start_0 'create' on an-a02n02: Old event | magic=0:0;16:106:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 106 action 16 (apc_snmp_node2_an-pdu03_start_0 on an-a02n02): arrived really late | rc=0 target-rc=0 call-id=96 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation apc_snmp_node2_an-pdu03_monitor_60000 'create' on an-a02n02: Change in recurring result | magic=0:0;15:107:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:408 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 107 action 15 (apc_snmp_node2_an-pdu03_monitor_60000 on an-a02n02): arrived after initial scheduling | rc=0 target-rc=0 call-id=99 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation apc_snmp_node2_an-pdu04_stop_0 'create' on an-a02n02: Old event | magic=0:0;14:103:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 103 action 14 (apc_snmp_node2_an-pdu04_stop_0 on an-a02n02): arrived really late | rc=0 target-rc=0 call-id=89 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv20-fileprocessor_monitor_0 'create' on an-a02n02: Old event | magic=0:7;26:101:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 26 (srv20-fileprocessor_monitor_0 on an-a02n02): arrived really late | rc=7 target-rc=7 call-id=65 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv16-machine-interface_monitor_0 'create' on an-a02n02: Old event | magic=0:7;22:101:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 22 (srv16-machine-interface_monitor_0 on an-a02n02): arrived really late | rc=7 target-rc=7 call-id=49 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv23-orderprocessing_monitor_0 'create' on an-a02n02: Old event | magic=0:7;29:101:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.2 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 101 action 29 (srv23-orderprocessing_monitor_0 on an-a02n02): arrived really late | rc=7 target-rc=7 call-id=77 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by deletion of lrm[@id='1']: Resource state removal | cib=0.351.3 source=abort_unless_down:327 path=/cib/status/node_state[@id='1']/lrm[@id='1'] complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE | input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted: Peer Cancelled | source=do_te_invoke:129 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.3 2 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.4 (null) Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=4 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_lrm_query_internal Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/561, version=0.351.4) Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv14-file-processor_monitor_0 'create' on an-a02n01: Old event | magic=0:7;10:0:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/564) Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 0 action 10 (srv14-file-processor_monitor_0 on an-a02n01): arrived really late | rc=7 target-rc=7 call-id=41 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation delay_node1_start_0 'create' on an-a02n01: Old event | magic=0:0;21:111:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 111 action 21 (delay_node1_start_0 on an-a02n01): arrived really late | rc=0 target-rc=0 call-id=138 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation delay_node1_monitor_60000 'create' on an-a02n01: Change in recurring result | magic=0:0;22:111:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:408 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 111 action 22 (delay_node1_monitor_60000 on an-a02n01): arrived after initial scheduling | rc=0 target-rc=0 call-id=139 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation delay_node2_stop_0 'create' on an-a02n01: Old event | magic=0:0;24:110:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/565) Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 110 action 24 (delay_node2_stop_0 on an-a02n01): arrived really late | rc=0 target-rc=0 call-id=134 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation apc_snmp_node1_an-pdu04_stop_0 'create' on an-a02n01: Old event | magic=0:0;1:4:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 4 action 1 (apc_snmp_node1_an-pdu04_stop_0 on an-a02n01): arrived really late | rc=0 target-rc=0 call-id=92 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation apc_snmp_node1_an-pdu03_start_0 'create' on an-a02n01: Old event | magic=0:1;38:113:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 113 action 38 (apc_snmp_node1_an-pdu03_start_0 on an-a02n01): arrived really late | rc=1 target-rc=0 call-id=140 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/566) Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv13-psql_monitor_0 'create' on an-a02n01: Old event | magic=0:7;9:0:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 0 action 9 (srv13-psql_monitor_0 on an-a02n01): arrived really late | rc=7 target-rc=7 call-id=37 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv22-machineinterface_start_0 'create' on an-a02n01: Old event | magic=0:0;22:8:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 8 action 22 (srv22-machineinterface_start_0 on an-a02n01): arrived really late | rc=0 target-rc=0 call-id=100 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv22-machineinterface_monitor_60000 'create' on an-a02n01: Change in recurring result | magic=0:0;24:9:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:408 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 9 action 24 (srv22-machineinterface_monitor_60000 on an-a02n01): arrived after initial scheduling | rc=0 target-rc=0 call-id=101 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv24-inventory_start_0 'create' on an-a02n01: Old event | magic=0:0;28:10:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 10 action 28 (srv24-inventory_start_0 on an-a02n01): arrived really late | rc=0 target-rc=0 call-id=104 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv24-inventory_monitor_60000 'create' on an-a02n01: Change in recurring result | magic=0:0;29:10:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:408 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 10 action 29 (srv24-inventory_monitor_60000 on an-a02n01): arrived after initial scheduling | rc=0 target-rc=0 call-id=105 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv15-tickets_monitor_0 'create' on an-a02n01: Old event | magic=0:7;11:0:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 0 action 11 (srv15-tickets_monitor_0 on an-a02n01): arrived really late | rc=7 target-rc=7 call-id=45 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv18-inventory_monitor_0 'create' on an-a02n01: Old event | magic=0:7;14:0:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 0 action 14 (srv18-inventory_monitor_0 on an-a02n01): arrived really late | rc=7 target-rc=7 call-id=57 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv21-tickets_start_0 'create' on an-a02n01: Old event | magic=0:0;19:7:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 7 action 19 (srv21-tickets_start_0 on an-a02n01): arrived really late | rc=0 target-rc=0 call-id=98 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv21-tickets_monitor_60000 'create' on an-a02n01: Change in recurring result | magic=0:0;21:8:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:408 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 8 action 21 (srv21-tickets_monitor_60000 on an-a02n01): arrived after initial scheduling | rc=0 target-rc=0 call-id=99 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv17-order-processing_monitor_0 'create' on an-a02n01: Old event | magic=0:7;13:0:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 0 action 13 (srv17-order-processing_monitor_0 on an-a02n01): arrived really late | rc=7 target-rc=7 call-id=53 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation ipmilan_node1_stop_0 'create' on an-a02n01: Old event | magic=0:0;26:108:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 108 action 26 (ipmilan_node1_stop_0 on an-a02n01): arrived really late | rc=0 target-rc=0 call-id=129 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation ipmilan_node2_start_0 'create' on an-a02n01: Old event | magic=0:0;18:109:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 109 action 18 (ipmilan_node2_start_0 on an-a02n01): arrived really late | rc=0 target-rc=0 call-id=130 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation ipmilan_node2_monitor_60000 'create' on an-a02n01: Change in recurring result | magic=0:0;19:109:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:408 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 109 action 19 (ipmilan_node2_monitor_60000 on an-a02n01): arrived after initial scheduling | rc=0 target-rc=0 call-id=131 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv19-uls_start_0 'create' on an-a02n01: Old event | magic=0:0;13:5:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 5 action 13 (srv19-uls_start_0 on an-a02n01): arrived really late | rc=0 target-rc=0 call-id=94 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv19-uls_monitor_60000 'create' on an-a02n01: Change in recurring result | magic=0:0;15:6:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:408 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 6 action 15 (srv19-uls_monitor_60000 on an-a02n01): arrived after initial scheduling | rc=0 target-rc=0 call-id=95 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation apc_snmp_node2_an-pdu03_stop_0 'create' on an-a02n01: Old event | magic=0:0;2:2:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 2 action 2 (apc_snmp_node2_an-pdu03_stop_0 on an-a02n01): arrived really late | rc=0 target-rc=0 call-id=89 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation apc_snmp_node2_an-pdu04_start_0 'create' on an-a02n01: Old event | magic=0:0;20:110:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 110 action 20 (apc_snmp_node2_an-pdu04_start_0 on an-a02n01): arrived really late | rc=0 target-rc=0 call-id=132 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation apc_snmp_node2_an-pdu04_monitor_60000 'create' on an-a02n01: Change in recurring result | magic=0:0;14:111:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:408 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 111 action 14 (apc_snmp_node2_an-pdu04_monitor_60000 on an-a02n01): arrived after initial scheduling | rc=0 target-rc=0 call-id=137 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv20-fileprocessor_start_0 'create' on an-a02n01: Old event | magic=0:0;16:6:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 6 action 16 (srv20-fileprocessor_start_0 on an-a02n01): arrived really late | rc=0 target-rc=0 call-id=96 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv20-fileprocessor_monitor_60000 'create' on an-a02n01: Change in recurring result | magic=0:0;18:7:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:408 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 7 action 18 (srv20-fileprocessor_monitor_60000 on an-a02n01): arrived after initial scheduling | rc=0 target-rc=0 call-id=97 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv23-orderprocessing_start_0 'create' on an-a02n01: Old event | magic=0:0;25:9:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 9 action 25 (srv23-orderprocessing_start_0 on an-a02n01): arrived really late | rc=0 target-rc=0 call-id=102 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv23-orderprocessing_monitor_60000 'create' on an-a02n01: Change in recurring result | magic=0:0;27:10:0:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:408 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 10 action 27 (srv23-orderprocessing_monitor_60000 on an-a02n01): arrived after initial scheduling | rc=0 target-rc=0 call-id=103 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by operation srv16-machine-interface_monitor_0 'create' on an-a02n01: Old event | magic=0:7;12:0:7:789131c0-5164-4a52-9822-30c5026c3827 cib=0.351.4 source=process_graph_event:412 complete=false Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 0 action 12 (srv16-machine-interface_monitor_0 on an-a02n01): arrived really late | rc=7 target-rc=7 call-id=49 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=an-a02n01/crmd/564, version=0.351.4) Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.4 2 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.5 (null) Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=5 Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/565, version=0.351.5) Jul 23 16:23:35 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=an-a02n01/crmd/566, version=0.351.5) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.5 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.6 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=6 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @operation_key=apc_snmp_node1_an-pdu03_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=39:115:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=0:1;39:115:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=119, @rc-code= Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']: Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/107, version=0.351.7) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 107 result for fail-count-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=INFINITY Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=INFINITY Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.7 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.8 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=8 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/108, version=0.351.8) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 108 result for last-failure-apc_snmp_node1_an-pdu03#start_0: OK | rc=0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]=1627082602 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]=1627082616 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by status-2-fail-count-apc_snmp_node1_an-pdu03.start_0 doing create fail-count-apc_snmp_node1_an-pdu03#start_0=INFINITY: Transient attribute change | cib=0.351.7 source=abort_unless_down:313 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=true Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 115 aborted by status-2-last-failure-apc_snmp_node1_an-pdu03.start_0 doing create last-failure-apc_snmp_node1_an-pdu03#start_0=1627082616: Transient attribute change | cib=0.351.8 source=abort_unless_down:313 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=true Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 23 16:23:09 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:23:22 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:23:22 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu03 has failed INFINITY times on an-a02n01 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu03 cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for ipmilan_node2 on an-a02n02 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for ipmilan_node1 on an-a02n01 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu03 on an-a02n02 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu04 on an-a02n02 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Stop apc_snmp_node2_an-pdu03 ( an-a02n02 ) due to node availability Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Move ipmilan_node2 ( an-a02n01 -> an-a02n02 ) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node2 (Started an-a02n02) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Move ipmilan_node1 ( an-a02n02 -> an-a02n01 ) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Recover apc_snmp_node1_an-pdu03 ( an-a02n02 ) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start apc_snmp_node1_an-pdu04 ( an-a02n02 ) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 116, saving inputs in /var/lib/pacemaker/pengine/pe-input-137.bz2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (handle_response) info: pe_calc calculation pe_calc-dc-1627082616-367 is obsolete Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 23 16:23:09 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:23:22 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:23:22 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu03 has failed INFINITY times on an-a02n01 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu03 has failed INFINITY times on an-a02n02 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu03 cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu03 cannot run anywhere Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for ipmilan_node2 on an-a02n02 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for ipmilan_node1 on an-a02n01 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu04 on an-a02n02 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Stop apc_snmp_node2_an-pdu03 ( an-a02n02 ) due to node availability Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Move ipmilan_node2 ( an-a02n01 -> an-a02n02 ) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node2 (Started an-a02n02) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Move ipmilan_node1 ( an-a02n02 -> an-a02n01 ) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Stop apc_snmp_node1_an-pdu03 ( an-a02n02 ) due to node availability Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Start apc_snmp_node1_an-pdu04 ( an-a02n02 ) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 117, saving inputs in /var/lib/pacemaker/pengine/pe-input-138.bz2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (do_te_invoke) info: Processing graph 117 (ref=pe_calc-dc-1627082616-368) derived from /var/lib/pacemaker/pengine/pe-input-138.bz2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation apc_snmp_node2_an-pdu03_stop_0 on an-a02n02 | action 14 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation ipmilan_node2_stop_0 locally on an-a02n01 | action 17 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of stop operation for ipmilan_node2 on an-a02n01 | transition_key=17:117:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=ipmilan_node2_stop_0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:ipmilan_node2 action:stop call_id:143 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation ipmilan_node1_stop_0 on an-a02n02 | action 22 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation apc_snmp_node1_an-pdu03_stop_0 on an-a02n02 | action 11 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation apc_snmp_node1_an-pdu04_start_0 on an-a02n02 | action 39 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-execd [121630] (log_finished) info: ipmilan_node2 stop (call 143) exited with status 0 (execution time 0ms, queue time 0ms) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/571) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) info: Result of monitor operation for ipmilan_node2 on an-a02n01: Cancelled | call=131 key=ipmilan_node2_monitor_60000 confirmed=true Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) notice: Result of stop operation for ipmilan_node2 on an-a02n01: ok | rc=0 call=143 key=ipmilan_node2_stop_0 confirmed=true cib-update=572 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.8 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.9 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=9 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='ipmilan_node2']/lrm_rsc_op[@id='ipmilan_node2_last_0']: @operation_key=ipmilan_node2_stop_0, @operation=stop, @crm-debug-origin=do_update_resource, @transition-key=17:117:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;17:117:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-c Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/571, version=0.351.9) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/572) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.9 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.10 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=10 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='ipmilan_node2']/lrm_rsc_op[@id='ipmilan_node2_last_0']: @transition-magic=0:0;17:117:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=143, @rc-code=0, @op-status=0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/572, version=0.351.10) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 117 action 17 (ipmilan_node2_stop_0 on an-a02n01) confirmed: ok | rc=0 call-id=143 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation ipmilan_node2_start_0 on an-a02n02 | action 18 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.10 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.11 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=11 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_0']: @operation_key=apc_snmp_node2_an-pdu03_stop_0, @operation=stop, @crm-debug-origin=do_update_resource, @transition-key=14:117:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;14:117:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code= Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/132, version=0.351.11) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.11 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.12 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=12 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='ipmilan_node1']/lrm_rsc_op[@id='ipmilan_node1_last_0']: @operation_key=ipmilan_node1_stop_0, @operation=stop, @crm-debug-origin=do_update_resource, @transition-key=22:117:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;22:117:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-c Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/133, version=0.351.12) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.12 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.13 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=13 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node2_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node2_an-pdu03_last_0']: @transition-magic=0:0;14:117:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=121, @rc-code=0, @op-status=0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/134, version=0.351.13) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 117 action 14 (apc_snmp_node2_an-pdu03_stop_0 on an-a02n02) confirmed: ok | rc=0 call-id=121 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.13 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.14 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=14 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='ipmilan_node1']/lrm_rsc_op[@id='ipmilan_node1_last_0']: @transition-magic=0:0;22:117:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=123, @rc-code=0, @op-status=0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/135, version=0.351.14) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 117 action 22 (ipmilan_node1_stop_0 on an-a02n02) confirmed: ok | rc=0 call-id=123 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation ipmilan_node1_start_0 locally on an-a02n01 | action 23 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of start operation for ipmilan_node1 on an-a02n01 | transition_key=23:117:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=ipmilan_node1_start_0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:ipmilan_node1 action:start call_id:144 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.14 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.15 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=15 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @operation_key=apc_snmp_node1_an-pdu03_stop_0, @operation=stop, @transition-key=11:117:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;11:117:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=16 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/136, version=0.351.15) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.15 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.16 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=16 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_0']: @operation_key=apc_snmp_node1_an-pdu04_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=39:117:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;39:117:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-cod Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/137, version=0.351.16) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.16 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.17 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=17 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu03']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_0']: @transition-magic=0:0;11:117:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=124, @rc-code=0, @op-status=0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/138, version=0.351.17) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 117 action 11 (apc_snmp_node1_an-pdu03_stop_0 on an-a02n02) confirmed: ok | rc=0 call-id=124 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.17 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.18 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=18 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='ipmilan_node2']/lrm_rsc_op[@id='ipmilan_node2_last_0']: @operation_key=ipmilan_node2_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=18:117:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;18:117:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/139, version=0.351.18) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/573) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.18 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.19 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=19 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='ipmilan_node1']/lrm_rsc_op[@id='ipmilan_node1_last_0']: @operation_key=ipmilan_node1_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=23:117:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;23:117:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/573, version=0.351.19) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-execd [121630] (log_finished) info: ipmilan_node1 start (call 144) exited with status 0 (execution time 280ms, queue time 0ms) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.19 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.20 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=20 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='ipmilan_node2']/lrm_rsc_op[@id='ipmilan_node2_last_0']: @transition-magic=0:0;18:117:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=126, @rc-code=0, @op-status=0, @exec-time=281 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (action_synced_wait) info: fence_ipmilan_metadata_1[4160197] exited with status 0 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/140, version=0.351.20) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/574) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) notice: Result of start operation for ipmilan_node1 on an-a02n01: ok | rc=0 call=144 key=ipmilan_node1_start_0 confirmed=true cib-update=574 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.20 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.21 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=21 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='ipmilan_node1']/lrm_rsc_op[@id='ipmilan_node1_last_0']: @transition-magic=0:0;23:117:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=144, @rc-code=0, @op-status=0, @exec-time=280 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 117 action 18 (ipmilan_node2_start_0 on an-a02n02) confirmed: ok | rc=0 call-id=126 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation ipmilan_node2_monitor_60000 on an-a02n02 | action 19 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/574, version=0.351.21) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 117 action 23 (ipmilan_node1_start_0 on an-a02n01) confirmed: ok | rc=0 call-id=144 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating monitor operation ipmilan_node1_monitor_60000 locally on an-a02n01 | action 24 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of monitor operation for ipmilan_node1 on an-a02n01 | transition_key=24:117:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=ipmilan_node1_monitor_60000 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/575) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.21 2 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.22 (null) Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=22 Jul 23 16:23:36 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='ipmilan_node1']: Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/109, version=0.351.27) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 109 result for fail-count-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * fail-count-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=INFINITY Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.27 2 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.28 (null) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=28 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/attrd/110, version=0.351.28) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: CIB update 110 result for last-failure-apc_snmp_node1_an-pdu04#start_0: OK | rc=0 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n01]=(null) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-attrd [121631] (attrd_cib_callback) info: * last-failure-apc_snmp_node1_an-pdu04#start_0[an-a02n02]=1627082629 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 117 aborted by status-2-fail-count-apc_snmp_node1_an-pdu04.start_0 doing create fail-count-apc_snmp_node1_an-pdu04#start_0=INFINITY: Transient attribute change | cib=0.351.27 source=abort_unless_down:313 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=true Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) info: Transition 117 aborted by status-2-last-failure-apc_snmp_node1_an-pdu04.start_0 doing create last-failure-apc_snmp_node1_an-pdu04#start_0=1627082629: Transient attribute change | cib=0.351.28 source=abort_unless_down:313 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=true Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 23 16:23:09 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:23:36 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:23:36 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_0 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:23:22 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu03 has failed INFINITY times on an-a02n01 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu03 has failed INFINITY times on an-a02n02 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu03 cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu03 cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu04 on an-a02n02 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu03 (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node2 (Started an-a02n02) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node2 (Started an-a02n02) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu03 (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Recover apc_snmp_node1_an-pdu04 ( an-a02n02 ) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 118, saving inputs in /var/lib/pacemaker/pengine/pe-input-139.bz2 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-controld [121633] (handle_response) info: pe_calc calculation pe_calc-dc-1627082629-378 is obsolete Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n01 is active Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n01 is online Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status_fencing) info: Node an-a02n02 is active Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (determine_online_status) info: Node an-a02n02 is online Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 23 16:23:09 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:23:36 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_failure_0 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:23:36 2021 | rc=1 id=apc_snmp_node1_an-pdu04_last_0 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (unpack_rsc_op_failure) warning: Unexpected result (error) was recorded for start of apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:23:22 2021 | rc=1 id=apc_snmp_node1_an-pdu03_last_failure_0 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu03 has failed INFINITY times on an-a02n01 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n01 after 1000000 failures (max=1000000) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu03 has failed INFINITY times on an-a02n02 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pe_get_failcount) info: apc_snmp_node1_an-pdu04 has failed INFINITY times on an-a02n02 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (check_migration_threshold) warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after 1000000 failures (max=1000000) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node2_an-pdu03 cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv13-psql cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv14-file-processor cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv15-tickets cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv16-machine-interface cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv17-order-processing cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource srv18-inventory cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__native_allocate) info: Resource apc_snmp_node1_an-pdu03 cannot run anywhere Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (RecurringOp) info: Start recurring monitor (60s) for apc_snmp_node1_an-pdu04 on an-a02n01 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu03 (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node2_an-pdu04 (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node2 (Started an-a02n02) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node2 (Started an-a02n02) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave ipmilan_node1 (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave delay_node1 (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv13-psql (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv14-file-processor (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv15-tickets (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv16-machine-interface (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv17-order-processing (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv18-inventory (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv19-uls (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv20-fileprocessor (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv21-tickets (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv22-machineinterface (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv23-orderprocessing (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave srv24-inventory (Started an-a02n01) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (rsc_action_default) info: Leave apc_snmp_node1_an-pdu03 (Stopped) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (log_list_item) notice: Actions: Recover apc_snmp_node1_an-pdu04 ( an-a02n02 -> an-a02n01 ) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-schedulerd[121632] (pcmk__log_transition_summary) notice: Calculated transition 119, saving inputs in /var/lib/pacemaker/pengine/pe-input-140.bz2 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-controld [121633] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-controld [121633] (do_te_invoke) info: Processing graph 119 (ref=pe_calc-dc-1627082629-379) derived from /var/lib/pacemaker/pengine/pe-input-140.bz2 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating stop operation apc_snmp_node1_an-pdu04_stop_0 on an-a02n02 | action 11 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.28 2 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.29 (null) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=29 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_0']: @operation_key=apc_snmp_node1_an-pdu04_stop_0, @operation=stop, @transition-key=11:119:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;11:119:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=16 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/144, version=0.351.29) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.29 2 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.30 (null) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=30 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_0']: @transition-magic=0:0;11:119:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=128, @rc-code=0, @op-status=0 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n02/crmd/145, version=0.351.30) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-controld [121633] (process_graph_event) info: Transition 119 action 11 (apc_snmp_node1_an-pdu04_stop_0 on an-a02n02) confirmed: ok | rc=0 call-id=128 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-controld [121633] (te_rsc_command) notice: Initiating start operation apc_snmp_node1_an-pdu04_start_0 locally on an-a02n01 | action 35 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-controld [121633] (do_lrm_rsc_op) notice: Requesting local execution of start operation for apc_snmp_node1_an-pdu04 on an-a02n01 | transition_key=35:119:0:789131c0-5164-4a52-9822-30c5026c3827 op_key=apc_snmp_node1_an-pdu04_start_0 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/580) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-execd [121630] (log_execute) info: executing - rsc:apc_snmp_node1_an-pdu04 action:start call_id:146 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.30 2 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.351.31 (null) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=31 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_0']: @operation_key=apc_snmp_node1_an-pdu04_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=35:119:0:789131c0-5164-4a52-9822-30c5026c3827, @transition-magic=-1:193;35:119:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=-1, @rc-cod Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=an-a02n01/crmd/580, version=0.351.31) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_apply_diff operation for section 'all' to all (origin=local/cibadmin/2) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.351.31 2 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.352.0 (null) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: -- /cib/configuration/resources/primitive[@id='apc_snmp_node2_an-pdu03']/meta_attributes[@id='apc_snmp_node2_an-pdu03-meta_attributes']/nvpair[@id='apc_snmp_node2_an-pdu03-meta_attributes-target-role'] Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @epoch=352, @num_updates=0 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=an-a02n01/cibadmin/2, version=0.352.0) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-controld [121633] (abort_transition_graph) notice: Transition 119 aborted by deletion of nvpair[@id='apc_snmp_node2_an-pdu03-meta_attributes-target-role']: Configuration change | cib=0.352.0 source=te_update_diff_v2:474 path=/cib/configuration/resources/primitive[@id='apc_snmp_node2_an-pdu03']/meta_attributes[@id='apc_snmp_node2_an-pdu03-meta_attributes']/nvpair[@id='apc_snmp_node2_an-pdu03-meta_attributes-target-role'] complete=false Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-fenced [121629] (update_cib_stonith_devices_v2) info: Updating device list from CIB: (meta) attribute deleted from resource Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-fenced [121629] (cib_devices_update) info: Updating devices to version 0.352.0 Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-fenced [121629] (stonith_device_register) notice: Added 'apc_snmp_node2_an-pdu03' to device list (8 active devices) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-46.raw Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Wrote version 0.352.0 of the CIB to disk (digest: 034e1af9317bc5065394204374d1e37d) Jul 23 16:23:49 an-a02n01.alteeve.com pacemaker-based [121628] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.O9VZWx (digest: /var/lib/pacemaker/cib/cib.e0DYvA) Jul 23 16:23:54 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_ping) info: Reporting our current digest to an-a02n01: 0102f40f964e86fc343ccb59bd571c29 for 0.352.0 (0x5619760950e0 0) Jul 23 16:23:55 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_1[4160914] error output [ 2021-07-23 16:23:55,695 ERROR: Returned 1: Timeout: No Response from 10.201.2.2:161 ] Jul 23 16:23:55 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_1[4160914] error output [ ] Jul 23 16:23:55 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_1[4160914] error output [ ] Jul 23 16:23:55 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_1[4160914] error output [ 2021-07-23 16:23:55,696 ERROR: Please use '-h' for usage ] Jul 23 16:23:55 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_1[4160914] error output [ ] Jul 23 16:23:55 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4160914] stderr: [ 2021-07-23 16:23:55,695 ERROR: Returned 1: Timeout: No Response from 10.201.2.2:161 ] Jul 23 16:23:55 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4160914] stderr: [ ] Jul 23 16:23:55 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4160914] stderr: [ ] Jul 23 16:23:55 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4160914] stderr: [ 2021-07-23 16:23:55,696 ERROR: Please use '-h' for usage ] Jul 23 16:23:55 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4160914] stderr: [ ] Jul 23 16:23:55 an-a02n01.alteeve.com pacemaker-fenced [121629] (internal_stonith_action_execute) info: Attempt 2 to execute fence_apc_snmp (monitor). remaining timeout is 14 Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_2[4161223] error output [ 2021-07-23 16:24:02,856 ERROR: Returned 1: Timeout: No Response from 10.201.2.2:161 ] Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_2[4161223] error output [ ] Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_2[4161223] error output [ ] Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_2[4161223] error output [ 2021-07-23 16:24:02,856 ERROR: Please use '-h' for usage ] Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_op_output) notice: fence_apc_snmp_monitor_2[4161223] error output [ ] Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4161223] stderr: [ 2021-07-23 16:24:02,856 ERROR: Returned 1: Timeout: No Response from 10.201.2.2:161 ] Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4161223] stderr: [ ] Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4161223] stderr: [ ] Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4161223] stderr: [ 2021-07-23 16:24:02,856 ERROR: Please use '-h' for usage ] Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_action) warning: fence_apc_snmp[4161223] stderr: [ ] Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-fenced [121629] (update_remaining_timeout) info: Attempted to execute agent fence_apc_snmp (monitor) the maximum number of times (2) allowed Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-fenced [121629] (log_operation) notice: Operation 'monitor' [4161223] using apc_snmp_node1_an-pdu04 returned -201 (Generic Pacemaker error) Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-execd [121630] (log_finished) info: apc_snmp_node1_an-pdu04 start (call 146) exited with status 1 (execution time 13323ms, queue time 0ms) Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-controld [121633] (process_lrm_event) notice: Result of start operation for apc_snmp_node1_an-pdu04 on an-a02n01: error | rc=1 call=146 key=apc_snmp_node1_an-pdu04_start_0 confirmed=true cib-update=581 Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-based [121628] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/581) Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: --- 0.352.0 2 Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: Diff: +++ 0.352.1 (null) Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib: @num_updates=1 Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']/lrm_rsc_op[@id='apc_snmp_node1_an-pdu04_last_0']: @transition-magic=0:1;35:119:0:789131c0-5164-4a52-9822-30c5026c3827, @call-id=146, @rc-code=1, @op-status=0, @exec-time=13323 Jul 23 16:24:02 an-a02n01.alteeve.com pacemaker-based [121628] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='apc_snmp_node1_an-pdu04']: