Skip to main content
  1. Home >
  2. Products >
  3. Computing Products >
  4. Servers >
  5. Fujitsu SPARC servers >
  6. Downloads >
  7. User Manuals >
  8. Fujitsu M10-4S/SPARC M10-4S Installation Guide >
  9. 10.2.1 Removing the server while the target physical partition (PPAR) is operating

Contents

  1. Preface
  2. Chapter 1 Understanding the Installation Flow
    1. Click 1.1 Workflow for the SPARC M10-4S
    2. Click 1.2 Workflow for Installing the PCI Expansion Unit
  3. Chapter 2 Planning and Preparing for System Installation
    1.  2.1 Safety Precautions
    2.  2.2 Items Requiring Confirmation before Installation
    3. Click 2.3 Confirming the Physical Specifications of the System
    4. Click 2.4 Confirming Rack Specifications
    5. Click 2.5 Checking Environmental Conditions
    6.  2.6 Checking Acoustic Noise Levels
    7.  2.7 Checking Cooling Conditions
    8. Click 2.8 Checking the Power Input Type
    9. Click 2.9 Preparing Power Supply Facilities
    10. Click 2.10 Checking External Interface Port Specifications
    11.  2.11 Checking the Functions of the Operation Panel
  4. Chapter 3 Installing the System
    1.  3.1 Preparing the Necessary Tools/Information for Installation
    2. Click 3.2 Confirming Delivered Components
    3. Click 3.3 Installing a Rack
    4. Click 3.4 Mounting the Chassis in a Rack
    5. Click 3.5 Mounting Optional Components
  5. Chapter 4 Configuring Building Block Connections
    1.  4.1 Setting the ID (BB-ID) Identifying a Chassis
    2. Click 4.2 Connecting Cables (for Direct Connections between Chassis)
    3. Click 4.3 Connecting Cables (for Connections through Crossbar Boxes)
  6. Chapter 5 Connecting Cables to the Chassis
    1.  5.1 Connecting Cables to the SPARC M10-4S
    2.  5.2 Connecting Cables to the PCI Expansion Unit
    3.  5.3 Connecting Cables to a Crossbar Box
    4. Click 5.4 Storing Cables
  7. Chapter 6 Performing an Initial System Diagnosis
    1.  6.1 Connecting the System Management Terminal to the Chassis
    2. Click 6.2 Turning On the Input Power
    3.  6.3 Logging In to the XSCF
    4.  6.4 Checking the XCP Version
    5.  6.5 Checking the Altitude Setting
    6.  6.6 Checking the Time Setting
    7.  6.7 Performing a Diagnosis Test
    8.  6.8 Checking the Component Status
  8. Chapter 7 Making the Initial System Settings
    1.  7.1 Setting the Password Policy
    2.  7.2 Setting a User Account and Password
    3. Click 7.3 Configuring the Telnet/SSH Service
    4.  7.4 Configuring the HTTPS Service
    5. Click 7.5 Configuring the XSCF Network
    6.  7.6 Configuring Memory Mirroring
    7.  7.7 Creating a Physical Partition Configuration List (PCL)
    8.  7.8 Assigning a System Board (PSB) to a Physical Partition (PPAR)
    9.  7.9 Setting a CPU Operational Mode for the Physical Partition
    10.  7.10 Synchronizing the Physical Partition (PPAR) Time and XSCF Time
    11. Click 7.11 Registering a CPU Activation Key
    12.  7.12 Assigning CPU Core Resources to a Physical Partition
    13.  7.13 Starting and Stopping a Physical Partition (PPAR)
    14. Click 7.14 Saving Configuration Information
  9. Chapter 8 Before Expanding/Reducing a System with a Building Block Configuration
    1.  8.1 Confirming the System Configuration
    2. Click 8.2 Confirming the Expansion Pattern of a Building Block Configuration
    3. Click 8.3 Confirming the Reduction Pattern of a Building Block Configuration
  10. Chapter 9 Expanding a System with a Building Block Configuration
    1. Click 9.1 Preparing the Necessary Tools and Delivered Components for Expansion
    2. Click 9.2 Adding the SPARC M10-4S
    3. Click 9.3 Adding Expansion Rack 1 (Expansion to Up to the 8BB Configuration)
    4.  9.4 Adding Expansion Rack 2
    5. Click 9.5 Adding Expansion Racks 1 and 2
  11. Chapter 10 Removing a System with a Building Block Configuration
    1.  10.1 Preparing the Necessary Tools for Removal
    2. Click 10.2 Removing the SPARC M10-4S
    3.  10.3 Removal of Expansion Rack 2, and Decrease of the Number of SPARC M10-4S Units to Four or Less
  12. Appendix A Troubleshooting
    1.  A.1 Understanding the Usual Problems and Their Corrective Actions
    2. Click A.2 Understanding Commands for Troubleshooting
    3. Click A.3 How to Restore a BB-ID from an Incorrect Setting
  13. Appendix B Cable Connection Information on Building Block Configurations
    1.  B.1 2BB Configuration (Direct Connections between Chassis)
    2.  B.2 3BB Configuration (Direct Connections between Chassis)
    3.  B.3 4BB Configuration (Direct Connections between Chassis)
    4.  B.4 2BB Configuration to 8BB Configuration (Connections through Crossbar Boxes)
    5.  B.5 9BB Configuration to 16BB Configuration (Connections through Crossbar Boxes)
    6.  B.6 Power Cord Connections in Expansion Racks
  14. Appendix C Setup Command Workflow
  15. Appendix D Installation Procedure Checklist
    1.  D.1 From Installation of a Standalone Configuration to Initial Diagnosis
    2.  D.2 From Installation of a Configuration with Direct Connections between Chassis to Initial Diagnosis
    3.  D.3 From Installation of a Crossbar Box Connection Configuration to Initial Diagnosis

10.2.1 Removing the server while the target physical partition (PPAR) is operating


10.2.1 Removing the server while the target physical partition (PPAR) is operating
This section describes the removal procedure on the system in "Figure 10-1 2BB configuration example" from a 2BB configuration to a 1BB configuration while the target physical partition (PPAR) is in operation. To perform maintenance while the physical partition (PPAR) is operating, use dynamic reconfiguration (DR) for the physical partition.
For the software requirements required for this operation, see the latest Fujitsu M10/SPARC M10 Systems Product Notes and Oracle VM Server for SPARC 3.2 Administration Guide.
For details on the XSCF commands executed in each step, see the Fujitsu SPARC M12 and Fujitsu M10/SPARC M10 XSCF Reference Manual.
Figure 10-1  2BB configuration example
Figure 10-1 2BB configuration example
  1. Log in to the master XSCF.
    Execute the showbbstatus command to confirm that the XSCF to which you have logged in is the master XSCF.
    If it is a standby XSCF, retry to log in to the master XSCF.
XSCF> showbbstatus
BB#00 (Master)
  1. Execute the showhardconf command, and confirm the chassis serial number of the system board (PSB) targeted for release.
    For details, see "A.2.1 Checking the component status."
Note - If the serial number of the target chassis is being used as the system serial number, the initbb command cannot release the chassis. In this case, execute the switchscf command to switch the master XSCF.
  1. Execute the console command to connect to the console of the control domain, and then log in.
XSCF> console -p 0
  1. Check the operation status and the resource use status of logical domains.
  1. a. Execute the ldm list-domain command to check the operation status of the logical domains.

    Check the operation status of the logical domains through the combination of [STATE] and [FLAGS]. When [STATE] is "active," the second character from the left of the string in [FLAGS] has one of the following meanings:

    "n": Oracle Solaris operating

    "t": OpenBoot PROM state

    "-": Another state (including cases in which [STATE] is not "active")

    In this example, you can see that the control domain, two guest domains, and two root domains are operating.
# ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- UART 16 14G 0.0% 10h 7m
guest0 active -n---- 5100 32 32G 0.0% 2h 20s
guest1 active -n---- 5101 32 32G 0.0% 2h 5s
root-dom0 active -n--v- 5000 24 24G 0.0% 2h 43s
root-dom1 active -n--v- 5001 24 24G 0.0% 2h 20s
  1. b. Execute the ldm list-devices command with the -a option specified to check the resource use status.

    The following example specifies the -a option to display all the resources bound to logical domains and all those not bound to logical domains.
# ldm list-devices -a
CORE
ID %FREE CPUSET
0 0 (0, 1)
4 0 (8, 9)
8 0 (16, 17)
(Omitted)
184 0 (368, 369)
188 100 (376, 377)
512 100 (1024, 1025)
516 100 (1032, 1033)
520 100 (1040, 1041)
524 100 (1048, 1049)
(Omitted)
VCPU
PID %FREE PM
0 0 no
1 0 no
8 0 no
9 0 no
(Omitted)
369 0 no
376 100 ---
377 100 ---
1024 100 ---
1025 100 ---
1032 100 ---
1033 100 ---
(Omitted)
  1. Release redundant configurations of system volumes and I/O devices of the control domain.
    This example shows the procedure to release the I/O device of the SPARC M10-4S to be removed that is used in the control domain so that the building block BB-ID#01 can be released. If other redundant software is in use, see the document of each redundant software for details on the procedure to release the configuration.
  1. a. Release redundant configurations of system volumes of the control domain.

    The following example shows the procedure to release the ZFS mirror function of system volumes of the control domain.

    a-1) Execute the zpool status command on the control domain to check the state of the mirror configuration.
# zpool status rpool
pool: rpool
state: ONLINE
scan: resilvered 28.7M in 0h0m with 0 errors on Tue Jan 21 10:10:01 2014
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c2t50000393E802CCE2d0s0 ONLINE 0 0 0
c3t50000393A803B13Ed0s0 ONLINE 0 0 0
errors: No known data errors
  1. a-2) Execute the zpool detach command to release the disk from the mirror configuration.
# zpool detach rpool c3t50000393A803B13Ed0
  1. a-3) Execute the zpool status command to confirm that the mirror configuration is released.
# zpool status rpool
pool: rpool
state: ONLINE
scan: resilvered 28.7M in 0h0m with 0 errors on Tue Jan 21 10:10:01 2014
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c2t50000393E802CCE2d0s0 ONLINE 0 0 0
errors: No known data errors
  1. If there are any other devices on BB#01 in use, release the redundant configurations and stop using the devices. For how to release the redundant configurations and how to stop using the devices, refer to the documents of software in the redundant configurations and Oracle Solaris.
  1. b. Delete the I/O configuration of the control domain.

    b-1) Out of the physical I/O devices assigned to the control domain, delete the root complex of BB#01.
  1. b-2) Change the control domain to the delayed reconfiguration mode.

    This operation is not required if the root complexes are reconfigured dynamically.

        The following shows the software conditions to reconfigure the root complexes dynamically.

         - XCP 2240 or later

         - Oracle VM Server for SPARC 3.2 or later

         - Oracle Solaris 11.2 SRU11.2.8 or later
# ldm start-reconf primary
Initiating a delayed reconfiguration operation on the primary domain.
All configuration changes for other domains are disabled until the primary
domain reboots, at which time the new configuration for the primary domain
will also take effect.
  1. b-3) Execute the ldm list-io command to check the root complexes assigned to the primary domain.

    The following example tells that the root complexes with the device of BB1 are PCIE8 and PCIE12.
# ldm list-io | grep primary
PCIE0 BUS PCIE0 primary IOV
PCIE4 BUS PCIE4 primary IOV
PCIE8 BUS PCIE8 primary IOV
PCIE12 BUS PCIE12 primary IOV
/BB0/CMUL/NET0 PCIE PCIE0 primary OCC
/BB0/CMUL/SASHBA PCIE PCIE0 primary OCC
/BB0/CMUL/NET2 PCIE PCIE4 primary OCC
/BB1/CMUL/NET0 PCIE PCIE8 primary OCC
/BB1/CMUL/SASHBA PCIE PCIE8 primary OCC
/BB1/CMUL/NET2 PCIE PCIE12 primary OCC
  1. b-4) Execute the ldm remove-io command to delete PCIE8 and PCIE12 from the domain "primary".
# ldm remove-io PCIE8 primary
# ldm remove-io PCIE12 primary
  1. b-5) Restart Oracle Solaris.

    This operation is not required if the root complexes are reconfigured dynamically.
# shutdown -i6 -g0 -y
  1. b-6) Execute the ldm list-io command to confirm that the root complexes of BB#01 are deleted from the control domain.
# ldm list-io | grep primary
PCIE0 BUS PCIE0 primary IOV
PCIE4 BUS PCIE4 primary IOV
/BB0/CMUL/NET0 PCIE PCIE0 primary OCC
/BB0/CMUL/SASHBA PCIE PCIE0 primary OCC
/BB0/CMUL/NET2 PCIE PCIE4 primary OCC
  1. c. Release the redundant configurations of virtual I/O devices assigned to the guest domain.

    To shut down the root domain (root-dom1) with the root complexes of BB#01 assigned before deleting the I/O devices, log in to each guest domain and release the redundant configuration of the virtual I/O device from root-dom1.

    For details on how to use software in redundant configurations, see the document of each software in the redundant configuration.
  1. The following shows an example of releasing the virtual network device (vnet1) from the configuration of IPMP. For command details, see the Oracle Solaris manual.

    c-1) Log in to the guest domain (guest0).
# ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- UART 64 56G 0.0% 4h 17m
guest0 active -n---- 5100 64 64G 0.0% 1h 13m
guest1 active -n---- 5101 64 64G 0.0% 1h 4m
root-dom0 active -n--v- 5000 32 32G 0.0% 1h 47m
root-dom1 active -n--v- 5001 32 32G 0.0% 1h 19m
# telnet localhost 5100
....
guest0#
  1. c-2) Execute the dladm show-phys command to check the correspondence between the virtual network interface (vnet1) and the network interface name (net1).
  1. c-3) Execute the ipmpstat -i command to confirm the configuration information of the network interface that configures IPMP.
guest0# ipmpstat -i
INTERFACE ACTIVE GROUP FLAGS LINK PROBE STATE
net0 yes ipmp0 -smbM-- up disabled ok
net1 no ipmp0 is----- up disabled ok
guest0# if_mpadm -d net1
guest0# ipmpstat -i
INTERFACE ACTIVE GROUP FLAGS LINK PROBE STATE
net0 yes ipmp0 -smbM-- up disabled ok
net1 no ipmp0 -s---d- up disabled offline
  1. c-4) Execute the if_mpadm -d command, release net1 from the IPMP group, and execute the ipmpstat -i command to confirm that net1 is released. In the next example, confirm that STATE becomes offline.

    Perform the same release work on the guest domain (guest1).
guest1# if_mpadm -d net1
guest1# ipmpstat -i
INTERFACE ACTIVE GROUP FLAGS LINK PROBE STATE
net0 yes ipmp0 -smbM-- up disabled ok
net1 no ipmp0 -s---d- up disabled offline
  1. d. Delete the virtual I/O devices assigned from the root domain to be stopped.

    Execute the ldm remove-vdisk command and the ldm remove-vnet command, and perform the following procedure to delete the virtual disk (vdisk) and the virtual network device (vnet) assigned from the root domain to be stopped.

    The following shows an example of executing the command to delete the virtual disk (vdisk11) and the virtual network device (vnet10) that uses the virtual I/O service of the BB#01 root domain (root1-dom1).
# ldm remove-vdisk vdisk11 guest0
# ldm remove-vnet vnet10 guest0
  1. Perform the same deletion work on the guest domain (guest1).
  1. Check the resource use status of the I/O devices and release all the I/O devices of the SPARC M10-4S to be removed.
  1. a. Confirm the logical domain where the root complexes of the SPARC M10-4S to be released are assigned.

    Execute the ldm list-io command to check the logical domain where the root complexes of BB#01 are assigned.

    In the next example, only root-dom1 owns the PCIe end point that starts with "/BB1/". Then, PCIE9, PCIE10, PCIE11, PCIE13, PCIE14, and PCIE15, which are the root complexes (BUS) of the PCIe end point, are assigned to root-dom1.
# ldm list-io
NAME TYPE BUS DOMAIN STATUS
---- ---- --- ------ ------

PCIE0 BUS PCIE0 primary IOV
PCIE1 BUS PCIE1 root-dom0IOV
PCIE2 BUS PCIE2 root-dom0IOV
PCIE3 BUS PCIE3 root-dom0IOV
PCIE4 BUS PCIE4 primary IOV
PCIE5 BUS PCIE5 root-dom0IOV
PCIE6 BUS PCIE6 root-dom0IOV
PCIE7 BUS PCIE7 root-dom0IOV
PCIE8 BUS PCIE8
PCIE9 BUS PCIE9 root-dom1IOV
PCIE10 BUS PCIE10 root-dom1IOV
PCIE11 BUS PCIE11 root-dom1IOV
PCIE12 BUS PCIE12
PCIE13 BUS PCIE13 root-dom1IOV
PCIE14 BUS PCIE14 root-dom1IOV
PCIE15 BUS PCIE15 root-dom1IOV
....
/BB1/CMUL/NET0 PCIE PCIE8 UNK
/BB1/CMUL/SASHBA PCIE PCIE8 UNK
/BB1/PCI0 PCIE PCIE9 root-dom1OCC
/BB1/PCI3 PCIE PCIE10 root-dom1OCC
/BB1/PCI4 PCIE PCIE10 root-dom1OCC
/BB1/PCI7 PCIE PCIE11 root-dom1OCC
/BB1/PCI8 PCIE PCIE11 root-dom1OCC
/BB1/CMUL/NET2 PCIE PCIE12 UNK
/BB1/PCI1 PCIE PCIE13 root-dom1OCC
/BB1/PCI2 PCIE PCIE13 root-dom1OCC
/BB1/PCI5 PCIE PCIE14 root-dom1OCC
/BB1/PCI6 PCIE PCIE14 root-dom1OCC
/BB1/PCI9 PCIE PCIE15 root-dom1OCC
/BB1/PCI10 PCIE PCIE15 root-dom1OCC
  1. b. Stop and release the root domain where the root complexes of the SPARC M10-4S to be released are assigned.

    In the next example, we can confirm that the ldm stop-domain command and the ldm unbind-domain command are executed, the root domain (root-dom1) is released, and the root domain enters the inactive state.
# ldm stop-domain root-dom1
LDom root-dom1 stopped
# ldm unbind-domain root-dom1
# ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- UART 16 14G 0.2% 4h 59m
guest0 active -n---- 5100 32 32G 0.0% 1h 55m
guest1 active -n---- 5101 32 32G 0.0% 1h 46m
root-dom0 active -n--v- 5000 24 24G 0.0% 2h 29m
root-dom1 inactive ------ 24 24G
  1. c. Confirm all the I/O devices of the building block to be removed are released.

    Execute the ldm list-io command to confirm that the I/O devices are released.
# ldm list-io
NAME TYPE BUS DOMAIN STATUS
---- ---- --- ------ ------

PCIE0 BUS PCIE0 primary IOV
PCIE1 BUS PCIE1 root-dom0IOV
PCIE2 BUS PCIE2 root-dom0IOV
PCIE3 BUS PCIE3 root-dom0IOV
PCIE4 BUS PCIE4 primary IOV
PCIE5 BUS PCIE5 root-dom0IOV
PCIE6 BUS PCIE6 root-dom0IOV
PCIE7 BUS PCIE7 root-dom0IOV
PCIE8 BUS PCIE8
PCIE9 BUS PCIE9
PCIE10 BUS PCIE10
PCIE11 BUS PCIE11
PCIE12 BUS PCIE12
PCIE13 BUS PCIE13
PCIE14 BUS PCIE14
PCIE15 BUS PCIE15
(Omitted)
  1. Go back to the XSCF shell and confirm the system board status of the SPARC M10-4S to be removed.
    Execute the showboards command to check the system board status.
    Confirm that the system board of the SPARC M10-4S to be removed is in the Assigned state and "y" is displayed for all of the [Pwr], [Conn], and [Conf] columns.
XSCF> showboards -p 0
PSB  PPAR-ID(LSB) Assignment  Pwr  Conn Conf Test    Fault
---- ------------ ----------- ---- ---- ---- ------- --------

00-0 00(00)       Assigned    y    y    y    Passed  Normal
01-0 00(01)       Assigned    y    y    y    Passed  Normal
  1. Release the system board from the physical partition.
  1. a. Execute the deleteboard command to release the system board (PSB) from the physical partition.

    The following example releases system board 01-0 from the physical partition to be in the system board pool.
XSCF> deleteboard -c unassign 01-0
PSB#01-0 will be unassigned from PPAR immediately. Continue?[y|n] :y
Start unconfigure preparation of PSB. [1200sec]
0end
Unconfigure preparation of PSB has completed.
Start unconfiguring PSB from PPAR. [43200sec]
0..... 30.end
Unconfigured PSB from PPAR.
PSB power off sequence started. [1200sec]
0..... 30..... 60..... 90.....120.....150.end
Operation has completed.
Note - In releasing the system board (PSB) by the deleteboard command, the hardware resource on the system board (PSB) is released from Oracle Solaris. Therefore, the command execution may take some time to complete.
  1. b. Execute the showresult command to confirm the end status of the previously executed deleteboard command.

    In the following example, 0 is returned as the end status, so the execution of the deleteboard command has completed correctly.
XSCF> showresult
0
  1. c. Execute the showboards command to check the system board (PSB) status.

    In the following example, system board 01-0 is in the system board pool.
XSCF> showboards -p 0
PSB PPAR-ID(LSB) Assignment Pwr Conn Conf Test Fault
---- ------------ ----------- ---- ---- ---- ------- --------

00-0 00(00) Assigned y y y Passed Normal
01-0 SP Available n n n Passed Normal
  1. Execute the ldm list-domain command on the control domain console of the physical partition, and confirm that the operation status of the logical domains has not changed after the system board (PSB) deletion.
  1. Execute the initbb command from the master XSCF to release the target SPARC M10-4S from the system and initialize it.
    In bb_id, specify the ID (BB-ID) identifying the chassis.
XSCF> initbb -b bb_id
Note - After the execution of the initbb command, the target chassis is released from the system and enters the stopped state. Do not disconnect the power cords or various cables until the XSCF STANDBY LED on the panel and the rear READY LED of the target chassis are off.
  1. Remove the power cords from the power supply units of the target SPARC M10-4S.
  2. Remove the XSCF BB control cables from the target SPARC M10-4S.
  3. Remove the crossbar cables from the target SPARC M10-4S.
Note - Do not hold the cable part when pulling out the crossbar cable. Pulling the cable part without the connector lock completely released may cause damage.
  1. You are performing removal to the 1BB configuration. So, remove the XSCF DUAL control cable.
  2. Remove the target SPARC M10-4S from the rack.
    Remove the SPARC M10-4S from the rack by reversing the mounting procedure. For the procedure for mounting in the rack, see "3.4.1 Mounting the SPARC M10-4S in a rack."
The removed SPARC M10-4S enters the factory default status because the initbb command is executed. If you want to use it as another system, see "1.1 Workflow for the SPARC M10-4S" to perform installation.

Top of Page