Skip to main content

Contents

  1. Preface
  2. Chapter 1 Before Starting Maintenance Work
    1.  1.1 Warning/Caution Indications
    2.  1.2 Warning Labels
    3.  1.3 Labels/Tags
    4.  1.4 Safety Precautions
    5.  1.5 Precautions on Static Electricity
    6.  1.6 Other Precautions
    7.  1.7 Emergency Power Off
  3. Chapter 2 Understanding the System Components
    1.  2.1 Identifying the Names and Locations of Components
    2. Click 2.2 Checking the Memory Configuration Rules
    3. Click 2.3 Confirming the Functions of the Operation Panel
    4. Click 2.4 Checking the LED Indications
    5. Click 2.5 Confirming the Types of Cable
  4. Chapter 3 Types of Maintenance
    1.  3.1 Types of Maintenance Supported in the SPARC M10-4/M10-4S
    2.  3.2 Active Maintenance
    3.  3.3 Inactive Maintenance
    4.  3.4 System-stopped Maintenance
  5. Chapter 4 Preparation and Precautions for Maintenance
    1. Click 4.1 Confirming the System Configuration
    2. Click 4.2 Troubleshooting
    3. Click 4.3 Maintenance Precautions
  6. Chapter 5 Preparations Required for Maintenance
    1.  5.1 Preparing Tools Required for Maintenance
    2.  5.2 Switching the Mode Switch to Service Mode
    3. Click 5.3 Checking the Operating Condition and Resource Usage Status
    4. Click 5.4 Preparations for Enabling Active Maintenance
    5.  5.5 Powering Off the Physical Partition Requiring Maintenance
    6. Click 5.6 Stopping the Entire System
    7.  5.7 Saving XSCF Setting Information
    8. Click 5.8 Releasing a SPARC M10-4/M10-4S FRU from the System with the replacefru Command
    9. Click 5.9 Accessing a FRU
  7. Chapter 6 Restoring the System
    1. Click 6.1 Restoring the Chassis
    2. Click 6.2 Incorporating a SPARC M10-4/M10-4S FRU into the System with the replacefru Command
    3. Click 6.3 Diagnosing a Replacement FRU
    4.  6.4 Restoring XSCF Setting Information
    5. Click 6.5 Incorporating a Chassis and I/O Device into a Physical Partition
    6.  6.6 Returning the Mode Switch to Locked Mode
    7.  6.7 Powering On the Physical Partition Requiring Maintenance
    8. Click 6.8 Starting the Entire System
  8. Chapter 7 Maintenance Flow
    1.  7.1 Maintenance Workflow
    2. Click 7.2 FRU Replacement Workflow
    3. Click 7.3 FRU Installation Workflow
    4. Click 7.4 FRU Removal Workflow
  9. Chapter 8 Maintaining the PCI Express Cards
    1.  8.1 Before Maintaining a PCI Express Card
    2.  8.2 PCI Express Card Configuration
    3.  8.3 FRU Removal and Installation Flow
    4. Click 8.4 Removing a PCI Express Card
    5. Click 8.5 Installing a PCI Express Card
  10. Chapter 9 Maintaining the CPU Memory Unit/Memory
    1.  9.1 Before Maintaining a CPU Memory Unit/Memory
    2. Click 9.2 Configuration of the CPU Memory Unit and Memory
    3.  9.3 Maintenance Precautions
    4.  9.4 FRU Removal and Installation Flow
    5. Click 9.5 Removing a CPU Memory Unit
    6.  9.6 Installing a PCI Express Cable
    7.  9.7 Switching the microSD Card
    8.  9.8 Removing Memory
    9.  9.9 Installing Memory
    10. Click 9.10 Installing a CPU Memory Unit
    11.  9.11 Checking the XCP Firmware Version
    12.  9.12 Checking the XCP Firmware Version (Building Block Configuration)
  11. Chapter 10 Maintaining the Internal Disks
    1.  10.1 Before Maintaining an Internal Disk
    2.  10.2 Configuration of the Internal Disks
    3.  10.3 FRU Removal and Installation Flow
    4.  10.4 Removing an Internal Disk
    5.  10.5 Installing an Internal Disk
  12. Chapter 11 Maintaining the Fan Units
    1.  11.1 Before Maintaining a Fan Unit
    2.  11.2 Configuration of the Fan Units
    3.  11.3 Removing a Fan Unit
    4.  11.4 Installing a Fan Unit
  13. Chapter 12 Maintaining the Power Supply Units
    1.  12.1 Before Maintaining a Power Supply Unit
    2.  12.2 Configuration of the Power Supply Units
    3.  12.3 Removing a Power Supply Unit
    4.  12.4 Installing a Power Supply Unit
  14. Chapter 13 Maintaining the PSU Backplane Unit/Operation Panel
    1.  13.1 Before Maintaining the PSU Backplane Unit/Operation Panel
    2.  13.2 Locations of the PSU Backplane Unit and Operation Panel
    3.  13.3 Maintenance Precautions
    4.  13.4 FRU Removal and Installation Flow
    5. Click 13.5 Removing the PSU Backplane Unit/Operation Panel
    6. Click 13.6 Installing the PSU Backplane Unit/Operation Panel
  15. Chapter 14 Maintaining the Crossbar Cables
    1.  14.1 Before Maintaining a Crossbar Cable
    2.  14.2 Configuration of the Crossbar Cable Connection Ports
    3.  14.3 Removing the Crossbar Cables
    4.  14.4 Installing the Crossbar Cables
  16. Chapter 15 Maintaining the Crossbar Units
    1.  15.1 Before Maintaining a Crossbar Unit
    2.  15.2 Configuration of the Crossbar Units
    3.  15.3 Removing a Crossbar Unit
    4.  15.4 Installing a Crossbar Unit
  17. Chapter 16 Maintaining the XSCF BB Control Cables
    1.  16.1 Before Maintaining an XSCF BB Control Cable
    2.  16.2 Configuration of the XSCF BB Control Ports
    3.  16.3 Removing an XSCF BB Control Cable
    4.  16.4 Installing an XSCF BB Control Cable
  18. Chapter 17 Maintaining the XSCF DUAL Control Cables
    1.  17.1 Before Maintaining an XSCF DUAL Control Cable
    2.  17.2 Configuration of the XSCF DUAL Control Ports
    3.  17.3 Removing an XSCF DUAL Control Cable
    4.  17.4 Installing an XSCF DUAL Control Cable
  19. Appendix A Component List
  20. Appendix B Component Specifications
    1.  B.1 CPU Memory Unit
    2.  B.2 Crossbar Unit
    3.  B.3 Power Supply Unit
    4.  B.4 Fan Unit
    5.  B.5 Internal Disk Drive
    6.  B.6 PCI Express Card
    7.  B.7 Backplanes
    8.  B.8 Operation Panel
  21. Appendix C Oracle Solaris Troubleshooting Commands
    1.  C.1 iostat Command
    2.  C.2 prtdiag Command
    3.  C.3 prtconf Command
    4.  C.4 netstat Command
    5.  C.5 ping Command
    6.  C.6 ps Command
    7.  C.7 prstat Command
  22. Appendix D External Interface Specifications
    1. Click D.1 Serial Port
    2.  D.2 USB Port
    3.  D.3 SAS Port
    4.  D.4 RESET Switch
  23. Appendix E Removing the Lithium Battery
    1.  E.1 Location of the Lithium Battery
    2.  E.2 Removing the Lithium Battery

5.4.1 Releasing the assignment of I/O devices


5.4.1 Releasing the assignment of I/O devices
This section describes the work to be done before maintenance using PCI hot plug (PHP) or dynamic reconfiguration (PPAR DR) of the physical partition.
  1. When using PCI hot plug (PHP)
    If the PCIe end point of the PCIe card requiring maintenance is assigned to an I/O domain or if the SR-IOV virtual function is assigned, return the PCIe end point to the root or control domain.
    (1) Stopping the use of an I/O device
    (2) Deleting the assignment of an I/O device
  2. When using dynamic reconfiguration (PPAR DR) for the physical partition
    If the PCIe end point of the PCIe card in the chassis requiring maintenance is assigned to an I/O domain or if the SR-IOV virtual function is assigned, return the PCIe end point to the root or control domain. Then, remove the root complex from the root or control domain.
    (1) Stopping the use of an I/O device
    (2) Deleting the assignment of an I/O device
    (3) Deleting the root complex
(1) Stopping the use of an I/O device
If there is any logical domain that is physically or virtually using the PCIe card mounted in the chassis requiring maintenance, stop the use of the I/O device to keep it unused.
With a redundant configuration, use the duplication function to release the target I/O device so that it will not be used. For details on the work, see the manual for the application being used.
With a non-redundant configuration, place the I/O device in the unused state, such as by deactivating the network or unmounting the disk to stop the use. For details on the work, see the Oracle Solaris documents and the manual for the application being used.
(2) Deleting the assignment of an I/O device
When using virtual I/O
When using a virtual I/O service, delete the virtual I/O device. For details, see the Oracle VM Server for SPARC Administration Guide from Oracle.
The following is a command execution example for deleting the virtual disk (vdisk11) and virtual network device (vnet10) that use the virtual I/O service of the control domain or root domain.
# ldm remove-vdisk vdisk11 guest0
# ldm remove-vnet vnet10 guest0
When using PCIe end point assignment
If the PCIe end point device of the chassis requiring maintenance is assigned to an I/O domain, delete the PCIe end point device from the I/O domain and return it to the root domain (includes the control domain). The work procedure varies depending on whether the dynamic reconfiguration function for the PCIe end point device is used. For details, see Table 5-2.
Table 5-2  Procedure that depends on whether the dynamic reconfiguration function for the PCIe end point device is used
-: Unnecessary
Item Task Command Dynamic reconfiguration function used (*1) Dynamic reconfiguration function not used (*2)
1 Enable the hotplug service. svcadm enable hotplug Yes Yes (*3)
2 Stop the I/O domain. ldm stop-domain - Yes
3 Delete the PCIe end point device from the I/O domain. ldm remove-io Yes Yes
4 Set the root domain to delayed reconfiguration. ldm start-reconf - Yes
5 Assign the PCIe end point device to the root domain. ldm add-io Yes Yes
6 Restart the root domain. shutdown -i6 -g0 -y - Yes
7 Disable the hotplug service. svcadm disable hotplug Yes Yes (*3)
*1 Dynamic reconfiguration of the PCIe end point device is supported by XCP 2230 or later and Oracle VM Server for SPARC 3.1.1.1 or later.
*2 If dynamic reconfiguration of the PCIe end point devices is not used, stop the I/O domain, delete the physical I/O device, and then return the physical I/O device to the root domain.
*3 If dynamic reconfiguration of the PCIe end point device is not used, the hotplug service operates regardless of its enable/disable setting.
Note - Note the following when deleting the physical I/O device (PCIe end point device) of the boot disk.
If the boot disk of the I/O domain is in a redundant configuration, you need to stop the I/O domain on Oracle VM Server for SPARC 3.1.1 or earlier. However, you do not need to stop it on Oracle VM Server for SPARC 3.1.1.1 or later.
  1. If the hotplug service is disabled for the I/O and root domains, execute the svcadm enable command to enable the hotplug service.
# svcadm enable hotplug
  1. Stop the I/O domain.
    If you are using the dynamic reconfiguration function for the PCIe end point device, this step is unnecessary. Go to step 3.
# ldm stop-domain <I/O domain name>‌
  1. Delete the physical I/O device from the I/O domain.
# ldm remove-io <‌device name>‌ <I/O domain name>‌
  1. Transition the root domain to delayed reconfiguration mode.
    If you are using the dynamic reconfiguration function for the PCIe end point device, this step is unnecessary. Go to step 5.
# ldm start-reconf <‌root domain name>‌
  1. Reassign the physical I/O device deleted in step 3 to the root domain.
# ldm add-io <‌device name>‌ <‌root domain name>‌
  1. Log in to the root domain, and restart Oracle Solaris.
    If you are using the dynamic reconfiguration function for the PCIe end point device, this step is unnecessary. Go to step 7.
# telnet localhost <‌port number>‌
...
# shutdown -i6 -g0 -y
  1. If you need to disable the hotplug service for the I/O and root domains, disable the hotplug service.
# svcadm disable hotplug
When using the SR-IOV virtual function
Suppose that a virtual function (VF) using the SR-IOV function on the PCIe card mounted in the PCI expansion unit requiring maintenance was created and assigned to a domain. In this case, delete and discard the virtual function from the control domain by using the ldm command. The work procedure differs between the dynamic SR-IOV function and the static SR-IOV function. For details, see Table 5-3.
For details on the procedure, see the Oracle VM Server for SPARC Administration Guide from Oracle.
Table 5-3  Release procedure that depends on the function: dynamic SR-IOV function or static SR-IOV function
-: Unnecessary
Item Task Command Dynamic SR-IOV function used (*1) Static SR-IOV function used
1 Enable the hotplug service. svcadm enable hotplug Yes Yes (*2)
2 Stop the I/O domain. ldm stop-domain - Yes
3 Delete the virtual function (VF) from the I/O domain. ldm remove-io Yes Yes
4 Set the root domain of the physical function (PF) to delayed reconfiguration. ldm start-reconf - Yes
5 Discard the virtual function (VF). ldm destroy-vf Yes Yes
6 Restart the root domain. shutdown -i6 -g0 -y - Yes
7 Disable the hotplug service. svcadm disable hotplug Yes Yes (*2)
*1 Dynamic reconfiguration for the SR-IOV virtual function is supported by XCP 2210 or later and Oracle VM Server for SPARC 3.1 or later.
*2 If static reconfiguration for the SR-IOV virtual function is used, the hotplug service operates regardless of its enable/disable setting.
  1. If the hotplug service is disabled for the I/O and root domains, execute the svcadm enable command to enable the hotplug service.
# svcadm enable hotplug
  1. Stop the I/O domain.
    If the dynamic SR-IOV function is used, this step is unnecessary. Go to step 3.
# ldm stop-domain <I/O domain name>‌
  1. The virtual function (VF) has been assigned to an I/O domain. Delete it from the I/O domain.
    Execute the ldm remove-io command to delete the virtual function (VF) assigned to the I/O domain from the I/O domain. If multiple VFs are assigned from the physical function (PF) of a single PCIe card, delete all the VFs corresponding to that PF from the I/O domain.
# ldm remove-io <VF name>‌ <I/O domain name>‌
  1. Transition the root domain that has the assigned physical function (PF) to delayed reconfiguration mode.
    If the dynamic SR-IOV function is used, this step is unnecessary. Go to step 5.
# ldm start-reconf <‌root domain name>‌
  1. Discard the virtual function (VF).
    Execute the ldm destroy-vf command to discard the virtual function (VF). However, the VF needs to be discarded in the reverse order to that in which it was created.
# ldm destroy-vf <VF name>‌
  1. Log in to the root domain, and restart Oracle Solaris.
    If the dynamic SR-IOV function is used, this step is unnecessary. Go to step 7.
# telnet localhost <‌port number>‌
...
# shutdown -i6 -g0 -y
  1. If you need to disable the hotplug service for the I/O and root domains, disable the hotplug service.
# svcadm disable hotplug
(3) Deleting the root complex
Deleting the root complex from the root domain
If the I/O device of the chassis requiring maintenance is assigned to the root domain, delete the physical I/O device (root complex) from the root domain. The work procedure varies depending on whether the dynamic reconfiguration function (Dynamic PCIe bus assignment) for the root complex is used. For details, see Table 5-4.
Table 5-4  Procedure that depends on whether the dynamic reconfiguration function (Dynamic PCIe bus assignment) for the root complex is used
-: Unnecessary
Item Task Command Dynamic reconfiguration function used (*1) Dynamic reconfiguration function not used (*2)
1 Stop the root domain. ldm stop-domain - Yes
2 Delete the physical I/O device (root complex) from the root domain. ldm remove-io Yes Yes
*1 Dynamic reconfiguration of the root complex is supported by XCP 2240 or later and Oracle VM Server for SPARC 3.2 or later. In addition, the root domain must be Oracle Solaris 11.2 SRU 11.2.8 or later.
*2 If dynamic reconfiguration of the root complex is not used, stop the root domain, and then delete the physical I/O device.
  1. Stop the root domain.
    If you are dynamically reconfiguring the root complex, this step is unnecessary. Go to step 2.
# ldm stop-domain <‌root domain name>‌
  1. Delete the physical I/O device (root complex) from the root domain.
# ldm remove-io <‌physical I/O device name>‌ <‌root domain name>‌
  1. Confirm that the I/O device of the chassis to be released is unused and unassigned.
# ldm list-io
Deleting the root complex from the control domain
If the I/O device of the chassis requiring maintenance is assigned to the control domain, delete the physical I/O device (root complex) from the control domain. The work procedure varies depending on whether the dynamic reconfiguration function (Dynamic PCIe bus assignment) for the root complex is used. For details, see Table 5-5.
Table 5-5  Procedure that depends on whether the dynamic reconfiguration function (Dynamic PCIe bus assignment) for the root complex is used
-: Unnecessary
Item Task Command Dynamic reconfiguration function used (*1) Dynamic reconfiguration function not used (*2)
1 Set the control domain to delayed reconfiguration. ldm start-reconf - Yes
2 Delete the physical I/O device from the control domain. ldm remove-io Yes Yes
3 Restart the control domain. shutdown -i6 -g0 -y - Yes
*1 Dynamic reconfiguration of the root complex is supported by XCP 2240 or later and Oracle VM Server for SPARC 3.2 or later. In addition, the control domain must be Oracle Solaris 11.2 SRU 11.2.8 or later.
*2 If dynamic reconfiguration of the root complex is not used, set the control domain to delayed reconfiguration, and then delete the physical I/O device.
  1. Transition the control domain to delayed reconfiguration mode.
    If you are dynamically reconfiguring the root complex, this step is unnecessary. Go to step 2.
# ldm start-reconf <‌control domain name>‌
  1. Delete the physical I/O device from the control domain.
# ldm remove-io <‌physical I/O device name>‌ <‌control domain name>‌
  1. Restart the control domain.
    If you are dynamically reconfiguring the root complex, this step is unnecessary.
# shutdown -i6 -g0 -y
Note - When set to delayed reconfiguration mode, the control domain does not have control over the guest domains. If you have set the control domain to delayed reconfiguration mode, restart the control domain as soon as possible.

Top of Page