Skip to main content

Contents

  1. Preface
  2. Chapter 1 Before Starting Maintenance Work
    1.  1.1 Warning/Caution Indications
    2.  1.2 Labels/Tags
    3.  1.3 Safety Precautions
    4.  1.4 Notes Regarding Static Electricity
    5.  1.5 Other Precautions
    6.  1.6 Emergency Power Off
    7. Click 1.7 Important Information About the XCP Firmware
  3. Chapter 2 Understanding the PCI Expansion Unit Components
    1.  2.1 Identifying the Names and Locations of Components
    2. Click 2.2 Checking the Status With the LEDs
  4. Chapter 3 Types of Maintenance
    1.  3.1 Maintenance Types of the PCI Expansion Unit
    2.  3.2 Types of Maintenance for the Link Card
    3. Click 3.3 Types of Maintenance for the PCIe Card
  5. Chapter 4 Preparation and Precautions for Maintenance
    1. Click 4.1 Troubleshooting
    2. Click 4.2 Precautions for Maintenance
  6. Chapter 5 Preparations Required for Maintenance
    1.  5.1 Preparing Tools Required for Maintenance
    2.  5.2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition
    3.  5.3 Checking the Firmware Version of the PCI Expansion Unit
    4. Click 5.4 Checking the Operating Condition and Resource Usage Status
    5. Click 5.5 Releasing the Link Card or PCIe Card From the System
    6.  5.6 Stopping the FRU Requiring Maintenance
    7. Click 5.7 Accessing a FRU
  7. Chapter 6 Understanding the Preparations for Restoring the System
    1. Click 6.1 Restoring the Chassis of the PCI Expansion Unit
    2.  6.2 Supplying Power to the FRU Requiring Maintenance
    3. Click 6.3 Mounting a Link Card or PCIe Card on the System
    4. Click 6.4 Restoring the Logical Domain to the Status It Had Before Maintenance
    5.  6.5 Turning Off the CHECK LED (Locator) of the PCI Expansion Unit
    6. Click 6.6 Updating the Firmware of the PCI Expansion Unit
  8. Chapter 7 Maintenance Flow
    1. Click 7.1 FRU Replacement Workflow for the PCI Expansion Unit
    2. Click 7.2 FRU Addition Workflow for the PCI Expansion Unit
    3. Click 7.3 FRU Removal Workflow for the PCI Expansion Unit
    4. Click 7.4 Link Card Maintenance Workflow
  9. Chapter 8 Maintaining the PCI Express Cards
    1.  8.1 Before Maintaining a PCI Express Card
    2.  8.2 Configuration of the PCIe Cards
    3. Click 8.3 Removing a PCI Express Card
    4. Click 8.4 Installing a PCI Express Card
  10. Chapter 9 Maintaining the Link Board
    1.  9.1 Before Maintaining the Link Board
    2.  9.2 Configuration of the Link Board
    3. Click 9.3 Removing the Link Board
    4. Click 9.4 Installing the Link Board
  11. Chapter 10 Maintaining the Link Cards
    1.  10.1 Before Maintaining a Link Card
    2.  10.2 Configuration of a Link Card
    3.  10.3 Removing a Link Card
    4.  10.4 Installing a Link Card
  12. Chapter 11 Maintaining the Link Cables
    1.  11.1 Before Maintaining a Link Cable
    2.  11.2 Configuration of the Link Cable Connection Ports
    3.  11.3 Removing a Link Cable
    4.  11.4 Installing a Link Cable
  13. Chapter 12 Maintaining the Management Cable
    1.  12.1 Before Maintaining the Management Cable
    2.  12.2 Configuration of the Management Cable Connection Port
    3.  12.3 Removing the Management Cable
    4.  12.4 Installing the Management Cable
  14. Chapter 13 Maintaining the Power Supply Units
    1.  13.1 Before Maintaining a Power Supply Unit
    2.  13.2 Configuration of the Power Supply Units
    3.  13.3 Removing a Power Supply Unit
    4.  13.4 Installing a Power Supply Unit
  15. Chapter 14 Maintaining the Fan Units
    1.  14.1 Before Maintaining a Fan Unit
    2.  14.2 Configuration of the Fan Units
    3. Click 14.3 Removing a Fan Unit
    4. Click 14.4 Installing a Fan Unit
  16. Chapter 15 Maintaining the PCI Tray
    1.  15.1 Before Maintaining the PCI Tray
    2.  15.2 Configuration of the PCI Tray
    3. Click 15.3 Removing the PCI Tray
    4. Click 15.4 Installing the PCI Tray
  17. Chapter 16 Maintaining the I/O Board
    1.  16.1 Before Maintaining the I/O Board
    2.  16.2 Configuration of the I/O Board
    3. Click 16.3 Removing the I/O Board
    4. Click 16.4 Installing the I/O Board
  18. Chapter 17 Maintaining the Fan Backplane
    1.  17.1 Before Maintaining the Fan Backplane
    2.  17.2 Fan Backplane Location
    3. Click 17.3 Removing the Fan Backplane
    4. Click 17.4 Installing the Fan Backplane
  19. Appendix A Component List
  20. Appendix B Component Specifications
    1.  B.1 PCI Express Card
    2.  B.2 Link Board
    3.  B.3 Power Supply Unit
    4.  B.4 Fan Unit
    5.  B.5 PCI Tray
    6.  B.6 I/O Board
    7.  B.7 Fan Backplane
  21. Appendix C Problems During Firmware Update

5.5.1 Releasing the Link Card or PCIe Card Using PCI Hot Plug (PHP)


5.5.1 Releasing the Link Card or PCIe Card Using PCI Hot Plug (PHP)
This section describes the procedure for releasing the link card or PCIe card from the system using PCI hot plug (PHP).
1) Stopping the use of the I/O device
If there is any logical domain that is physically or virtually using the PCIe card mounted in the chassis requiring maintenance, stop the use of the I/O device to keep it unused.
Releasing the I/O Device in the Case of a Redundant Configuration
With the duplication function, release the target I/O device so that it will not be used. For details on the work, see the manual for the application being used.
Stop the Use of the I/O Device in the Case of a Non-Redundant Configuration
Place the I/O device in the unused state by deactivating the network, unmounting the disk, or other means. For details on the work, see the document of Oracle Solaris or the manual for the application being used.
2) Deleting the assignment of the I/O
For Cases Where Virtual I/O is Used
If the virtual I/O service is used, remove the virtual I/O device. For details, see the Oracle VM Server for SPARC Administration Guide released by Oracle.
The following command execution example removes a virtual disk (vdisk11) and a virtual network device (vnet10) that are using the virtual I/O service in the control domain or the root domain.
# ldm remove-vdisk vdisk11 guest0
# ldm remove-vnet vnet10 guest0
For Cases Where PCIe End Point Assignment is Used
If the PCIe end point device of the chassis requiring maintenance is assigned to an I/O domain, remove the PCIe end point device from the I/O domain and return it to the root domain (including the control domain). The work procedure depends on whether the dynamic reconfiguration function for PCIe end point devices is used. For details, see Table 5-2.
Table 5-2  Differences in the Procedure According to Whether the Dynamic Reconfiguration Function for PCIe End Point Devices is Used
 
-: Unnecessary
Item Task Command Dynamic Reconfiguration
is Used (*1)		 Dynamic Reconfiguration
is Not Used (*2)
1 Enabling the hotplug service svcadm enable hotplug Perform Perform (*3)
2 Stopping the I/O domain ldm stop-domain - Perform
3 Removing the PCIe end point device from the I/O domain ldm remove-io Perform Perform
4 Setting the root domain in delayed reconfiguration mode ldm start-reconf - Perform
5 Assigning the PCIe end point device to the root domain ldm add-io Perform Perform
6 Restarting the root domain shutdown -i6 -g0 -y - Perform
7 Disabling the hotplug service svcadm disable hotplug Perform Perform (*3)
*1 Dynamic reconfiguration for PCIe end point devices is supported by XCP 2230 or later and Oracle VM Server for SPARC 3.1.1.1 or later.
*2 If dynamic reconfiguration for PCIe end point devices is not used, stop the I/O domain, remove the physical I/O device, and then restore the physical I/O device to the root domain.
*3 If dynamic reconfiguration for PCIe end point devices is not used, the hotplug service operates whether it is enabled or disabled.
Note - Note the following when removing a physical I/O device (PCIe end point device) of the boot disk:
- If the boot disk is in a redundant configuration, stop the I/O domain for Oracle VM Server for SPARC 3.1.1 or earlier. (For Oracle VM Server for SPARC 3.1.1.1 or later, you do not need to stop the I/O domain.)

- If the boot disk is not in a redundant configuration, stop the I/O domain.
  1. If the hotplug service is disabled for the I/O and root domains, execute the svcadm enable hotplug command to enable the hotplug service.
# svcadm enable hotplug
  1. Stop the I/O domain.
    This step is unnecessary if the dynamic reconfiguration function for PCIe end point devices is used. Perform step 3.
# ldm stop-domain <‌I/O domain name>‌
  1. Remove the physical I/O device from the I/O domain.
# ldm remove-io <‌device name>‌ <‌I/O domain name>‌
  1. Transition the root domain to delayed reconfiguration mode.
    This step is unnecessary if the dynamic reconfiguration function for PCIe end point devices is used. Perform step 5.
# ldm start-reconf <‌root domain name>‌
  1. Reassign the physical I/O device removed in step 3 to the root domain.
# ldm add-io <‌device name>‌ <‌root domain name>‌
  1. Log in to the root domain, and restart Oracle Solaris.
    This step is unnecessary if the dynamic reconfiguration function for PCIe end point devices is used. Perform step 7.
# telnet localhost <‌port number>‌
...
# shutdown -i6 -g0 -y
  1. If you need to disable the hotplug service for the I/O and root domains, disable the hotplug service.
# svcadm disable hotplug
In Cases Where the Virtual Function of SR-IOV is Used
If the virtual function (VF) using the SR-IOV function has been created for a PCIe card mounted on the PCI expansion unit and it has been assigned to the domain, execute the ldm command from the control domain to remove and destroy the virtual function. The work procedure is different for the dynamic SR-IOV function or for static SR-IOV function. For details, see Table 5-3.
For details on the procedure, see the Oracle VM Server for SPARC Administration Guide from Oracle.
Table 5-3  Differences in the Releasing Procedure Between the Dynamic SR-IOV Function and Static SR-IOV Function
 
-: Unnecessary
Item Task Command Dynamic SR-IOV Function
is Used (*1)		 Static SR-IOV Function
is Used
1 Enabling the hotplug service svcadm enable hotplug Perform Perform (*2)
2 Stopping the I/O domain ldm stop-domain - Perform
3 Removing the VF from the I/O domain ldm remove-io Perform Perform
4 Setting the root domain of the PF in delayed reconfiguration mode ldm start-reconf - Perform
5 Destroying the VF ldm destroy-vf Perform Perform
6 Restarting the root domain shutdown -i6 -g0 -y - Perform
7 Disabling the hotplug service svcadm disable hotplug Perform Perform (*2)
*1 Dynamic reconfiguration for the virtual function of SR-IOV is supported by XCP 2210 or later and Oracle VM Server for SPARC 3.1 or later.
*2 If static reconfiguration for the virtual function of SR-IOV is used, the hotplug service operates whether it is enabled or disabled.
  1. If the hotplug service is disabled for the I/O and root domains, execute the svcadm enable hotplug command to enable the hotplug service.
# svcadm enable hotplug
  1. Stop the I/O domain.
    This step is unnecessary if the dynamic SR-IOV function is used. Perform step 3.
# ldm stop-domain <‌I/O domain name>‌
  1. Remove the virtual function (VF) assigned to the I/O domain, from the I/O domain.
    Execute the ldm remove-io command to remove the virtual function (VF) assigned to the I/O domain from the I/O domain. If multiple VFs are assigned from the physical function (PF) of a single PCIe card, remove all the VFs corresponding to that PF from the I/O domain.
# ldm remove-io <‌VF name>‌ <‌I/O domain name>‌
  1. Transition the root domain that has the assigned PF to delayed reconfiguration mode.
    This step is unnecessary if the dynamic SR-IOV function is used. Perform step 5.
# ldm start-reconf <‌root domain name>‌
  1. Destroy the virtual function (VF).
    Execute the ldm destroy-vf command to destroy the virtual function (VF). However, the VF needs to be destroyed in the reverse order from which it was created.
# ldm destroy-vf <‌VF name>‌
  1. Log in to the root domain, and restart Oracle Solaris.
    This step is unnecessary if the dynamic SR-IOV function is used. Perform step 7.
# telnet localhost <‌port number>‌
...
# shutdown -i6 -g0 -y
  1. If you need to disable the hotplug service for the I/O and root domains, disable the hotplug service.
# svcadm disable hotplug
3) Releasing the link card or PCIe card using PHP
The following describes the procedure for releasing the link card or PCIe card requiring maintenance from the system using PHP.
Releasing the link card also releases the PCI expansion unit requiring maintenance from the system.
  1. If the link card or PCIe card requiring maintenance belongs to the root domain, connect to the console of the root domain.
    If it belongs to the control domain, this step is not necessary since the control domain console is being used.
  1. Log in as an Oracle Solaris super-user.
  1. If the hotplug service has been disabled, execute the svcadm enable hotplug command to enable the hotplug service.
# svcadm enable hotplug
  1. Execute the cfgadm -a command to check the Ap_ID of the link card or PCIe card requiring maintenance and make a note of it.
# cfgadm -a
  1. Execute the cfgadm -c command to release the link card or PCIe card requiring maintenance from the logical domain.
    Enter the Ap_ID of the link card or PCIe card that you noted in advance.
# cfgadm -c unconfigure Ap_ID
  1. Execute the cfgadm -c command to stop the power supply to the link card or PCIe card requiring maintenance.
    Enter the Ap_ID of the link card or PCIe card that you noted in advance.
# cfgadm -c disconnect Ap_ID
  1. Execute the cfgadm -a command to confirm that the link card or PCIe card requiring maintenance can be removed.
# cfgadm -a
  1. Execute the cfgadm command to cause the locator LED of the link card or PCIe card to blink.
    Enter the Ap_ID of the link card or PCIe card that you noted in advance.
# cfgadm -x led=attn, mode=blink Ap_ID
  1. If you need to disable the hotplug service, disable the hotplug service.
# svcadm disable hotplug

Top of Page