Skip to main content
  1. Home >
  2. Products >
  3. Computing Products >
  4. Servers >
  5. Fujitsu SPARC servers >
  6. Downloads >
  7. User Manuals >
  8. PCI Expansion Unit for Fujitsu SPARC M12 and Fujitsu M10/SPARC M10 Service Manual >
  9. 5.5.2 Releasing the Link Card or PCIe Card Using Physical Partition Dynamic Reconfiguration (DR)

Contents

  1. Preface
  2. Chapter 1 Before Starting Maintenance Work
    1.  1.1 Warning/Caution Indications
    2.  1.2 Labels/Tags
    3.  1.3 Safety Precautions
    4.  1.4 Notes Regarding Static Electricity
    5.  1.5 Other Precautions
    6.  1.6 Emergency Power Off
    7. Click 1.7 Important Information About the XCP Firmware
  3. Chapter 2 Understanding the PCI Expansion Unit Components
    1.  2.1 Identifying the Names and Locations of Components
    2. Click 2.2 Checking the Status With the LEDs
  4. Chapter 3 Types of Maintenance
    1.  3.1 Maintenance Types of the PCI Expansion Unit
    2.  3.2 Types of Maintenance for the Link Card
    3. Click 3.3 Types of Maintenance for the PCIe Card
  5. Chapter 4 Preparation and Precautions for Maintenance
    1. Click 4.1 Troubleshooting
    2. Click 4.2 Precautions for Maintenance
  6. Chapter 5 Preparations Required for Maintenance
    1.  5.1 Preparing Tools Required for Maintenance
    2.  5.2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition
    3.  5.3 Checking the Firmware Version of the PCI Expansion Unit
    4. Click 5.4 Checking the Operating Condition and Resource Usage Status
    5. Click 5.5 Releasing the Link Card or PCIe Card From the System
    6.  5.6 Stopping the FRU Requiring Maintenance
    7. Click 5.7 Accessing a FRU
  7. Chapter 6 Understanding the Preparations for Restoring the System
    1. Click 6.1 Restoring the Chassis of the PCI Expansion Unit
    2.  6.2 Supplying Power to the FRU Requiring Maintenance
    3. Click 6.3 Mounting a Link Card or PCIe Card on the System
    4. Click 6.4 Restoring the Logical Domain to the Status It Had Before Maintenance
    5.  6.5 Turning Off the CHECK LED (Locator) of the PCI Expansion Unit
    6. Click 6.6 Updating the Firmware of the PCI Expansion Unit
  8. Chapter 7 Maintenance Flow
    1. Click 7.1 FRU Replacement Workflow for the PCI Expansion Unit
    2. Click 7.2 FRU Addition Workflow for the PCI Expansion Unit
    3. Click 7.3 FRU Removal Workflow for the PCI Expansion Unit
    4. Click 7.4 Link Card Maintenance Workflow
  9. Chapter 8 Maintaining the PCI Express Cards
    1.  8.1 Before Maintaining a PCI Express Card
    2.  8.2 Configuration of the PCIe Cards
    3. Click 8.3 Removing a PCI Express Card
    4. Click 8.4 Installing a PCI Express Card
  10. Chapter 9 Maintaining the Link Board
    1.  9.1 Before Maintaining the Link Board
    2.  9.2 Configuration of the Link Board
    3. Click 9.3 Removing the Link Board
    4. Click 9.4 Installing the Link Board
  11. Chapter 10 Maintaining the Link Cards
    1.  10.1 Before Maintaining a Link Card
    2.  10.2 Configuration of a Link Card
    3.  10.3 Removing a Link Card
    4.  10.4 Installing a Link Card
  12. Chapter 11 Maintaining the Link Cables
    1.  11.1 Before Maintaining a Link Cable
    2.  11.2 Configuration of the Link Cable Connection Ports
    3.  11.3 Removing a Link Cable
    4.  11.4 Installing a Link Cable
  13. Chapter 12 Maintaining the Management Cable
    1.  12.1 Before Maintaining the Management Cable
    2.  12.2 Configuration of the Management Cable Connection Port
    3.  12.3 Removing the Management Cable
    4.  12.4 Installing the Management Cable
  14. Chapter 13 Maintaining the Power Supply Units
    1.  13.1 Before Maintaining a Power Supply Unit
    2.  13.2 Configuration of the Power Supply Units
    3.  13.3 Removing a Power Supply Unit
    4.  13.4 Installing a Power Supply Unit
  15. Chapter 14 Maintaining the Fan Units
    1.  14.1 Before Maintaining a Fan Unit
    2.  14.2 Configuration of the Fan Units
    3. Click 14.3 Removing a Fan Unit
    4. Click 14.4 Installing a Fan Unit
  16. Chapter 15 Maintaining the PCI Tray
    1.  15.1 Before Maintaining the PCI Tray
    2.  15.2 Configuration of the PCI Tray
    3. Click 15.3 Removing the PCI Tray
    4. Click 15.4 Installing the PCI Tray
  17. Chapter 16 Maintaining the I/O Board
    1.  16.1 Before Maintaining the I/O Board
    2.  16.2 Configuration of the I/O Board
    3. Click 16.3 Removing the I/O Board
    4. Click 16.4 Installing the I/O Board
  18. Chapter 17 Maintaining the Fan Backplane
    1.  17.1 Before Maintaining the Fan Backplane
    2.  17.2 Fan Backplane Location
    3. Click 17.3 Removing the Fan Backplane
    4. Click 17.4 Installing the Fan Backplane
  19. Appendix A Component List
  20. Appendix B Component Specifications
    1.  B.1 PCI Express Card
    2.  B.2 Link Board
    3.  B.3 Power Supply Unit
    4.  B.4 Fan Unit
    5.  B.5 PCI Tray
    6.  B.6 I/O Board
    7.  B.7 Fan Backplane
  21. Appendix C Problems During Firmware Update

5.5.2 Releasing the Link Card or PCIe Card Using Physical Partition Dynamic Reconfiguration (DR)


5.5.2 Releasing the Link Card or PCIe Card Using Physical Partition Dynamic Reconfiguration (DR)
This section describes the procedure for releasing the physical partition containing the link card from the system using dynamic reconfiguration (DR). The procedure can be performed only for a system with a building block configuration of the SPARC M12-2S/M10-4S.
Note - To use the physical partition dynamic reconfiguration (PPAR DR) function, you need to configure a logical domain beforehand according to "2.5 Dynamic Reconfiguration Conditions and Settings" in the Fujitsu SPARC M12 and Fujitsu M10/SPARC M10 Domain Configuration Guide.
If the PCIe end point of the PCIe card in the chassis requiring maintenance is assigned to an I/O domain or if the SR-IOV virtual function is assigned, return the PCIe end point to the root or control domain. Then, remove the root complex from the root or control domain.
1) Stopping the use of the I/O device
If there is any logical domain that is physically or virtually using the PCIe card mounted in the chassis requiring maintenance, stop the use of the I/O device to keep it unused.
Releasing the I/O Device in the Case of a Redundant Configuration
With the duplication function, release the target I/O device so that it will not be used. For details on the work, see the manual for the application being used.
Stop the Use of the I/O Device in the Case of a Non-Redundant Configuration
Place the I/O device in the unused state by deactivating the network, unmounting the disk, or other means. For details on the work, see the document of Oracle Solaris or the manual for the application being used.
2) Deleting the assignment of the I/O
For Cases Where Virtual I/O is Used
If the virtual I/O service is used, remove the virtual I/O device. For details, see the Oracle VM Server for SPARC Administration Guide released by Oracle.
The following command execution example removes a virtual disk (vdisk11) and a virtual network device (vnet10) that are using the virtual I/O service in the control domain or the root domain.
# ldm remove-vdisk vdisk11 guest0
# ldm remove-vnet vnet10 guest0
For Cases Where PCIe End Point Assignment is Used
If the PCIe end point device of the chassis requiring maintenance is assigned to an I/O domain, remove the PCIe end point device from the I/O domain and return it to the root domain (including the control domain). The work procedure depends on whether the dynamic reconfiguration function for PCIe end point devices is used. For details, see Table 5-4.
Table 5-4  Differences in the Procedure According to Whether the Dynamic Reconfiguration Function for PCIe End Point Devices is Used
 
-: Unnecessary
Item Task Command Dynamic Reconfiguration
is Used (*1)		 Dynamic Reconfiguration
is Not Used (*2)
1 Enabling the hotplug service svcadm enable hotplug Perform Perform (*3)
2 Stopping the I/O domain ldm stop-domain - Perform
3 Removing the PCIe end point device from the I/O domain ldm remove-io Perform Perform
4 Setting the root domain in delayed reconfiguration mode ldm start-reconf - Perform
5 Assigning the PCIe end point device to the root domain ldm add-io Perform Perform
6 Restarting the root domain shutdown -i6 -g0 -y - Perform
7 Disabling the hotplug service svcadm disable hotplug Perform Perform (*3)
*1 Dynamic reconfiguration for PCIe end point devices is supported by XCP 2230 or later and Oracle VM Server for SPARC 3.1.1.1 or later.
*2 If dynamic reconfiguration for PCIe end point devices is not used, stop the I/O domain, remove the physical I/O device, and then restore the physical I/O device to the root domain.
*3 If dynamic reconfiguration for PCIe end point devices is not used, the hotplug service operates whether it is enabled or disabled.
Note - Note the following when removing a physical I/O device (PCIe end point device) of the boot disk:
- If the boot disk is in a redundant configuration, stop the I/O domain for Oracle VM Server for SPARC 3.1.1 or earlier. (For Oracle VM Server for SPARC 3.1.1.1 or later, you do not need to stop the I/O domain.)

- If the boot disk is not in a redundant configuration, stop the I/O domain.
  1. If the hotplug service is disabled for the I/O and root domains, execute the svcadm enable hotplug command to enable the hotplug service.
# svcadm enable hotplug
  1. Stop the I/O domain.
    This step is unnecessary if the dynamic reconfiguration function for PCIe end point devices is used. Perform step 3.
# ldm stop-domain <‌I/O domain name>‌
  1. Remove the physical I/O device from the I/O domain.
# ldm remove-io <‌device name>‌ <‌I/O domain name>‌
  1. Transition the root domain to delayed reconfiguration mode.
    This step is unnecessary if the dynamic reconfiguration function for PCIe end point devices is used. Perform step 5.
# ldm start-reconf <‌root domain name>‌
  1. Reassign the physical I/O device removed in step 3 to the root domain.
# ldm add-io <‌device name>‌ <‌root domain name>‌
  1. Log in to the root domain, and restart Oracle Solaris.
    This step is unnecessary if the dynamic reconfiguration function for PCIe end point devices is used. Perform step 7.
# telnet localhost <‌port number>‌
...
# shutdown -i6 -g0 -y
  1. If you need to disable the hotplug service for the I/O and root domains, disable the hotplug service.
# svcadm disable hotplug
In Cases Where the Virtual Function of SR-IOV is Used
If the virtual function (VF) using the SR-IOV function has been created for a PCIe card mounted on the PCI expansion unit and it has been assigned to the domain, execute the ldm command from the control domain to remove and destroy the virtual function. The work procedure is different for the dynamic SR-IOV function or for static SR-IOV function. For details, see Table 5-5.
For details on the procedure, see the Oracle VM Server for SPARC Administration Guide from Oracle.
Table 5-5  Differences in the Releasing Procedure Between the Dynamic SR-IOV Function and Static SR-IOV Function
 
-: Unnecessary
Item Task Command Dynamic SR-IOV Function
is Used (*1)		 Static SR-IOV Function
is Used
1 Enabling the hotplug service svcadm enable hotplug Perform Perform (*2)
2 Stopping the I/O domain ldm stop-domain - Perform
3 Removing the VF from the I/O domain ldm remove-io Perform Perform
4 Setting the root domain of the PF in delayed reconfiguration mode ldm start-reconf - Perform
5 Destroying the VF ldm destroy-vf Perform Perform
6 Restarting the root domain shutdown -i6 -g0 -y - Perform
7 Disabling the hotplug service svcadm disable hotplug Perform Perform (*2)
*1 Dynamic reconfiguration for the virtual function of SR-IOV is supported by XCP 2210 or later and Oracle VM Server for SPARC 3.1 or later.
*2 If static reconfiguration for the virtual function of SR-IOV is used, the hotplug service operates whether it is enabled or disabled.
  1. If the hotplug service is disabled for the I/O and root domains, execute the svcadm enable hotplug command to enable the hotplug service.
# svcadm enable hotplug
  1. Stop the I/O domain.
    This step is unnecessary if the dynamic SR-IOV function is used. Perform step 3.
# ldm stop-domain <‌I/O domain name>‌
  1. Remove the virtual function (VF) assigned to the I/O domain, from the I/O domain.
    Execute the ldm remove-io command to remove the virtual function (VF) assigned to the I/O domain from the I/O domain. If multiple VFs are assigned from the physical function (PF) of a single PCIe card, remove all the VFs corresponding to that PF from the I/O domain.
# ldm remove-io <‌VF name>‌ <‌I/O domain name>‌
  1. Transition the root domain that has the assigned PF to delayed reconfiguration mode.
    This step is unnecessary if the dynamic SR-IOV function is used. Perform step 5.
# ldm start-reconf <‌root domain name>‌
  1. Destroy the virtual function (VF).
    Execute the ldm destroy-vf command to destroy the virtual function (VF). However, the VF needs to be destroyed in the reverse order from which it was created.
# ldm destroy-vf <‌VF name>‌
  1. Log in to the root domain, and restart Oracle Solaris.
    This step is unnecessary if the dynamic SR-IOV function is used. Perform step 7.
# telnet localhost <‌port number>‌
...
# shutdown -i6 -g0 -y
  1. If you need to disable the hotplug service for the I/O and root domains, disable the hotplug service.
# svcadm disable hotplug
3) Removing the root complex
Remove the Root Complex From the Root Domain
If the I/O device of the chassis requiring maintenance is assigned to the root domain, remove the physical I/O device (root complex) from the root domain. The work procedure depends on whether the dynamic reconfiguration function for root complexes is used. For details, see "Table 5-6."
Table 5-6  Differences in the Procedure According to Whether the Dynamic Reconfiguration Function for Root Complexes is Used
 
-: Unnecessary
Item Task Command Dynamic Reconfiguration
is Used (*1)		 Dynamic Reconfiguration
is Not Used (*2)
1 Stopping the root domain ldm stop-domain - Perform
2 Removing the physical I/O device (root complex) from the root domain ldm remove-io Perform Perform
*1 Dynamic reconfiguration for root complexes is supported by XCP 2240 or later and Oracle VM Server for SPARC 3.2 or later. In addition, Oracle Solaris 11.2 SRU 11.2.8 or later must be used for the control domain.
*2 If dynamic reconfiguration for root complexes is not used, stop the root domain, and then remove the physical I/O device.
  1. Stop the root domain.
    This step is unnecessary if root complexes are dynamically reconfigured. Perform step 2.
# ldm stop-domain <‌root domain name>‌
  1. Remove the physical I/O device (root complex) from the root domain.
# ldm remove-io <physical I/O device name> <root domain name>
  1. Confirm that the I/O device of the chassis to be released is unused and unassigned.
# ldm list-io
Remove the Root Complex From the Control Domain
If the I/O device of the chassis requiring maintenance is assigned to the control domain, remove the physical I/O device (root complex) from the control domain. The work procedure depends on whether the dynamic reconfiguration function for root complexes is used. For details, see "Table 5-7."
Table 5-7  Differences in the Procedure According to Whether the Dynamic Reconfiguration Function for Root Complexes is Used
 
-: Unnecessary
Item Task Command Dynamic Reconfiguration
is Used (*1)		 Dynamic Reconfiguration
is Not Used (*2)
1 Setting the control domain in delayed reconfiguration mode ldm start-reconf - Perform
2 Removing the physical I/O device from the control domain ldm remove-io Perform Perform
3 Restarting the control domain shutdown -i6 -g0 -y - Perform
*1 Dynamic reconfiguration for root complexes is supported by XCP 2240 or later and Oracle VM Server for SPARC 3.2 or later. In addition, Oracle Solaris 11.2 SRU 11.2.8 or later must be used for the control domain.
*2 If dynamic reconfiguration for root complexes is not used, set the control domain in delayed reconfiguration mode, and then remove the physical I/O device.
  1. Transition the control domain to delayed reconfiguration mode.
    This step is unnecessary if root complexes are dynamically reconfigured. Perform step 2.
# ldm start-reconf <‌control domain name>‌
  1. Remove the physical I/O device from the control domain.
# ldm remove-io <‌physical I/O device name>‌ <‌control domain name>‌
  1. Restart the control domain.
    This step is unnecessary if the dynamic reconfiguration function for root complexes is used.
# shutdown -i6 -g0 -y
Note - When set to delayed reconfiguration mode, the control domain does not have control over the guest domains. If you have set the control domain to delayed reconfiguration mode, restart the control domain as soon as possible.
4) Releasing the link card using dynamic reconfiguration (DR)
For details on the procedure for dynamic reconfiguration (DR), see "9.4.3 Dynamically Releasing the SPARC M12-2S From the Physical Partition" in the Fujitsu SPARC M12-2/M12-2S Service Manual or "5.4.3 Releasing a chassis requiring maintenance from the physical partition" in the Fujitsu M10-4/Fujitsu M10-4S/SPARC M10-4/SPARC M10-4S Service Manual.

Top of Page