Skip to main content

6.3.2 Example of Operations for Releasing a System Board


6.3.2 Example of Operations for Releasing a System Board
If the physical partition from which the system board (PSB) is released is in the operation state, use the deleteboard command with the -c disconnect or -c unassign option specified to release the PSB from the physical partition.

If the -c disconnect option is specified, the PSB is released from the physical partition but remains assigned to the physical partition. Therefore, restarting the physical partition or executing the addboard command incorporates the released PSB into the physical partition again.

If you release a PSB from a physical partition by specifying the -c unassign option, the PSB enters the system pool state and can be incorporated and assigned to another physical partition. For a PSB that has been released by specifying the -c disconnect option, executing the deleteboard command with the -c unassign option specified changes the state of the PSB from the assigned state to the system board pool state.
When releasing a PSB from a physical partition, use free resources in the physical partition to move the logical domains that use the resources of the PSB to be released. You can specify how to obtain the resources when the logical domain is moved according to the free resource status. To do this, specify the -m option for the deleteboard command.
- If the -m option is not specified or -m unbind=none is specified
If free resources cannot be obtained, or if the physical I/O assignment is not released in advance, the release with the deleteboard command fails.
- If -m unbind=resource is specified
With Oracle VM Server for SPARC 3.4 or later, you can select a policy for unbinding resources through the PPAR DR policy set in the ldmd/fj_ppar_dr_policy property of the ldmd service. With Oracle VM Server for SPARC earlier than Oracle VM Server for SPARC 3.4, this policy is always "targeted".
For details on the policy, see "8.15 Setting the Physical Partition Dynamic Reconfiguration Policy" in the Fujitsu SPARC M12 and Fujitsu M10/SPARC M10 System Operation and Administration Guide.
  1. ldmd/fj_ppar_dr_policy = auto
    This PPAR DR policy setting specifies to automatically use the latest policy. For a system that uses Oracle VM Server for SPARC 3.4, the "auto" setting works in the same way as the "ratio" setting. The "auto" setting is the default policy.
  1. ldmd/fj_ppar_dr_policy = ratio
    This policy is functional if all the logical domains in the physical partition run Oracle Solaris 11.3 or later and XCP 2271 or later is applied to the system. Otherwise, the system operates in the same way as when "targeted" is set.
    If the policy is set to "ratio" and the remaining PSBs do not have enough free resources to move the resources from the PSB to be released, the resources are automatically released by reducing the resources from the existing domains. In this case, resources are automatically reduced from all the existing domains proportionally as much as possible. Because resources are busy or due to other reasons, exact proportional reduction does not always work. The proportional reduction is on a best effort basis.
  1. ldmd/fj_ppar_dr_policy = targeted
    If free resources cannot be obtained, resources for the move are obtained from the resources deleted from each logical domain that uses CPU/memory on the destination PSB. A policy defines the order of selecting the logical domain from which to delete resources. This policy defines the following order: the default logical domain (with nothing specified), logical domain with the master specified, I/O domains, root domain, and control domain.
    If free resources cannot be obtained from the destination PSB, release with the deleteboard command fails.
- If -m unbind=shutdown is specified
The operation is the same as when -m unbind=resource is specified. If no more available resources can be obtained, the logical domain at the source shuts down. To use this option, ldmd/fj_ppar_dr_policy must be set to "targeted". Otherwise, the logical domain may not shut down.
Note - For a physical partition with the PPAR DR function disabled, you cannot release a PSB by specifying the -c disconnect or -c unassign option when the physical partition is in operation.
Note - When a PSB is released with the deleteboard command, the hardware resources on the PSB are released from Oracle Solaris. Therefore, it may take some time before the command finishes.
The operation example presented here uses XSCF shell commands and the ldm command of Logical Domains Manager, which is management software of Oracle VM Server for SPARC.
  1. Log in to the XSCF.
  2. Connect to the control domain console of the physical partition.
  3. Execute the ldm list-domain command to check the operation status of the logical domain.
  1. The following example shows that the control domain, two guest domains, and one root domain are operating.
# ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- UART 14 8G 64% 2h 54m
guest0 active -n---- 5000 16 8G 42% 2h 54m
guest1 active -n---- 5001 16 8G 11% 2h 54m
domain01 active -n---- 5002 16 8G 7.3% 2h 54m
  1. Execute the ldm list-devices command with the -a option specified to check the resource usage status.
  1. In the following example, the -a option is specified to display all resources bound to the logical domain and all resources that are not bound.
# ldm list-devices -a
CORE
ID %FREE CPUSET
0 0 (0, 1)
4 0 (8, 9)
8 0 (16, 17)
(Omitted)
184 0 (368, 369)
188 100 (376, 377)
512 100 (1024, 1025)
516 100 (1032, 1033)
520 100 (1040, 1041)
524 100 (1048, 1049)
(Omitted)
VCPU
PID %FREE PM
0 0 no
1 0 no
8 0 no
9 0 no
(Omitted)
369 0 no
376 100 ---
377 100 ---
1024 100 ---
1025 100 ---
1032 100 ---
1033 100 ---
(Omitted)
  1. Execute the ldm list-io command to check the usage status of the I/O devices.
    In the following example, primary, which is a control domain, and domain01, which is a root domain, use I/O devices.
# ldm list-io
NAME TYPE BUS DOMAIN STATUS
---- ---- --- ------ ------

PCIE0 BUS PCIE0 primary IOV
PCIE1 BUS PCIE1 primary IOV
PCIE2 BUS PCIE2 primary IOV
PCIE3 BUS PCIE3 primary IOV
PCIE8 BUS PCIE8 domain01 IOV
PCIE9 BUS PCIE9 domain01 IOV
PCIE10 BUS PCIE10 domain01 IOV
PCIE11 BUS PCIE11 domain01 IOV
/BB0/CMUL/NET0 PCIE PCIE0 primary OCC
/BB0/CMUL/NET2 PCIE PCIE0 primary OCC
/BB0/CMUL/SASHBA PCIE PCIE0 primary OCC
/BB0/PCI0 PCIE PCIE1 primary EMP
/BB0/PCI1 PCIE PCIE1 primary EMP
(Omitted)
  1. Among the services provided by the root domain, release all I/O devices on the PSB to be deleted.
  1. In the following example, domain01, which is a root domain, is stopped, and of the PCIe root complexes owned by domain01, those related to the PSB to be deleted are released.
Note - To reconfigure dynamically, the ldm stop-domain command to stop the root domain is not required. Start operation from the I/O device release.
# ldm stop-domain domain01
# ldm remove-io PCIE11 domain01
# ldm remove-io PCIE10 domain01
# ldm remove-io PCIE9 domain01
# ldm remove-io PCIE8 domain01
# ldm unbind-domain domain01
  1. Execute the ldm list-io command again to check the usage status of the I/O devices.
# ldm list-io
NAME TYPE BUS DOMAIN STATUS
---- ---- --- ------ ------

PCIE0 BUS PCIE0 primary IOV
PCIE1 BUS PCIE1 primary IOV
PCIE2 BUS PCIE2 primary IOV
PCIE3 BUS PCIE3 primary IOV
PCIE8 BUS PCIE8
PCIE9 BUS PCIE9
PCIE10 BUS PCIE10
PCIE11 BUS PCIE11
/BB0/CMUL/NET0 PCIE PCIE0 primary OCC
/BB0/CMUL/NET2 PCIE PCIE0 primary OCC
/BB0/CMUL/SASHBA PCIE PCIE0 primary OCC
/BB0/PCI0 PCIE PCIE1 primary EMP
/BB0/PCI1 PCIE PCIE1 primary EMP
(Omitted)
  1. Execute the deleteboard command with the -c unassign option specified to release the PSB from the physical partition.
  1. In the following example, PSB 01-0 is released from the physical partition, and is placed in the system board pool state.
XSCF> deleteboard -c unassign 01-0
PSB#01-0 will be unassigned from PPAR immediately. Continue?[y|n] :y
Start unconfigure preparation of PSB. [1200sec]
0end
Unconfigure preparation of PSB has completed.
Start unconfiguring PSB from PPAR. [43200sec]
0..... 30.end
Unconfigured PSB from PPAR.
PSB power off sequence started. [1200sec]
0..... 30..... 60..... 90.....120.....150.end
Operation has completed.
  1. Execute the showresult command to check the exit status of the deleteboard command just executed.
  1. In the following example, exit status of 0 is returned, indicating normal termination of the deleteboard command.
XSCF> showresult
0
  1. Execute the showboards command to check the PSB status.
  1. The following example shows that PSB 01-0 is in the system board pool state.
XSCF> showboards -p 0
PSB PPAR-ID(LSB) Assignment Pwr Conn Conf Test Fault
---- ------------ ----------- ---- ---- ---- ------- --------

00-0 00(00) Assigned y y y Passed Normal
01-0 SP Available n n n Passed Normal
  1. Execute the ldm list-domain command on the control domain console of the physical partition to confirm that the operation status of the logical domain has not changed after the deletion of the PSB.
  2. When you remove a building block, execute the initbb command on the master XSCF to release the target SPARC M12-2S/M10-4S from the system.
Note - For details of system reduction, see "Chapter 9 Reducing a System With a Building Block Configuration" in the SPARC M12-2S Installation Guide or "Chapter 10 Removing a System with a Building Block Configuration" in the SPARC M10-4S Installation Guide.