A.3.3 Example of the Active Replacement Procedure (for Oracle VM Server for SPARC 3.1.x)
A.3.3 Example of the Active Replacement Procedure (for Oracle VM Server for SPARC 3.1.x)
This section describes an example of the procedure for actively replacing the BB#01 SPARC M10-4S using PPAR DR for the 2BB configuration system described in Figure A-3 Example of the 2BB Configuration for Operational Continuity (With Free Resources).
| Note - If the XSCF in SPARC M10-4S to be actively replaced is defective, you cannot perform active replacement using PPAR DR. You must stop the physical partition to which the SPARC M10-4S to be actively replaced belongs and then perform maintenance with the input power to the SPARC M10-4S to be replaced turned off. |
- Log in to the master XSCF.
Execute the showbbstatus command to check that the XSCF to which you have logged in is the master XSCF.
If you have logged in to a standby XSCF, log out and then log in to the master XSCF again.
| XSCF> showbbstatus BB#00 (Master) |
- Execute the showhardconf command to check that [Status] of the XSCF in the SPARC M10-4S to be replaced is "Normal."
| XSCF> showhardconf SPARC M10-4S; + Serial: 2081230011; Operator_Panel_Switch:Locked; + System_Power:On; System_Phase:Cabinet Power On; Partition#0 PPAR_Status:Running; BB#00 Status:Normal; Role:Master; Ver:2003h; Serial:2081231002; + FRU-Part-Number: CA07361-D202 A1 ; + Power_Supply_System: ; + Memory_Size:256 GB; CMUL Status:Normal; Ver:0101h; Serial:PP1236052K ; + FRU-Part-Number:CA07361-D941 C4 /7060911 ; + Memory_Size:128 GB; Type: A ; CPU#0 Status:Normal; Ver:4142h; Serial:00322658; + Freq:3.000 GHz; Type:0x10; + Core:16; Strand:2; ： BB#01 Status:Normal; Role:Standby; Ver:0101h;Serial:7867000297; + FRU-Part-Number: CA20393-B50X A2 ; + Power_Supply_System: ; + Memory_Size:256 GB; CMUL Status:Normal; Ver:0101h; Serial:PP123406CB ; + FRU-Part-Number:CA07361-D941 C4 /7060911 ; + Memory_Size:128 GB; Type: A ; ： |
| Note - If the XSCF in SPARC M10-4S to be actively replaced is defective, you cannot perform active replacement using PPAR DR. You must stop the physical partition to which the SPARC M10-4S to be actively replaced belongs and then perform maintenance with the input power to the SPARC M10-4S to be replaced turned off. |
- Execute the showbbstatus command to confirm that the XSCF in the SPARC M10-4S to be replaced is not the master XSCF.
| XSCF> showbbstatus BB#00 (Master) |
- If the SPARC M10-4S to be replaced is the master XSCF, execute the switchscf command to switch the XSCF.
| XSCF> switchscf -t Standby The XSCF unit switch between the Active and Standby states. Continue? [y|n] :y |
| Note - Confirm that the XSCF has been switched and rebooted before you release the SPARC M10-4S. |
- Execute the console command to connect to the console of the control domain and then log in to it.
| XSCF> console -p 0 |
- Check the operation status and resource usage status of the logical domain.
- a. Execute the ldm list-domain command to check the operation status of the logical domain.To check the logical domain operation status, check the [STATE] and [FLAGS] combination. If [STATE] indicates "active", the second character from the left of the string in [FLAGS] has the following meaning."n": Oracle Solaris is operating"t": OpenBoot PROM status"-": In another state (including [STATE] other than "active")The following example shows that the control domain, two root domains, and two guest domains are operating.
- Check whether all domains are in "active", which indicates that Oracle Solaris is in operating state, or "inactive" state. If there is a domain with OpenBoot PROM status or bound status, the dynamic reconfiguration of the physical partition may fail.
| # ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- UART 16 14G 0.0% 10h 7m guest0 active -n---- 5100 32 32G 0.0% 2h 20s guest1 active -n---- 5101 32 32G 0.0% 2h 5s root-dom0 active -n--v- 5000 24 24G 0.0% 2h 43s root-dom1 active -n--v- 5001 24 24G 0.0% 2h 20s |
- b. Execute the ldm list-devices command with the -a option specified to check the resource usage status.In the following example, the -a option is specified to display all resources bound to the logical domain and all resources that are not bound.
| # ldm list-devices -a CORE ID %FREE CPUSET 0 0 (0, 1) 4 0 (8, 9) 8 0 (16, 17) (Omitted) 944 0 (1888, 1889) 948 0 (1896, 1897) 952 0 (1904, 1905) 956 0 (1912, 1913) VCPU PID %FREE PM 0 0 no 1 0 no 8 0 no 9 0 no (Omitted) 1904 0 no 1905 0 no 1912 0 no 1913 0 no (Omitted) |
- Release the redundant configuration of the system volume and I/O devices in the control domain.
This step describes how to release the I/O devices of the SPARC M10-4S to be replaced and which are used in the control domain in order enable the release of BB-lD#01 SPARC M10-4S, If you are using other redundant configuration software, see the documentation for the software for that redundant configuration for details on how to cancel the configuration.
- a. Cancel the redundant configuration of the system volume in the control domain.The following example describes how to cancel the ZFS mirroring function for the system volume in the control domain.Execute the zpool status command in the control domain to check the mirroring configuration status.
| # zpool status rpool pool: rpool state: ONLINE scan: resilvered 28.7M in 0h0m with 0 errors on Tue Jan 21 10:10:01 2014 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c2t50000393E802CCE2d0s0 ONLINE 0 0 0 c3t50000393A803B13Ed0s0 ONLINE 0 0 0 errors: No known data errors |
- Execute the zpool detach command to release the disk from the mirroring configuration.
| # zpool detach rpool c3t50000393A803B13Ed0 |
- Execute the zpool status command to confirm that the mirroring configuration has been canceled.
| # zpool status rpool pool: rpool state: ONLINE scan: resilvered 28.7M in 0h0m with 0 errors on Tue Jan 21 10:10:01 2014 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c2t50000393E802CCE2d0s0 ONLINE 0 0 0 errors: No known data errors |
- If you are using other devices in BB#01, remove the redundant configuration or stop using those devices. For details on how to cancel a redundant configuration or stop using the devices, see the documentation for the software for that redundant configuration and Oracle Solaris.
- b. Delete the I/O configuration of the control domain.From among the physical I/O devices assigned to the control domain, delete the root complexes in BB#01 through delayed reconfiguration.First, place the control domain in delayed reconfiguration mode.
| # ldm start-reconf primary Initiating a delayed reconfiguration operation on the primary domain. All configuration changes for other domains are disabled until the primary domain reboots, at which time the new configuration for the primary domain will also take effect. |
- Execute the ldm list-io command to check the root complexes assigned to the primary.The following example shows that the root complexes with BB1 devices are PCIE8 and PCIE12.
| # ldm list-io | grep primary PCIE0 BUS PCIE0 primary IOV PCIE4 BUS PCIE4 primary IOV PCIE8 BUS PCIE8 primary IOV PCIE12 BUS PCIE12 primary IOV /BB0/CMUL/NET0 PCIE PCIE0 primary OCC /BB0/CMUL/SASHBA PCIE PCIE0 primary OCC /BB0/CMUL/NET2 PCIE PCIE4 primary OCC /BB1/CMUL/NET0 PCIE PCIE8 primary OCC /BB1/CMUL/SASHBA PCIE PCIE8 primary OCC /BB1/CMUL/NET2 PCIE PCIE12 primary OCC |
- Execute the ldm remove-io command to delete PCIE8 and PCIE12 from the primary, and then restart Oracle Solaris.
| # ldm remove-io PCIE8 primary ------------------------------------------------------------------------------ Notice: The primary domain is in the process of a delayed reconfiguration. Any changes made to the primary domain will only take effect after it reboots. ------------------------------------------------------------------------------ # ldm remove-io PCIE12 primary ------------------------------------------------------------------------------ Notice: The primary domain is in the process of a delayed reconfiguration. Any changes made to the primary domain will only take effect after it reboots. ------------------------------------------------------------------------------ # shutdown -i6 -g0 -y .... |
- Once Oracle Solaris has started, execute the ldm list-io command to confirm that the root complexes in BB#01 have been deleted from the control domain.
| # ldm list-io | grep primary PCIE0 BUS PCIE0 primary IOV PCIE4 BUS PCIE4 primary IOV /BB0/CMUL/NET0 PCIE PCIE0 primary OCC /BB0/CMUL/SASHBA PCIE PCIE0 primary OCC /BB0/CMUL/NET2 PCIE PCIE4 primary OCC |
- c. Cancel the redundant configuration of the virtual I/O devices assigned to a guest domain.To first shut down the root domain (root-dom1) to which the root complexes in BB#01 are assigned, log in to each guest domain, and then cancel the redundant configuration of the virtual I/O device from root-dom1.For details on how to use the redundant configuration software, see the documentation about the software for that redundant configuration.
- In the following example, a virtual network device (vnet1) is canceled from the IPMP configuration. For details on the command, see the manual for Oracle Solaris.Log in to the guest domain (guest0).
| # ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- UART 64 56G 0.0% 4h 17m guest0 active -n---- 5100 64 64G 0.0% 1h 13m guest1 active -n---- 5101 64 64G 0.0% 1h 4m root-dom0 active -n--v- 5000 32 32G 0.0% 1h 47m root-dom1 active -n--v- 5001 32 32G 0.0% 1h 19m # telnet localhost 5100 .... guest0# |
- Execute the dladm show-phys command to check the mapping between the virtual network interface (vnet1) and the network interface name (net1).
| guest0# dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE net0 Ethernet up 0 unknown vnet0 net1 Ethernet up 0 unknown vnet1 |
- Execute the ipmpstat -i command to check the configuration information for the network interfaces configuring IPMP.
| guest0# ipmpstat -i INTERFACE ACTIVE GROUP FLAGS LINK PROBE STATE net0 yes ipmp0 -smbM-- up disabled ok net1 no ipmp0 is----- up disabled ok |
- Execute the if_mpadm -d command to release net1 from the IPMP group, and then execute the ipmpstat -i command to confirm that it has been released. The following example confirms that STATE is offline.
| guest0# if_mpadm -d net1 guest0# ipmpstat -i INTERFACE ACTIVE GROUP FLAGS LINK PROBE STATE net0 yes ipmp0 -smbM-- up disabled ok net1 no ipmp0 -s---d- up disabled offline |
- Execute the ipadm delete-ip command to delete net1.
| guest0# ipadm delete-ip net1 |
- Similarly, perform the same release processing for the guest domain (guest1).
- d. Remove the virtual I/O devices assigned from the root domain to be stopped.Execute the ldm remove-vdisk and ldm remove-vnet commands to remove the assigned virtual disk (vdisk) and virtual network device (vnet) from the root domain to be stopped according to the following step.The following example shows the execution of the commands for removing the virtual disk (vdisk10) and virtual network device (vnet10) that use the virtual I/O service of the BB#01 root domain (root-dom1).Perform the same deletion for the guest domain (guest1).
| # ldm remove-vdisk vdisk10 guest0 # ldm remove-vnet vnet10 guest0 |
- Check the resource usage status of the I/O devices, and then cancel all the I/O devices in the SPARC M10-4S to be replaced.
- a. Check the logical domain to which the root complexes in the SPARC M10-4S to be released are assigned.Execute the ldm list-io command to check the logical domain to which the root complexes in BB#01 are assigned.The following example shows that only root-dom1 has PCIe endpoints starting with "/BB1/." You can see that the PCIe endpoint root complexes (BUS) PCIE9, PCIE10, PCIE11, PCIE13, PCIE14, and PCIE15 are assigned to root-dom1.
| # ldm list-io NAME TYPE BUS DOMAIN STATUS ---- ---- --- ------ ------ PCIE0 BUS PCIE0 primary IOV PCIE1 BUS PCIE1 root-dom0 IOV PCIE2 BUS PCIE2 root-dom0 IOV PCIE3 BUS PCIE3 root-dom0 IOV PCIE4 BUS PCIE4 primary IOV PCIE5 BUS PCIE5 root-dom0 IOV PCIE6 BUS PCIE6 root-dom0 IOV PCIE7 BUS PCIE7 root-dom0 IOV PCIE8 BUS PCIE8 PCIE9 BUS PCIE9 root-dom1 IOV PCIE10 BUS PCIE10 root-dom1 IOV PCIE11 BUS PCIE11 root-dom1 IOV PCIE12 BUS PCIE12 PCIE13 BUS PCIE13 root-dom1 IOV PCIE14 BUS PCIE14 root-dom1 IOV PCIE15 BUS PCIE15 root-dom1 IOV .... /BB1/CMUL/NET0 PCIE PCIE8 UNK /BB1/CMUL/SASHBA PCIE PCIE8 UNK /BB1/PCI0 PCIE PCIE9 root-dom1OCC /BB1/PCI3 PCIE PCIE10 root-dom1OCC /BB1/PCI4 PCIE PCIE10 root-dom1OCC /BB1/PCI7 PCIE PCIE11 root-dom1OCC /BB1/PCI8 PCIE PCIE11 root-dom1OCC /BB1/CMUL/NET2 PCIE PCIE12 UNK /BB1/PCI1 PCIE PCIE13 root-dom1OCC /BB1/PCI2 PCIE PCIE13 root-dom1OCC /BB1/PCI5 PCIE PCIE14 root-dom1OCC /BB1/PCI6 PCIE PCIE14 root-dom1OCC /BB1/PCI9 PCIE PCIE15 root-dom1OCC /BB1/PCI10 PCIE PCIE15 root-dom1OCC |
- b. Stop the root domain to which the root complexes in SPARC M10-4S to be released are assigned and then release the SPARC M10-4S.The following example executes the ldm stop-domain and ldm unbind-domain commands to release the root domain (root-dom1) and shows that the root domain is in the inactive state.
| # ldm stop-domain root-dom1 LDom root-dom1 stopped # ldm unbind-domain root-dom1 # ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- UART 16 14G 0.2% 4h 59m guest0 active -n---- 5100 32 32G 0.0% 1h 55m guest1 active -n---- 5101 32 32G 0.0% 1h 46m root-dom0 active -n--v- 5000 24 24G 0.0% 2h 29m root-dom1 inactive ------ 24 24G |
- c. Confirm that all of the I/O devices in the SPARC M10-4S to be replaced have been canceled.Execute the ldm list-io command to confirm that all the I/O devices have been released.
| # ldm list-io NAME TYPE BUS DOMAIN STATUS ---- ---- --- ------ ------ PCIE0 BUS PCIE0 primary IOV PCIE1 BUS PCIE1 root-dom0 IOV PCIE2 BUS PCIE2 root-dom0 IOV PCIE3 BUS PCIE3 root-dom0 IOV PCIE4 BUS PCIE4 primary IOV PCIE5 BUS PCIE5 root-dom0 IOV PCIE6 BUS PCIE6 root-dom0 IOV PCIE7 BUS PCIE7 root-dom0 IOV PCIE8 BUS PCIE8 PCIE9 BUS PCIE9 PCIE10 BUS PCIE10 PCIE11 BUS PCIE11 PCIE12 BUS PCIE12 PCIE13 BUS PCIE13 PCIE14 BUS PCIE14 PCIE15 BUS PCIE15 (Omitted) |
- Release the system board (PSB<BB>) of the SPARC M10-4S from the physical partition.
- a. Execute the deleteboard -c disconnect command to release the PSB from the physical partition.
| XSCF> deleteboard -c disconnect 01-0 PSB#01-0 will be unconfigured from PPAR immediately. Continue?[y|n] :y Start unconfigure preparation of PSB. [1200sec] 0end Unconfigure preparation of PSB has completed. Start unconfiguring PSB from PPAR. [7200sec] 0..... 30..... 60....end Unconfigured PSB from PPAR. PSB power off sequence started. [1200sec] 0..... 30..... 60..... 90.....120.....150.....end Operation has completed. |
- b. Execute the showresult command to check the exit status of the deleteboard command that was just executed.An end value of 0 indicates the normal termination of the deleteboard command.If the end value is other than 0 or if an error message is displayed upon executing the deleteboard command, it indicates abnormal termination of the deleteboard command. By referring to "C.1.2 deleteboard" based on the error message, identify the error and then take corrective action.
| XSCF> showresult 0 |
- c. Execute the showboards command to check the PSB status.Confirm that the PSB in the SPARC M10-4S to be replaced is in the “Assigned" state and that the [Pwr], [Conn], and [Conf] columns all show "n."
| XSCF> showboards -p 0 PSB PPAR-ID(LSB) Assignment Pwr Conn Conf Test Fault ---- ------------ ----------- ---- ---- ---- ------- -------- 00-0 00(00) Assigned y y y Passed Normal 01-0 00(01) Assigned n n n Passed Normal |
- Execute the replacefru command to replace the SPARC M10-4S.
| XSCF> replacefru |
| Note - For details on the replacement of SPARC M10-4Ss by using the replacefru command, see "5.8 Releasing a SPARC M10-4/M10-4S FRU from the System with the replacefru Command" and "6.2 Incorporating a SPARC M10-4/M10-4S FRU into the System with the replacefru Command" in the Fujitsu M10-4/Fujitsu M10-4S/SPARC M10-4/SPARC M10-4S Service Manual. |
- Incorporate the PSB into the physical partition.
- a. Execute the showboards command to check the PSB status.Confirm that the PSB in the SPARC M10-4S to be replaced is in the Assigned state and that the [Pwr], [Conn], and [Conf] columns all show "n."
| XSCF> showboards -p 0 PSB PPAR-ID(LSB) Assignment Pwr Conn Conf Test Fault ---- ------------ ----------- ---- ---- ---- ------- -------- 00-0 00(00) Assigned y y y Passed Normal 01-0 00(01) Assigned n n n Passed Normal |
- b. Execute the addboard -c configure command to incorporate the PSB into the physical partition.To recover the original logical domain configuration, execute the addboard -c configure command with the -m bind=resource option specified.
| XSCF> addboard -c configure -m bind=resource -p 0 01-0 PSB#01-0 will be configured into PPAR-ID 0. Continue?[y|n] :y Start connecting PSB to PPAR. [3600sec] 0..... 30..... 60..... 90.....120.....150.....180.....210.....240..... 270.....300.....330.....360.....390.....420.....450.....480.....510..... 540.....570.....600.....630.....660.....690.....720.....750.....780..... 810.....840.....870.....900.....930.....960.....end Connected PSB to PPAR. Start configuring PSB to Logical Domains (LDoms) Manager. [1800sec] 0.....end Configured PSB to Logical Domains (LDoms) Manager. Operation has completed. |
| Note - If an error message appears during execution of the addboard command, see "C.1.1 addboard," and then identify the error and take corrective action. |
- c. Execute the showresult command to check the exit status of the addboard command that was just executed.An end value of 0 indicates the normal termination of the addboard command.If the end value is other than 0 or if an error message is displayed upon executing the addboard command, it indicates abnormal termination of the addboard command. By referring to "C.1.1 addboard" based on the error message, identify the error and then take corrective action.
| XSCF> showresult 0 |
- d. Execute the showboards command to check the PSB status.Confirm that both of the [Conn] and [Conf] columns show "y" after the PSB in the replaced SPARC M10-4S has been successfully incorporated.
| XSCF> showboards -p 0 PSB PPAR-ID(LSB) Assignment Pwr Conn Conf Test Fault ---- ------------ ----------- ---- ---- ---- ------- -------- 00-0 00(00) Assigned y y y Passed Normal 01-0 00(01) Assigned y y y Passed Normal |
- Check the logical domain operation status.
- a. Execute the console command to connect to the console of the control domain and then log in to it.
| XSCF> console -p 0 |
- b. Execute the ldm list-domain command to confirm that the logical domain operation status has not changed after the addition of the SPARC M10-4S PSB (BB).To check the logical domain operation status, check the [STATE] and [FLAGS] combination. If [STATE] indicates "active", the second character from the left of the string in [FLAGS] has the following meaning."n": Oracle Solaris is operating"t": OpenBoot PROM status"-": In another state (including [STATE] other than "active")
| # ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- UART 16 14G 0.0% 10h 30m guest0 active -n---- 5100 32 32G 0.0% 2h 50s guest1 active -n---- 5101 32 32G 0.0% 2h 30s root-dom0 active -n--v- 5000 24 24G 0.0% 3h 20s root-dom1 inactive ------ 24 24G |
- Restart the use of the I/O devices.
- a. Reassign root complexes.Execute the ldm bind-domain and ldm start-domain commands to start the root domain in the unbind state to which root complexes in the replaced SPARC M10-4S were assigned.The following example starts the root domain (root-dom1) in the unbind state, and confirms that it has started.
| # ldm bind-domain root-dom1 # ldm start-domain root-dom1 LDom root-dom1 started # ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- UART 16 16G 0.2% 3h 8m guest0 active -n---- 5100 32 32G 0.0% 3h 8m guest1 active -n---- 5101 32 32G 0.0% 3h 8m root-dom0 active -n--v- 5000 24 24G 0.0% 3h 8m root-dom1 active -n--v- 5001 24 24G 7.3% 8s |
- Execute the ldm list-io command to confirm that the physical I/O devices are assigned to the root domain that has just started.
| # ldm list-io NAME TYPE BUS DOMAIN STATUS ---- ---- --- ------ ------ PCIE0 BUS PCIE0 primary IOV PCIE1 BUS PCIE1 root-dom0 IOV PCIE2 BUS PCIE2 root-dom0 IOV PCIE3 BUS PCIE3 root-dom0 IOV PCIE4 BUS PCIE4 primary IOV PCIE5 BUS PCIE5 root-dom0 IOV PCIE6 BUS PCIE6 root-dom0 IOV PCIE7 BUS PCIE7 root-dom0 IOV PCIE8 BUS PCIE8 PCIE9 BUS PCIE9 root-dom1 IOV PCIE10 BUS PCIE10 root-dom1 IOV PCIE11 BUS PCIE11 root-dom1 IOV PCIE12 BUS PCIE12 PCIE13 BUS PCIE13 root-dom1 IOV PCIE14 BUS PCIE14 root-dom1 IOV PCIE15 BUS PCIE15 root-dom1 IOV .... /BB1/CMUL/NET0 PCIE PCIE8 UNK /BB1/CMUL/SASHBA PCIE PCIE8 UNK /BB1/PCI0 PCIE PCIE9 root-dom1OCC /BB1/PCI3 PCIE PCIE10 root-dom1OCC /BB1/PCI4 PCIE PCIE10 root-dom1OCC /BB1/PCI7 PCIE PCIE11 root-dom1OCC /BB1/PCI8 PCIE PCIE11 root-dom1OCC /BB1/CMUL/NET2 PCIE PCIE12 UNK /BB1/PCI1 PCIE PCIE13 root-dom1OCC /BB1/PCI2 PCIE PCIE13 root-dom1OCC /BB1/PCI5 PCIE PCIE14 root-dom1OCC /BB1/PCI6 PCIE PCIE14 root-dom1OCC /BB1/PCI9 PCIE PCIE15 root-dom1OCC /BB1/PCI10 PCIE PCIE15 root-dom1OCC |
- b. Add the virtual I/O device from the root domain to the guest domain.Execute the ldm add-vdisk and ldm add-vnet commands to add, to each guest domain, the virtual disk (vdisk) and virtual network device (vnet) supported for the virtual I/O service of the started root domain.The following example indicates the execution of the command for adding the virtual disk (vdisk10) and virtual network device (vnet10) that use the virtual I/O service of BB#01 root domain (root-dom1).Perform the same addition for the guest domain (guest1).
| # ldm add-vdisk id=1 vdisk10 vol10@vds1 guest0 # ldm add-vnet id=1 vnet10 vsw10 guest0 |
| Note - To add the virtual I/O device again, it is necessary to specify the ID assigned beforehand. You can check the ID from the result of execution of the ldm list -l command in the status used before the virtual I/O device is removed. |
- c. Incorporate the virtual I/O devices assigned to a guest domain into the redundant configuration.Once the root domain (root-dom1) to which root complexes in BB#1 have been assigned has started, the virtual I/O device services corresponding to each guest domain are also started.Log in to each guest domain, and then incorporate the virtual I/O devices from root-dom1 that were previously canceled into the redundant configuration. For details on how to use the redundant configuration software, see the documentation for the software for that redundant configuration.
- The following describes an example of incorporating a virtual network device (vnet1) into the IPMP configuration. For details on the commands, see the manual for Oracle Solaris.First, log in to the guest domain (guest0).
| # ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- UART 16 14G 0.0% 4h 17m guest0 active -n---- 5100 32 32G 0.0% 1h 13m guest1 active -n---- 5101 32 32G 0.0% 1h 4m root-dom0 active -n---- 5000 24 24G 0.0% 1h 47m root-dom1 active -n---- 5001 24 24G 0.0% 1h 19m # telnet localhost 5100 .... guest0# |
- Execute the dladm show-phys command to check the mapping between the virtual network interface (vnet1) and the network interface name (net1).
| guest0# dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE net0 Ethernet up 0 unknown vnet0 net1 Ethernet up 0 unknown vnet1 |
- Execute the ipadm create-ip command, the ipadm set-ifprop command, and the ipadm add-ipmp command to register net1 as a standby device of ipmp0.
| guest0# ipadm create-ip net1 guest0# ipadm set-ifprop -p standby=on -m ip net1 guest0# ipadm add-ipmp -i net1 ipmp0 |
- Execute the ipmpstat -i command to confirm that STATE of the network interface configuring IPMP indicates ok.Perform the same step for the other guest domain (guest1).
| guest0# ipmpstat -i INTERFACE ACTIVE GROUP FLAGS LINK PROBE STATE net0 yes ipmp0 -smbM-- up disabled ok net1 no ipmp0 -s---d- up disabled ok |
- Restore the system volume and I/O devices on the control domain to a redundant configuration.
- a. Add the root complex configuration for the control domain.By means of delayed configuration, add the root complexes in BB#01 that were previously removed from the control domain.First, place the control domain in delayed reconfiguration mode.
| # ldm start-reconf primary Initiating a delayed reconfiguration operation on the primary domain. All configuration changes for other domains are disabled until the primary domain reboots, at which time the new configuration for the primary domain will also take effect. |
- Execute the ldm list-io command to check the unassigned root complexes.The following example shows that root complexes with BB1 devices, PCIE8 and PCIE12 are not assigned.
| # ldm list-io NAME TYPE BUS DOMAIN STATUS ---- ---- --- ------ ------ PCIE0 BUS PCIE0 primary IOV PCIE1 BUS PCIE1 root-dom0 IOV PCIE2 BUS PCIE2 root-dom0 IOV PCIE3 BUS PCIE3 root-dom0 IOV PCIE4 BUS PCIE4 primary IOV PCIE5 BUS PCIE5 root-dom0 IOV PCIE6 BUS PCIE6 root-dom0 IOV PCIE7 BUS PCIE7 root-dom0 IOV PCIE8 BUS PCIE8 PCIE9 BUS PCIE9 root-dom1 IOV PCIE10 BUS PCIE10 root-dom1 IOV PCIE11 BUS PCIE11 root-dom1 IOV PCIE12 BUS PCIE12 PCIE13 BUS PCIE13 root-dom1 IOV (Omitted) |
- Execute the ldm add-io command to add PCIE8 and PCIE12 to primary, and then restart Oracle Solaris.
| # ldm add-io PCIE8 primary ------------------------------------------------------------------------------ Notice: The primary domain is in the process of a delayed reconfiguration. Any changes made to the primary domain will only take effect after it reboots. ------------------------------------------------------------------------------ # ldm add-io PCIE12 primary ------------------------------------------------------------------------------ Notice: The primary domain is in the process of a delayed reconfiguration. Any changes made to the primary domain will only take effect after it reboots. ------------------------------------------------------------------------------ # shutdown -i6 -g0 -y .... |
- Once Oracle Solaris has started, execute the ldm list-io command to confirm that the root complexes in BB#01 have been added to the control domain.
| # ldm list-io | grep primary PCIE0 BUS PCIE0 primary IOV PCIE4 BUS PCIE4 primary IOV PCIE8 BUS PCIE8 primary PCIE12 BUS PCIE12 root-dom1 /BB0/CMUL/NET0 PCIE PCIE0 primary OCC /BB0/CMUL/SASHBA PCIE PCIE0 primary OCC /BB0/CMUL/NET2 PCIE PCIE4 primary OCC |
- b. Place the system volume in the control domain in a redundant configuration.Execute the zpool status command in the control domain to check the mirroring configuration status.The following example describes how to configure the ZFS mirroring function for the system volume in the control domain.
| # zpool status rpool pool: rpool state: ONLINE scan: resilvered 29.1M in 0h0m with 0 errors on Thu Jan 23 17:27:59 2014 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 c2t50000393E802CCE2d0s0 ONLINE 0 0 0 errors: No known data errors |
- Execute the zpool attach command to incorporate the disks into a mirroring configuration.
| # zpool attach rpool c2t50000393E802CCE2d0s0 c3t50000393A803B13Ed0s0 Make sure to wait until resilver is done before rebooting. # |
- Execute the zpool status command, and then confirm that the mirroring configuration has been established.Use the zpool status command to confirm whether synchronization processing (resilver) is completed.The following shows an example of the display during synchronization processing.
| # zpool status rpool pool: rpool state: DEGRADED status: One or more devices is currently being resilvered. The pool will continue to function in a degraded state. action: Wait for the resilver to complete. Run 'zpool status -v' to see device specific details. scan: resilver in progress since Mon Jan 27 15:55:47 2014 21.1G scanned out of 70.6G at 120M/s, 0h7m to go 21.0G resilvered, 29.84% done config: NAME STATE READ WRITE CKSUM rpool DEGRADED 0 0 0 mirror-0 DEGRADED 0 0 0 c2t50000393E802CCE2d0s0 ONLINE 0 0 0 c3t50000393A803B13Ed0s0 DEGRADED 0 0 0 (resilvering) errors: No known data errors |
- Once synchronization processing is complete, the displayed screen will be as follows:
| # zpool status rpool pool: rpool state: ONLINE scan: resilvered 70.6G in 0h9m with 0 errors on Mon Jan 27 16:05:34 2014 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c2t50000393E802CCE2d0s0 ONLINE 0 0 0 c3t50000393A803B13Ed0s0 ONLINE 0 0 0 errors: No known data errors |
- If you are using other devices in BB#01, establish a redundant configuration or resume the use of the devices. For details on how to establish a redundant configuration or resume the use of devices, see the documentation for the software for that redundant configuration and Oracle Solaris.
< Previous Page | Next Page >