A.4.3 Procedure for Expansion (When Dynamic Assignment of the PCIe Bus is Available)
A.4.3 Procedure for Expansion (When Dynamic Assignment of the PCIe Bus is Available)
The following shows the procedure for expanding a SPARC M10-4S with a 1BB configuration to a 2 BB configuration through dynamic reconfiguration according to the configuration example given in "Figure A-5 Example of the Configuration After Expansion From 1BB to 2BB." The procedure is for an environment where the PCIe bus dynamic assignment is available (XCP 2240 or later with Oracle VM Server for SPARC 3.2 or later and the root domain with Oracle Solaris 11.2 SRU11.2.8.4.0 or later).
This description also applies to SPARC M12.
- Log in to the master XSCF.
Execute the showbbstatus command to check that the XSCF to which you have logged in is the master XSCF.
If the standby XSCF is being used, log in to the master XSCF again.
| XSCF> showbbstatus BB#00 (Master) |
- Execute the showsscp command to check whether the IP address of the SP to SP communication protocol (SSCP) is the default value or a user-specified value.
| XSCF> showsscp |
- When the IP address is the default value and you are using the default value of the IP address of the SPARC M10-4S to be expanded, go to the next step.
To set a user value, set the IP address with the setsscp command. Then, use the applynetwork command to apply and check the IP address of the SSCP of the SPARC M10-4S to be expanded. Afterwards, execute the rebootxscf command to complete the setting and then go to the next step. For details on the procedure, see "7.5.6 Applying network settings" in the Fujitsu M10-4S/SPARC M10-4S Installation Guide.
- Expand SPARC M10-4S.a. Execute the addfru command to expand SPARC M10-4S according to the message.
| Note - If the IP address of the SSCP has not yet been set, executing the addfru command results in an error. |
| Note - If the addfru command is executed, the firmware version of the SPARC M10-4S to be expanded is automatically set to that of the SPARC M10-4S on which the master XSCF is running. In this example, the firmware version of the SPARC M10-4S (BB-ID 1) to be expanded is automatically set to that of the SPARC M10-4S (BB-ID 0) on which the master XSCF is running. |
- The following example expands BB#1.
| XSCF> addfru ------------------------------------------------------------------------------ Maintenance/Addition Menu Please select the chassis including added FRU. No. FRU Status --- ------------------- -------------- 1 /BB#0 Normal 2 /BB#1 Unmount 3 /BB#2 Unmount 4 /BB#3 Unmount ------------------------------------------------------------------------------ Select [1-16|c:cancel] :2 Maintenance/Addition Menu Please select the BB or a type of FRU to be added. 1. BB itself 2. PSU (Power Supply Unit) ------------------------------------------------------------------------------ Select [1,2|c:cancel] :1 Maintenance/Addition Menu Please select a FRU to be added. No. FRU Status --- ------------------- -------------- 1 /BB#1 Unmount Select [1|b:back] :1 |
- b. Mount SPARC M10-4S to be expanded, to the rack.While the previously mentioned addfru command is executed, if the message "After the added device is connected with the system, please turn on the breaker of the BB#1." appears, mount the SPARC M10-4S to be expanded in the rack.For details on the mounting procedure, see "3.4.1 Mounting the SPARC M10-4S in a rack" in the Fujitsu M10-4S/SPARC M10-4S Installation Guide.
- c. Set the identification ID (BB-ID) of the SPARC M10-4S to be expanded.See "4.1 Setting the ID (BB-ID) Identifying a Chassis" in the Fujitsu M10-4S/SPARC M10-4S Installation Guide.
- d. Connect the crossbar cable.Connect the crossbar cable between the existing SPARC M10-4S and the expanded SPARC M10-4S. For the connection cable routing figure and cable list, see "Appendix B Cable Connection Information on Building Block Configurations" in the Fujitsu M10-4S/SPARC M10-4S Installation Guide. For details on direct connections between chassis, see "4.2 Connecting Cables (for Direct Connections between Chassis)" in the Fujitsu M10-4S/SPARC M10-4S Installation Guide.
- e. Connect the XSCF BB control cable.Connect the existing SPARC M10-4S and expanded SPARC M10-4S through the XSCF BB control cable. For the routing figure and cable list of the XSCF BB control cables, see "Appendix B Cable Connection Information on Building Block Configurations" in the Fujitsu M10-4S/SPARC M10-4S Installation Guide. For details on direct connections between chassis, see "4.2 Connecting Cables (for Direct Connections between Chassis)" in the Fujitsu M10-4S/SPARC M10-4S Installation Guide.
- f. Connect the XSCF DUAL control cable.Connect the existing SPARC M10-4S and expanded SPARC M10-4S through the XSCF DUAL control cable. For details, see "4.2 Connecting Cables (for Direct Connections between Chassis)" in the Fujitsu M10-4S/SPARC M10-4S Installation Guide.
- g. Connect the serial cable or LAN cable.Connect the serial cable to the XSCF serial port of the SPARC M10-4S to be expanded. Moreover, connect the cable to each LAN port of the XSCF-LAN, GbE port, and PCIe card. For details, see "5.1 Connecting Cables to the SPARC M10-4S" in the Fujitsu M10-4S/SPARC M10-4S Installation Guide.
- h. Connect the input power to the SPARC M10-4S to be expanded.Connect the power cord of the SPARC M10-4S to be expanded to the input power.
- i. Enter "f" from the input screen of the addfru command.Enter "f" on the input screen for the addfru command executed in step a. to perform SPARC M10-4S expansion.
| 2) Please select[f:finish] :f Waiting for BB#1 to enter install state. [This operation may take up to 20 minute(s)] (progress scale reported in seconds) 0..... 30.... done Waiting for BB#1 to enter ready state. [This operation may take up to 45 minute(s)] (progress scale reported in seconds) 0..... 30..... 60... done Do you want to start to diagnose BB#1? [s:start|c:cancel] : |
- j. Skip the diagnosis of the SPARC M10-4S to be expanded to exit the addfru command.Input "c" to the input screen of the addfru command on the master XSCF, and then skip the diagnosis processing for the SPARC M10-4S to be expanded. If "The addition of BB#1 has completed." appears, input "f" and then input "c" to exit the addfru command.
| Do you want to start to diagnose BB#1? [s:start|c:cancel] :c Diagnostic tests are about to be skipped. Running diagnostic tests are strongly recommended before using BB#1. Are you sure you want to skip testing? [y:yes|n:no] :y ------------------------------------------------------------------------------ Maintenance/Addition Menu Status of the added FRU. FRU Status ------------------- -------------- /BB#1 Normal ------------------------------------------------------------------------------ [Warning:007] Running diagnostic tests on BB#1 is strongly recommended after addfru has completed. The addition of BB#1 has completed.[f:finish] :f ------------------------------------------------------------------------------ Maintenance/Addition Menu Please select the chassis including added FRU. No. FRU Status --- ------------------- -------------- 1 /BB#0 Normal 2 /BB#1 Normal ------------------------------------------------------------------------------ Select [1,2|c:cancel] :c |
- Diagnose the expanded SPARC M10-4S.a. Execute the testsb command to perform a diagnosis test for the expanded SPARC M10-4S.Execute the testsb command to perform the diagnosis test. Specify the physical system board (PSB) number of the expanded SPARC M10-4S and check the initial diagnosis and connection I/O.
| XSCF> testsb -v -p -s -y 01-0 Initial diagnosis is about to start, Continue?[y|n] :y PSB#01-0 power on sequence started. |
- If an error is displayed, see "A.2.4 Checking diagnosis results" in the Fujitsu M10-4S/SPARC M10-4S Installation Guide.
- b. Execute the diagxbu command to perform the crossbar cable diagnosis test.Execute the diagxbu command to perform the crossbar cable diagnosis test for the physical partition of the destination and for the expanded SPARC M10-4S. Specify the identification ID (BB-ID) of the expanded SPARC M10-4S and the physical partition ID (PPAR-ID) of the destination.
| XSCF> diagxbu -y -b 01 -p 00 XBU diagnosis is about to start, Continue?[y|n] :y Power on sequence started. [7200sec] 0..... 30..... 60..end XBU diagnosis started. [7200sec] 0..... 30..... 60..... 90.....120.....150.....180.....210.....240...../ 270.....300.....330.....360.....390.....420.....450.....480.....510.....\ 540.....570.....600.....630.....660.....690.....720.....750.....780.....\ 810.....840.....870.....900.....930...end completed. Power off sequence started. [1200sec] 0..... 30..... 60..... 90.....120.....150.....180.end completed. |
- c. Execute the showlogs error command to check that no error is displayed.
| XSCF> showlogs error |
- If an error is displayed, see "A.2.4 Checking diagnosis results" in the Fujitsu M10-4S/SPARC M10-4S Installation Guide.
- d. Execute the showhardconf command to check the configuration and status of the expanded SPARC M10-4S.Execute the showhardconf command to check the hardware configuration (CPU, memory, or the like) of the expanded SPARC M10-4S, and then check that Status of each unit is Normal.
| XSCF> showhardconf SPARC M10-4S; + Serial: 2081230011; Operator_Panel_Switch:Service; + System_Power:On; System_Phase:Cabinet Power On; Partition#0 PPAR_Status:Running; BB#00 Status:Normal; Role:Master; Ver:2003h; Serial:2081231002; + FRU-Part-Number: CA07361-D202 A1 ; + Power_Supply_System: ; + Memory_Size:256 GB; CMUL Status:Normal; Ver:0101h; Serial:PP1236052K ; + FRU-Part-Number:CA07361-D941 C4 /7060911 ; + Memory_Size:128 GB; Type: A ; CPU#0 Status:Normal; Ver:4142h; Serial:00322658; + Freq:3.000 GHz; Type:0x10; + Core:16; Strand:2; ： BB#01 Status:Normal; Role:Standby; Ver:0101h;Serial:7867000297; + FRU-Part-Number: CA20393-B50X A2 ; + Power_Supply_System: ; + Memory_Size:256 GB; CMUL Status:Normal; Ver:0101h; Serial:PP123406CB ; + FRU-Part-Number:CA07361-D941 C4 /7060911 ; + Memory_Size:128 GB; Type: A ; ： |
- Set the XSCF network of the expanded SPARC M10-4S.a. Execute the testsb command to perform a diagnosis test for the expanded SPARC M10-4S.Set the XSCF network for the expanded SPARC M10-4S. For details, see "7.5.2 Setting an Ethernet (XSCF-LAN) IP address" and "7.5.3 Setting a takeover IP address" in the Fujitsu M10-4S/SPARC M10-4S Installation Guide. After making this setting, execute the applynetwork command to apply and check the setting. Afterwards, execute the rebootxscf command to complete the setting and then go to the next step. For details on the procedure, see "7.5.6 Applying network settings" in the Fujitsu M10-4S/SPARC M10-4S Installation Guide.
- Set mirror mode for the memory of the expanded SPARC M10-4S.If you are not using memory mirror mode, this step is not necessary.For details on memory mirror mode, see "14.1 Configuring Memory Mirroring" in the Fujitsu SPARC M12 and Fujitsu M10/SPARC M10 System Operation and Administration Guide.a. If you are making a mirror configuration of the memory in the expanded SPARC M10-4S, use the setupfru command to set memory mirror mode.The following example places all the memories for the CPUs under the SPARC M10-4S physical system board (PSB 01-0) in memory mirror mode.
| XSCF> setupfru -m y sb 01-0 |
- b. Execute the showfru command to check the mirror mode setting of the memory.
| XSCF> showfru sb 01-0 Device Location Memory Mirror Mode sb 01-0 cpu 01-0-0 yes cpu 01-0-1 yes cpu 01-0-2 yes cpu 01-0-3 yes |
- Register the system board of the expanded SPARC M10-4S in the physical partition configuration information.a. Execute the showpcl command to check the physical partition configuration information.
| XSCF> showpcl -p 0 PPAR-ID LSB PSB Status 00 Running 00 00-0 |
- b. Execute the setpcl command to register the system board in the physical partition configuration information.Execute the setpcl command to register the expanded SPARC M10-4S system board in the physical partition configuration information for the built-in destination.In the following example, physical system board (PSB) 01-0 is mapped to logical system board (LSB) 01 of physical partition 0.
| XSCF> setpcl -p 0 -a 01=01-0 |
- c. Execute the showpcl command to check the physical partition configuration information.
| XSCF> showpcl -p 0 PPAR-ID LSB PSB Status 00 Running 00 00-0 01 01-0 |
- Register the CPU Activation key to assign CPU core resources.a. Execute the showcodusage command to check the CPU core resource information.Execute the showcodusage command to check whether the physical partition contains an assignable CPU core resource.As shown here, the system has 128 mounted CPU core resources and 128 registered CPU Activations, 64 of the CPU core resources are in use, and the number of CPU Activations currently not in use is 64.
| XSCF> showcodusage -p resource Resource In Use Installed CoD Permitted Status -------- ------ --------- ------------- ------ PROC 64 128 128 OK: 64 cores available Note: Please confirm the value of the "In Use" by the ldm command of Oracle VM Server for SPARC. The XSCF may take up to 20 minutes to reflect the "In Use" of logical domains. |
| Note - If the number of registered CPU Activations is not enough for the number of CPUs to be used, purchase CPU Activations and add the CPU Activation keys. For details on how to add a CPU Activation key, see "5.3 Adding CPU Core Resources" in the Fujitsu SPARC M12 and Fujitsu M10/SPARC M10 System Operation and Administration Guide. |
- b. Execute the showcod command to check the CPU core resource information for the physical partition.Execute the showcod command to check information on the CPU core resources assigned to the physical partition.
| XSCF> showcod -p 0 PROC Permits assigned for PPAR 0: 64 |
- If the assigned resources are insufficient, execute the setcod command to assign CPU core resources to the physical partition.The following example adds 64 CPU core resources to the physical partition.
| XSCF> setcod -p 0 -s cpu -c add 64 PROC Permits assigned for PPAR 0 : 64 -> 128 PROC Permits assigned for PPAR will be changed. Continue? [y|n] :y Completed. |
| Note - XSCF firmware of versions XCP 2250 and earlier do not support the -c add, -c delete, and -c set options. Specify the options of the setcod command as shown below to interactively add and delete CPU core resources. XSCF> setcod -s cpu |
- When the CPU core resources have been assigned, execute the showcod command again to check the information for the CPU core resources assigned to the physical partition.
| XSCF> showcod -p 0 PROC Permits assigned for PPAR 0: 128 |
- Check the logical domain operation status.a. Execute the console command to connect to the console of the control domain and then log in to it.
| XSCF> console -p 0 |
- b. Execute the ldm list-domain command to check the operation status of the logical domain.To check the logical domain operation status, check the [STATE] and [FLAGS] combination. If [STATE] indicates "active", the second character from the left of the string in [FLAGS] has the following meaning."n": Oracle Solaris is operating"t": OpenBoot PROM status"-": In another state (including [STATE] other than "active")The following example shows that the control domain, one root domain, and one guest domain are operating.
- Check whether all domains are in "active", which indicates that Oracle Solaris is in operating state, or "inactive" state. If there is a domain with OpenBoot PROM status or bound status, the dynamic reconfiguration of the physical partition may fail.
| # ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- UART 32 28G 0.0% 1h 33m guest0 active -n---- 5100 64 64G 3.1% 2s root-dom0 active -n--v- 5000 32 32G 3.1% 17m |
- Incorporate the system board (PSB<BB>) of the expanded SPARC M10-4 into the physical partition.a. Return to the XSCF shell, and then execute the showboards command to check the PSB status.Execute the showboards command to check that the PSB status of the expanded SPARCM10-4S is "SP" (system board pool).
| XSCF> showboards -p 0 PSB PPAR-ID(LSB) Assignment Pwr Conn Conf Test Fault ---- ------------ ----------- ---- ---- ---- ------- -------- 00-0 00(00) Assigned y y y Passed Normal 01-0 SP Available n n n Passed Normal |
- b. Execute the addboard command to incorporate the PSB into the physical partition.Execute the addboard -c configure command to incorporate the PSB into the physical partition.
| XSCF> addboard -c configure -p 0 01-0 PSB#01-0 will be configured into PPAR-ID 0. Continue?[y|n] :y Start connecting PSB to PPAR. [3600sec] 0..... 30..... 60..... 90.....120.....150.....180.....210.....240..... 270.....300.....330.....360.....390.....420.....450.....480.....510..... 540.....570.....600.....630.....660.....690.....720.....750.....780..... 810.....840.....870.....900.....930.....960.....end Connected PSB to PPAR. Start configuring PSB to Logical Domains (LDoms) Manager. [1800sec] 0.....end Configured PSB to Logical Domains (LDoms) Manager. Operation has completed. |
| Note - If an error message appears during execution of the addboard command, see "C.1.1 addboard," and then identify the error and take corrective action. |
- c. Execute the showresult command to check the exit status of the addboard command that was just executed.An end value of 0 indicates the normal termination of the addboard command.If the end value is other than 0 or if an error message is displayed upon executing the addboard command, it indicates abnormal termination of the addboard command. By referring to "C.1.1 addboard" based on the error message, identify the error and then take corrective action.
| XSCF> showresult 0 |
- d. Execute the showboards command to check the PSB status.Confirm that the PSB in the added SPARC M10-4S is in the Assigned state and that the Pwr, Conn, and Conf columns all show "y."
| XSCF> showboards -p 0 PSB PPAR-ID(LSB) Assignment Pwr Conn Conf Test Fault ---- ------------ ----------- ---- ---- ---- ------- -------- 00-0 00(00) Assigned y y y Passed Normal 01-0 00(01) Assigned y y y Passed Normal |
- Check the logical domain operation status.a. Execute the console command to connect to the console of the control domain and then log in to it.
| XSCF> console -p 0 |
- b. Execute the ldm list-domain command to check the operation status of the logical domain.To check the logical domain operation status, check the [STATE] and [FLAGS] combination. If [STATE] indicates "active", the second character from the left of the string in [FLAGS] has the following meaning.
"n": Oracle Solaris is operating
"t": OpenBoot PROM status
"-": In another state (including [STATE] other than "active")The following example shows that the control domain, one root domain, and one guest domain are operating.
| # ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- UART 32 28G 0.0% 2h 3m guest0 active -n---- 5100 64 64G 3.1% 33m root-dom0 active -n--v- 5000 32 32G 3.1% 47m |
- Change the configuration of the existing logical domain and then add a new logical domain.
The resources of the added SPARC M10-4S are assigned to the existing logical domain or to a newly configured logical domain. The following example performs the procedure for creating a mirroring configuration according to the configuration example. In other words, add the CPU cores and memory of the control domain and then add guest domain guest1 and root domain root-dom1 to create a redundant configuration of the virtual I/O for each guest domain. Furthermore, create a mirroring configuration of the system volume of the control domain with an internal hard disk.a. Execute the ldm list-devices command to check the state of the added hardware resources.
| # ldm list-devices CORE ID %FREE CPUSET 128 100 (256, 257) 132 100 (264, 265) 136 100 (272, 273) 140 100 (280, 281) .... MEMORY PA SIZE BOUND 0x700000000000 32G 0x720000000000 32G 0x740000000000 32G 0x760050000000 31488M .... |
- b. Execute the ldm add-core command and the ldm add-memory command to add the CPU cores and memory resources of the control domain.Execute the ldm add-core and ldm add-memory commands to add the CPU cores and memory of the control domain. Check that the CPU cores and memory have been added by executing the ldm list-domain command.
| # ldm add-core 16 primary # ldm add-memory 56G primary # ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- UART 64 56G 0.0% 2h 13m guest0 active -n---- 5100 64 64G 0.1% 43m root-dom0 active -n--v- 5000 32 32G 0.1% 57m |
- c. Create a new logical domain to which to assign the CPU cores and memory resources.The example here adds the root domain (root-dom1) and guest domain (guest1) to which CPU cores and memory can be assigned. However, suppose that when a domain is configured, the number of CPU cores is first set with the ldm set-core command and then the memory size to be assigned is set with the ldm set-memory command. In this case, the memory blocks assigned to the logical domain become difficult to distribute (fragmentation) and memory placement becomes easy to consider at building block release.The following shows command execution examples.
| # ldm add-domain root-dom1 # ldm add-domain guest1 # ldm set-core 16 root-dom1 # ldm set-core 32 guest1 # ldm set-memory 32G root-dom1 # ldm set-memory 64G guest1 |
| Note - To enable physical partition dynamic reconfiguration when the version of Oracle VM Server for SPARC is earlier than 3.2, set a multiple of (number of CPU cores x 256 MB) for the memory size to be assigned to each logical domain. |
- d. Assign the root complex to the root domain.The following example assigns the root complexes of the added SPARC M10-4S to the root domain (root-dom1).Execute the ldm list-io command to check the root complexes to be added to the root domain (root-dom1).A device whose NAME begins with "/BB1" is on BB-ID 1 and the string shown in BUS of that device indicates the root complex.
| # ldm list-io NAME TYPE BUS DOMAIN STATUS ---- ---- --- ------ ------ PCIE0 BUS PCIE0 primary IOV PCIE1 BUS PCIE1 root-dom0 IOV PCIE2 BUS PCIE2 root-dom0 IOV PCIE3 BUS PCIE3 root-dom0 IOV PCIE4 BUS PCIE4 primary IOV PCIE5 BUS PCIE5 root-dom0 IOV PCIE6 BUS PCIE6 root-dom0 IOV PCIE7 BUS PCIE7 root-dom0 IOV PCIE8 BUS PCIE8 PCIE9 BUS PCIE9 PCIE10 BUS PCIE10 PCIE11 BUS PCIE11 PCIE12 BUS PCIE12 PCIE13 BUS PCIE13 PCIE14 BUS PCIE14 PCIE15 BUS PCIE15 .... /BB1/CMUL/NET0 PCIE PCIE8 /BB1/CMUL/SASHBA PCIE PCIE8 /BB1/PCI0 PCIE PCIE9 /BB1/PCI3 PCIE PCIE10 /BB1/PCI4 PCIE PCIE10 /BB1/PCI7 PCIE PCIE11 /BB1/PCI8 PCIE PCIE11 /BB1/CMUL/NET2 PCIE PCIE12 /BB1/PCI1 PCIE PCIE13 /BB1/PCI2 PCIE PCIE13 /BB1/PCI5 PCIE PCIE14 /BB1/PCI6 PCIE PCIE14 /BB1/PCI9 PCIE PCIE15 /BB1/PCI10 PCIE PCIE15 |
- Execute the ldm add-io command to add the root complexes to the root domain (root-dom1). Then, execute the ldm list-io command to perform a check.The following example adds BB1 root complexes (PCIE9, PCIE10, PCIE11, PCIE13, PCIE14, and PCIE15) other than root complexes (PCIE8 and PCIE12) of the above-mentioned on-board devices (/BB1/CMUL/NET0, /BB1/CMUL/SASHBA, and /BB1/CMUL/NET2), to root-dom1.
| # ldm add-io PCIE9 root-dom1 # ldm add-io PCIE10 root-dom1 .... # ldm list-io NAME TYPE BUS DOMAIN STATUS ---- ---- --- ------ ------ PCIE0 BUS PCIE0 primary IOV PCIE1 BUS PCIE1 root-dom0 IOV PCIE2 BUS PCIE2 root-dom0 IOV PCIE3 BUS PCIE3 root-dom0 IOV PCIE4 BUS PCIE4 primary IOV PCIE5 BUS PCIE5 root-dom0 IOV PCIE6 BUS PCIE6 root-dom0 IOV PCIE7 BUS PCIE7 root-dom0 IOV PCIE8 BUS PCIE8 PCIE9 BUS PCIE9 root-dom1 IOV PCIE10 BUS PCIE10 root-dom1 IOV PCIE11 BUS PCIE11 root-dom1 IOV PCIE12 BUS PCIE12 PCIE13 BUS PCIE13 root-dom1 IOV PCIE14 BUS PCIE14 root-dom1 IOV PCIE15 BUS PCIE15 root-dom1 IOV |
- e. Assign the console device of the root domain and then start it.Assign the console device to the root domain (root-dom1) with the ldm set-vconsole command and then start it with the ldm bind-domain and ldm start-domain commands.
| # ldm set-vconsole port=5001 vcc0 root-dom1 # ldm bind-domain root-dom1 # ldm start-domain root-dom1 LDom root-dom1 started # telnet localhost 5001 .... |
- f. Install Oracle Solaris in the root domain.Install Oracle Solaris in the started root domain. For details on how to install Oracle Solaris, see documentation related to Oracle Solaris.
- g. Check the physical device to be assigned to the virtual service of the root domain.After installing Oracle Solaris, log in to the root domain and then execute the format command to check the name of the device to be assigned to the virtual disk service.
Executing the ldm list-io -l command on the control domain before logging in to the root domain displays the device path associated with the PCIe endpoint. So, you can check the path through which the disk is connected, by comparing the name of that device path with the name of the device path of the disk displayed with the format command.
| # ldm list-io -l NAME TYPE BUS DOMAIN STATUS ---- ---- --- ------ ------ .... /BB1/PCI0 PCIE PCIE9 root-dom1OCC [pci@8900/pci@4/pci@0/pci@0] /BB1/PCI3 PCIE PCIE10 root-dom1OCC [pci@8a00/pci@4/pci@0/pci@0] /BB1/PCI4 PCIE PCIE10 root-dom1OCC [pci@8a00/pci@4/pci@0/pci@8].... # telnet localhost 5001 .... root-dom1# format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c3t500000E01BDA70B2d0 <FUJITSU-MBB2147RC-3703 cyl 14087 alt 2 hd 24 sec 848> / pci@8900/pci@4/pci@0/pci@0/scsi@0/iport@f/disk@w500000e01bda70b2,0 1. c3t50000393A803B13Ed0 <TOSHIBA-MBF2300RC-3706 cyl 46873 alt 2 hd 20 sec 625> /pci@8a00/pci@4/pci@0/pci@0/scsi@0/iport@f/disk@w50000393a803b13e,0 .... Specify disk (enter its number): ^C |
- Subsequently, executing the dladm show-phys -L command enables you to check the Ethernet interface name and physical location assigned to the root domain.
| root-dom1# dladm show-phys -L LINK DEVICE LOC net0 igb0 /BB1/PCI0 net1 igb1 /BB1/PCI0 net2 igb2 /BB1/PCI0 net3 igb3 /BB1/PCI0 |
- h. Assign the virtual I/O service to the root domain.Return to the control domain to add the virtual I/O service to the root domain (root-dom1). For details on the I/O device setting, see the Oracle VM Server for SPARC Administration Guide provided by Oracle Corporation.The following example adds the virtual disk service (vds1) and virtual switch network interfaces (vsw10 and vsw11) to the added root domain (root-dom1).
| # ldm add-vdiskserver vds1 root-dom1 # ldm add-vdiskserverdevice /dev/dsk/cXtXXXXXXXXXXXXXXXXdXsX vol0@vds1 # ldm add-vdiskserverdevice /dev/dsk/cXtXXXXXXXXXXXXXXXXdXsX vol1@vds1 # ldm add-vswitch net-dev=net1 vsw10 root-dom1 # ldm add-vswitch net-dev=net2 vsw11 root-dom1 |
- i. Assign the virtual I/O device to the guest domain.Add the virtual I/O device that uses the added virtual I/O service, to each guest domain.The following example adds the virtual I/O devices (vdisk10, vnet10) that use the virtual I/O service of root-dom1 to the existing guest domain (guest0). It also adds the virtual I/O devices (vdisk1, vdisk11, vnet1, and vnet11) that use the virtual I/O services of root-dom0 and root-dom1 and the virtual console (vcons), to a guest domain (guest1) to be newly added.
| # ldm add-vdisk vdisk10 vol0@vds1 guest0 # ldm add-vnet vnet10 vsw10 guest0 # ldm add-vdisk vdisk1 vol1@vds0 guest1 # ldm add-vdisk vdisk11 vol1@vds1 guest1 # ldm add-vnet vnet1 vsw1 guest1 # ldm add-vnet vnet11 vsw11 guest1 # ldm set-vconsole port=5101 guest1 |
- j. Start the added guest domain.Execute the ldm bind-domain and ldm start-domain commands to start the added guest domain (guest1).
| # ldm bind-domain guest1 # ldm start-domain guest1 LDom guest1 started |
- k. Install Oracle Solaris in the added guest domain.Install Oracle Solaris in the added guest domain. For details on how to install Oracle Solaris, see the documentation related to Oracle Solaris.
- l. Establish a redundant configuration for the I/O of each guest domain.Change the virtual I/O device added to each guest domain to establish a redundant configuration. For details on how to set the redundant configuration, see the documentation related to the software for each redundant configuration.
- m. Establish a redundant configuration for the system volume of the control domain.The example here assigns the on-board device of the added SPARC M10-4S to the control domain to create a redundant configuration.
- Add the root complexes (PCIE8 and PCIE12) of the added SPARC M10-4S to the control domain.
| # ldm add-io PCIE8 primary # ldm add-io PCIE12 primary |
- Then, establish a redundant configuration for the system volume. The following example checks the disk with the format command to establish a mirror configuration with ZFS. For details on how to set the redundant configuration, see the documentation related to the software for each redundant configuration.
| # format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c2t50000393E802CCE2d0 <TOSHIBA-MBF2300RC-3706 cyl 46873 alt 2 hd 20 sec 625> /pci@8000/pci@4/pci@0/pci@0/scsi@0/iport@f/disk@w50000393e802cce2,0 /dev/chassis/FUJITSU-BBEXP.500000e0e06d027f/0123_HDD00/disk 1. c3t50000393A803B13Ed0 <TOSHIBA-MBF2300RC-3706 cyl 46873 alt 2 hd 20 sec 625> /pci@8800/pci@4/pci@0/pci@0/scsi@0/iport@f/disk@w50000393a803b13e,0 /dev/chassis/FUJITSU-BBEXP.500000e0e06d243f/022U_HDD01/disk Specify disk (enter its number): ^C # zpool status rpool pool: rpool state: ONLINE scan: resilvered 70.6G in 0h9m with 0 errors on Mon Jan 27 16:05:34 2014 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 c2t50000393E802CCE2d0s0 ONLINE 0 0 0 errors: No known data errors # zpool attach rpool c2t50000393E802CCE2d0s0 c3t50000393A803B13Ed0s0 Make sure to wait until resilver is done before rebooting |
- Execute the zpool status command, and then confirm that the mirroring configuration has been established. At this time, execute the zpool status command to check whether the synchronization processing (resilver) has been completed. The following example indicates that synchronization processing is in progress.
| # zpool status rpool pool: rpool state: DEGRADED status: One or more devices is currently being resilvered. The pool will continue to function in a degraded state. action: Wait for the resilver to complete. Run 'zpool status -v' to see device specific details. scan: resilver in progress since Mon Jan 27 15:55:47 2014 21.1G scanned out of 70.6G at 120M/s, 0h7m to go 21.0G resilvered, 29.84% done config: NAME STATE READ WRITE CKSUM rpool DEGRADED 0 0 0 mirror-0 DEGRADED 0 0 0 c2t50000393E802CCE2d0s0 ONLINE 0 0 0 c3t50000393A803B13Ed0s0 DEGRADED 0 0 0 (resilvering) errors: No known data errors |
- Once synchronization processing is complete, the displayed screen will be as follows:
| # zpool status rpool pool: rpool state: ONLINE scan: resilvered 70.6G in 0h9m with 0 errors on Mon Jan 27 16:05:34 2014 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c2t50000393E802CCE2d0s0 ONLINE 0 0 0 c3t50000393A803B13Ed0s0 ONLINE 0 0 0 errors: No known data errors |
- n. Check and adjust the memory placement of each logical domain.Execute the ldm list-devices -a memory command to check the memory block placement.
| # ldm list-devices -a memory MEMORY PA SIZE BOUND 0x700000000000 32G root-dom1 0x720000000000 32G guest1 0x740000000000 32G guest1 0x760000800000 1272M _sys_ 0x760050000000 16G primary 0x760450000000 10G primary 0x7606d0000000 4864M 0x780000000000 32G guest0 0x7a0000000000 32G guest0 0x7c0000000000 256M primary 0x7c0010000000 3584M root-dom0 0x7c00f0000000 24832M primary 0x7c0700000000 4G root-dom0 0x7e0000800000 1272M _sys_ 0x7e0050000000 512M _sys_ 0x7e0070000000 256M _sys_ 0x7e0080000000 25088M root-dom0 0x7e06a0000000 3584M primary 0x7e0780000000 2G primary |
- Suppose that memory blocks assigned to a logical domain or unassigned memory blocks are divided into small fragments. You can collect the fragmented memory blocks into a large continuous region by unbinding the logical domain and then binding it again. This enables the SPARC M10-4S to be easily added or deleted.For details on the procedure, see step 11 in "A.2.2 Example of the Physical Partition Configuration Procedure" of "A.2 For New Installations from XCP 2220 or Later, Where the Logical Domain Configuration Has No Free Space for Hardware Resources."
- o. Save logical domain configuration information.Execute the ldm list-domain command to check that the configured logical domain has started.
| # ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- UART 64 56G 64% 2h 54m guest0 active -n---- 5100 64 64G 42% 2h 54m guest1 active -n---- 5101 64 64G 11% 2h 54m root-dom0 active -n--v- 5000 32 32G 20% 2h 54m root-dom1 active -n--v- 5001 32 32G 18% 2h 54m |
- Execute the ldm add-spconfig command to save the logical domain configuration information to the XSCF.
| # ldm add-spconfig ldm-set4 # ldm list-spconfig factory-default ldm-set1 ldm-set2 ldm-set3 ldm-set4 [current] |
< Previous Page | Next Page >