Login or Register to Ask a Question and Join Our Community


Solaris

IPMP in the Service Domain (Oracle VM Sparc)

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems Solaris IPMP in the Service Domain (Oracle VM Sparc)
# 1  
Old 03-12-2014
IPMP in the Service Domain (Oracle VM Sparc)
Hello

Using VM server 3.1, I would like to configure a vswitch based on IPMP of 2 physical port so that IPMP is only configured on the primary domain and no more on guest domains.

Regarding the documentation (Configuring IPMP in a Logical Domains Environment - Oracle VM Server for SPARC 3.0 Administration Guide), it should be possible possible, but when I try to create the switch, I get the following error message :
root@xxxxxxxx:~# ldm add-vswitch net-dev=ipmp1 linkprop=phys-state primary-vsw5 primary
NIC ipmp1 is not valid on service domain primary

Here is my config :
Code:
root@xxxxxxxxxx:~# ldm list-services primary
VCC
    NAME             LDOM             PORT-RANGE
    primary-vcc0     primary          5000-5100
 
 
VSW
    NAME             LDOM             MAC               NET-DEV   ID   DEVICE     LINKPROP   DEFAULT-VLAN-ID PVID VID                  MTU   MODE   INTER-VNET-LINK
    primary-sw0      primary          00:14:4f:f9:16:a7 net1      0    switch@0   phys-state 1               1                         1500         on
    primary-sw1      primary          00:14:4f:fa:13:50 net5      1    switch@1   phys-state 1               1                         1500         on
    primary-sw2      primary          00:14:4f:fa:4d:a8 net2      2    switch@2   phys-state 1               1                         1500         on
    primary-sw3      primary          00:14:4f:f8:09:cd net6      3    switch@3   phys-state 1               1                         1500         on
 
 
VDS
    NAME             LDOM             VOLUME         OPTIONS          MPGROUP        DEVICE
    primary-vds0     primary
 
root@xxxxxxxxx:~# ipadm
NAME              CLASS/TYPE STATE        UNDER      ADDR
ipmp0             ipmp       ok           --         --
   ipmp0/v4       static     ok           --         108.1.2.224/24
ipmp1             ipmp       down         --         --
ipmp2             ipmp       down         --         --
lo0               loopback   ok           --         --
   lo0/v4         static     ok           --         127.0.0.1/8
net0              ip         ok           ipmp0      --
net1              ip         ok           ipmp1      --
net2              ip         ok           ipmp2      --
net4              ip         ok           ipmp0      --
net5              ip         ok           ipmp1      --
net6              ip         ok           ipmp2      --
net8              ip         ok           --         --
   net8/v4        static     ok           --         169.254.182.77/24

Any help will be appreciated ...

Thanks in advance

Best regards
Last edited by Franklin52; 03-12-2014 at 04:24 AM.. Reason: Please use code tags
# 2  
Old 03-12-2014
Hi.

You have two options, either use vsw's for primary domain or use the physical interfaces under the vsw (net0 etc.)

1. Use the physical interfaces directly on control domain (net0 etc.)
create ip on then create ipmp group, add net0/net1 to ipmp. # this is how i do it.
2. Use vsw's, using add-vnet commands

Code:
 ldm add-vnet <ifname> primary-sw0 primary

You should see the new interface now under dladm show-phys. Do it for both vsw's (sw0,sw1) and add those new interfaces to ipmp group on control domain.
# 3  
Old 03-12-2014
Hello Peasant,

thanks for your reply !

I was on the way for the second solution and tried it :
Code:
root@XXXXXX:~# dladm show-phys
LINK              MEDIA                STATE      SPEED  DUPLEX    DEVICE
net5              Ethernet             up         1000   full      igb1
net0              Ethernet             up         1000   full      ixgbe0
net4              Ethernet             up         1000   full      igb0
net2              Ethernet             up         1000   full      ixgbe2
net6              Ethernet             up         1000   full      igb2
net7              Ethernet             unknown    0      unknown   igb3
net1              Ethernet             up         1000   full      ixgbe1
net3              Ethernet             unknown    0      unknown   ixgbe3
net8              Ethernet             up         10     full      usbecm2
net10             Ethernet             up         1000   full      vsw0
net9              Ethernet             up         1000   full      vsw1
net12             Ethernet             up         1000   full      vsw2
net11             Ethernet             up         1000   full      vsw3
root@OGFUDEV01:~# ipadm
NAME              CLASS/TYPE STATE        UNDER      ADDR
ipmp0             ipmp       ok           --         --
   ipmp0/v4       static     ok           --         108.1.2.224/24
ipmp1             ipmp       down         --         --
ipmp2             ipmp       down         --         --
lo0               loopback   ok           --         --
   lo0/v4         static     ok           --         127.0.0.1/8
net0              ip         ok           ipmp0      --
net1              ip         ok           ipmp1      --
net2              ip         ok           ipmp2      --
net4              ip         ok           ipmp0      --
net5              ip         ok           ipmp1      --
net6              ip         ok           ipmp2      --
net8              ip         ok           --         --
   net8/v4        static     ok           --         169.254.182.77/24
net9              ip         down         --         --
net10             ip         down         --         --
root@XXXXXXXX:~# ipadm create-ipmp -i net9 -i net10 ipmp5
root@XXXXXXXX:~# ipadm
NAME              CLASS/TYPE STATE        UNDER      ADDR
ipmp0             ipmp       ok           --         --
   ipmp0/v4       static     ok           --         108.1.2.224/24
ipmp1             ipmp       down         --         --
ipmp2             ipmp       down         --         --
ipmp5             ipmp       down         --         --
lo0               loopback   ok           --         --
   lo0/v4         static     ok           --         127.0.0.1/8
net0              ip         ok           ipmp0      --
net1              ip         ok           ipmp1      --
net2              ip         ok           ipmp2      --
net4              ip         ok           ipmp0      --
net5              ip         ok           ipmp1      --
net6              ip         ok           ipmp2      --
net8              ip         ok           --         --
   net8/v4        static     ok           --         169.254.182.77/24
net9              ip         ok           ipmp5      --
net10             ip         ok           ipmp5      --

root@XXXXXX:~# ipmpstat -g
GROUP       GROUPNAME   STATE     FDT       INTERFACES
ipmp5       ipmp5       ok        --        net10 net9
ipmp2       ipmp2       ok        --        net6 net2
ipmp1       ipmp1       ok        --        net5 net1
ipmp0       ipmp0       ok        --        net0 (net4)

And then ??? how can I provide IPMP network to my guest LDOM ?
Last edited by DukeNuke2; 03-12-2014 at 06:37 AM..
# 4  
Old 03-13-2014
I was thinking you want to use ipmp on control domain (primary).

If you want to use IPMP inside guest domain, you will need to add one interfaces from each of the VSW's and make ipmp inside guest ldom.

You cannot assign created ipmp interfaces on control domain to guest domains.

So
Code:
ldm add-vnet vnet0 primary-sw0 <guest ldom> # ( interface name vnet can be named as you like)
ldm add-vnet vnet1 primary-sw1 <guest ldom>

Login to guest ldom and configure ipmp from two interfaces you just added
# 5  
Old 03-13-2014
regarding the Oracle documentation (Oracle VM Server for SPARC 3.1 Administration Guide, Chapter 8 / Configuring and Using IPMP in the Service Domain), it should be possible to define and IPMP group over vswitches in the service domain so that guest LDOM only see one interface. Then you configure IPMP once for all the LDOMs ...

Sorry for not being clear
# 6  
Old 03-14-2014
I think you misunderstood the document.

You can use IPMP in service domain (control domain, hypervisor) in two ways i described earlier.

The document states the third option (which i haven't tried but i see no reason not to work), is to use vsw interfaces on control/service domain to make an ipmp group.

So you have 8 physical interfaces (net0 to net7), consisting of two cards with 4 ports each with their drivers (ixgbe, igb).

You use those interfaces to create VSW (one per interface, so 4 total VSW, 4 are still unused and 2 of those unused are not connected with cable).
I see you have chosen to configure 4 VSWs with 2 ports from each card (hopefully each card is connected to different physical switch)

This creates additional interfaces which you can see under show-phys (DEVICE vsw).

After that for control/service/primary domain you can :

1. Create an ipmp group on control domain consisting of two or more physical interfaces, preferably from different cards (DEVICE ixgbe, igb under show phys)

2. Create an ipmp group on control domain consisting of two or more vsw net interfaces preferably from different cards (DEVICE vsw under show phys)

3. Use ldm commands to add vnet's to primary domain (control/service domain) and use those interfaces (DEVICE vnet under show phys after adding via ldm) to create ipmp group.

For guest ldoms, you will need to add 2 or more vnet's from each VSW and configure ipmp inside ldom (same as third option above, but for guest ldoms).

Also, do you tend to use different VLAN's inside you guest ldoms now or in the future ?
If you even suspect you will use it, configure it now, since it will be a hussle later.

Hope that clears things out.

Regards
Peasant.

---------- Post updated at 08:29 ---------- Previous update was at 08:12 ----------

Just couple of more hints regarding VM.

For VDS, use one VDS - one guest LDOM, don't put everything in primary-vds.

Disable extended-mappin-space everywhere since i noticed sometimes live migration fails with this on.

You might want to disable INTER-VNET-LINK (to off)
I had a situation when network between two guests, that reside same physical machine on the same subnet, just stops working.

I have ran some internal test with those two options on/off (which should improve network throughput), and didn't notice any performance gains, only experienced issue above Smilie

Depends on which patchset you are, perhaps those issues are now fixed.

Regards
Peasant.
# 7  
Old 03-14-2014
Dear Peasant, thanks for being so patient !!

I think I understood most of what you said ! To summarize, there is no way to do what I would like to do ! Actually, with Oracle VM for Sparc, you can't do like with vmware : create a vswitch with multiple physical NICs and present a single load-balanced/redundant virtual NIC to your VMs/LDOMs ... am I right ?

Thanks also for your tips !

Regards
Login or Register to Ask a Question

Previous Thread | Next Thread

6 More Discussions You Might Find Interesting

1. Solaris

When do you need a separate service/IO domain?

Looking at latest recommendations - http://www.oracle.com/technetwork/server-storage/vm/ovmsparc-best-practices-2334546.pdf - specifically regarding domain roles. At the moment, we just have a physical host, primary control domain and then guest ldoms. We then export things like vdisks,vnet etc... (2 Replies)
Discussion started by: psychocandy
2 Replies

2. Solaris

Oracle VM for SPARC - control/io domain dilemna

Hi all, My internal SAS disks (4 of them) are all sitting on the same controller/pcie device which is now being own by my default primary aka io aka control domain. I have created a vdisk server, that serves slices on these 4 disks to guest domain and everything is working fine. The issue... (2 Replies)
Discussion started by: javanoob
2 Replies

3. Solaris

SunOS sun4v sparc ntp service in maintenance mode.

Hi experts, This is a production server. Host information's are below SunOS hostname_srv 5.10 Generic_150400-09 sun4v sparc sun4v Now issue with ntp service, This host have zone in it with 9 hosts, Every hosts have ntp service issue. While i check for the service status it's in... (3 Replies)
Discussion started by: babinlonston
3 Replies

4. Solaris

Can we configure link based IPMP in private connectivity in Oracle RAC

Hi I would like to know whether we can configure link based IPMP in private connectivity in Oracle RAC Regarsd ---------- Post updated at 04:35 PM ---------- Previous update was at 04:27 PM ---------- Here I am taking about in case of private connectivity through cross cable (6 Replies)
Discussion started by: sb200
6 Replies

5. Solaris

Service error on sparc server running solaris 10

Hi, I am getting following service error on one of the sparc servers running solaris 10 - Code : $ svcs -a | grep "maintenance" maintenance Nov_08 svc:/application/management/sma:default $ svcs -xv svc:/application/management/sma:default (net-snmp SNMP daemon) State: maintenance... (8 Replies)
Discussion started by: sunadmin
8 Replies

6. Linux

Oracle Linux on SPARC

Hi Oracle Linux users, You can probably guess from the title what the question is: Does anyone know if Oracle Linux (the Unbreakable variety I think that is) comes in a SPARC release or, if not, will there be one some time soon ? Many thanks, P;):D:b: (2 Replies)
Discussion started by: patcom
2 Replies
Login or Register to Ask a Question