AIX

Hi all,

In my system, I have HMC 7 with Power Machine 6 & 7. On the managed system, we have many lpars.
In some lpars, I can see they are using virtual fiber channel to connect to DS8K storage. In search with google, I understand that it is configured with VIOS server to share the physical FC channel with NPIV method.

But the situation is, in the same managed system, there are other lpars have physical FC, the WWpN not starting with C.
So my concern is, what method these lpars use to connect to SAN? How can they can "share/use" the physical FC without VIOS virtual FC?

lpar has virtual FC channel

Code:

[root@xxxxxx] / > lsdev -Cc adapter | grep fcs
fcs0    Available 02-00    8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs1    Available 02-01    8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs2    Available 04-00    8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs3    Available 04-01    8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs4    Available 06-00    8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs5    Available 06-01    8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)

Lpar has "physical" FC channel

Code:

[root@xxxxx] / > lsdev -Cc adapter | grep fcs
fcs0 Available 00-00 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs1 Available 00-01 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs2 Available 01-00 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs3 Available 01-01 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs4 Available 05-00 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs5 Available 05-01 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)

Both lpars below belongs to 1 power machine 6.

Please advise me on this situation.

Moderator's Comments:

edit by bakunin: Please use CODE-tags for data and terminal output as well as code. Thank you. Furthermore i am transferring this thread to the AIX board.

I am not sure if i understand your question correctly. If the folowing answer is not what you expected please clarify what exactly you want to know:

Basically they both use the same way of communicating with the SAN - a "sort-of SCSI"-protocol transported over FC. NPIV basically means that a single (physical) FC port can use several (virtual) IDs - WWPNs - to identify itself. Think of it like an IP-alias for a network interface.

You may want to replace the physical FC-adapters with virtual ones, even if you only configure one LPAR to use it: if you assign physical adapters to an LPAR this partition will never be eligible for LPM (live partition mobility) and hence also not for live update (replace even the kernel without reboot, possible since AIX 7.2). My suggestion is to assign all the physical adapters to the VIOSes and give only virtual adapters to the LPARs. Notice that you will need to change the zoning for this to work.

Every virtual FC adapter (port) has actually TWO WWPNs which should both be zoned. The reason is that for LPM you need one WWPN to hold the connection on the originating side while the other WWPN is configured on the side you migrate to during the transition.

I hope this helps.

bakunin

Hi Bakunin,

Thank for your reply.

Let me clarify again. On the power machine which has physical FCs attached to SAN via SAN switch, as can be seen on my environment, there are 2 LPARs as below:

1. First LPAR with virtual FC (assigned by VIOS) and can use the LUN allocated to it, this point can be understood as it uses NPIV.

2. Second LPAR does not use VIOS and virtual FC, because with the lsdev -Cc adapter | grep fcs as showed, the FC adapter are "physical ones", not virtual as it does not start with C in the WWPN.

Code:

[root@xxxxx] / > lsdev -Cc adapter | grep fcs
fcs0 Available 00-00 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs1 Available 00-01 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs2 Available 01-00 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs3 Available 01-01 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs4 Available 05-00 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs5 Available 05-01 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)

My question is, how can it connects and uses the LUN which is allocated to it? As my thought, it is utilizing also the physical FC port of power machine.

if first LPAR uses NPIV, so what technique does second LPAR use? It can use Power Machine physical FC directly and without need the VIOS and virtual FC?

Hope my explanation is clear.
Phat

hmm.... instead of checking if the WWPN starts with "C" or with something else consult the LPARs profile on the HMC (or IVM, if you have no HMC). You will either see a (or several) physical adapter(s) there (along with the WWPNs) or virtual adapters coming from the VIOS(es). This is a surefire method, checking the WWPNs for starting characters is not.

OK, at least i begin to understand where your confusion comes from: "NPIV" is NOT a type of connection or a protocol at all. NPIV is a way of giving one (physical) adapter several WWPNs at once. A WWPN is a unique identifier and every "initiator" - that is a port of an FC adapter, a FC-switch-port or a LUN - has one. Think of it like a MAC-address for network interfaces. NPIV now allows to give assign several WWPNs to one such port or, respectively, create several virtual (logical) adapters all with different WWPNs which are based on a single physical adapter. So the VIOS gets assigned the physical adapter (which has its own WWPN), then creates several virtual adapters ech with its own WWPN and gives each of these virtual adapters to different LPARs. The LPARs will use the WWPN of the virtual adapter assigned but the traffice is (in reality) routed through the physical adapter. This is why i compared it to an IP-alias: an IP alias is an interface having two or more (instead of one) IP addresses. But this doesn't result in a certain (form of) communication. The communcation will still be "HTTP" or "TCP" or whatever.

OK, let us consider a normal TCP/IP network again. A network is a connection of any to any. That means: each station can communicate with every other station. The other might respond or not, there might be authorisation issues, but the basic means of communication are there. The situation in an FC-network is fundamentally different: there is no "authorisation" as such but only certain connections are allowed and all others are forbidden: if you connect via ssh to a system you are asked for, say, a password. That means the network itself doesn't care about your authorisation because the ssh-connection is made. Only the system there cares about that and either allows or denies you access. In an FC-network nobody is asked for any password (or other forms of authorisation) but instead unauthorised (or rather: unwanted) connections are not even possible from the start. You would not want the disks of server A being visible in server B or vice versa.

Basically you need a set of rules that adapter A is allowed to connect to LUN B using FC-switch-port C. Creating and maintaining such a set of rules is called "zoning" and it is done by configuring the FC-switch (or "switches", because this is usually a redundant, load-balancing and high-available system). Basically a "zone" is a rule that allows a certain system (or, to be precise, its FC-adapters identified by its WWPN) to access a certain LUN over a certain way. You may have several zones for a single LUN allowing i.e. two system to access the LUN for instance in case of a cluster. All this is a bit reminiscent of firewall rules.

The process in detail is rather complex with a large margin for making things very complicated and i am not an expert in that sector - not even by a stretch. I suggest you consult specialised pages if you want to understand details like i.e. "port-based zoning", etc.. But at a very fundamental level this is it what it .

I hope this helps.

bakunin

Hi Bakunin,

Take a look on first lpar, we can see the virtual fiber, it comes from VIOS and routed to physical FC of Power machine.

Code:

[root@xxx] / > lsdev -Cc adapter | grep fcs
fcs0 Available 12-T1 Virtual Fibre Channel Client Adapter
fcs1 Available 12-T1 Virtual Fibre Channel Client Adapter
fcs2 Available 12-T1 Virtual Fibre Channel Client Adapter
fcs3 Available 12-T1 Virtual Fibre Channel Client Adapter

Take a look on second lpar, it's not virtual. I wonder if it is actual physical FC from Power Machine assigned directly to lpar.

Code:

[root@zzz] / > lsdev -Cc adapter | grep fcs
fcs0 Available 00-00 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs1 Available 00-01 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs2 Available 01-00 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs3 Available 01-01 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs4 Available 05-00 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs5 Available 05-01 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)

--- Post updated at 09:27 AM ---

I wonder where those physical FC come from? As virtual FC comes from VIOS.

Moderator's Comments:

edit by bakunin: see my post below, please use CODE-tags. Thank you.

Yes, correct.

I have no idea, but let me repeat:

What a LPAR consists of (CPUs, memory, physical adapters, virtual adapters, ...) is stated in the LPARs profile. You can view/modify it either via the graphical HMC web-interface or via a commandline login on the HMC using the [icode]lssyscfg[icode] command. Enter help lssyscfg if you are unsure how to do that.

I cannot tell you how these adapters got into the system because i do not administrate your systems. So just look it up at the source (see above), everything else is just conjecture.

Finally: please use CODE-tags for terminal output too. I have edited them in for you now, but please use them yourself. It makes text-based output (file contents, terminal output and code) more readable. Thank you.

I hope this helps.

bakunin

In the HMC profile, I can see there are 3 physical FC assigned to the server as attached.

I also see 6 FC ports which is equivalent from 3x2=6

Code:

[root@zzz] / > lsdev -Cc adapter | grep fcs
fcs0 Available 00-00 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs1 Available 00-01 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs2 Available 01-00 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs3 Available 01-01 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs4 Available 05-00 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
fcs5 Available 05-01 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)

Can we say the "physical" FC adapter here are actually the real physical FC adapter from Power Machine which are assigned to the lpar not via VIOS (with virtual FC adapter) ?