Sponsored Content
Full Discussion: iSCSI issue on RHEL 5
Operating Systems Linux Red Hat iSCSI issue on RHEL 5 Post 302667433 by zer0sig on Friday 6th of July 2012 09:24:09 AM
Old 07-06-2012
I've found that RHEL gets bound up pretty easily when the iSCSI/FCOE storage requests don't come back as expected.
I was building a small CAD cluster last year, and the build was fairly heavily automated (PXE boot to a kickstart minimal image to HP-SAS for provisioning (we were also automating loading/patching/etc through it, which was dandy..when it worked).. these servers were each using a volume group of ~20 SAN-allocated disks, and there were scripts to initialize, test and lay down a raw Oracle filesystem.

Well, these machines were running dual 10Gb NICS to eachother, and half of the volume group was allocated to each NIC. So the series of scripts, in between the base OS and Oracle installs runs the init disk routine. These machines have 16 cores, and the CPU utilization goes from almost nothing to 100% and the build is taking longer than expected. Noticing that the machine was not responding right, I start going through the logs. I notice one core is unresponsive and the kernel is throwing panic messages.

Turns out, one of the 10Gb cards was actually faulty, but with the 6 or so different links and the other 15 cores running normally, this one was stuck trying to execute a temporary script, and all it was doing was sending requests via iSCSI and not getting them back. The kernel was so caught up with this that it brought the entire machine to a crawl, as though it were in a race condition. kill the script and all is hunky-dory except for the build, which of course failed. Once the NOC team got to check on the links, they found the problem, replaced the card, and a little manual scrubbing of the now-complete volume group and the rest went smoothly.

I haven't seen how other PC-based Unix-style OSes fare under these conditions, but I can't help but think that there should be some failsafes to keep one core running one simple script from turning the machine into a $40K paperweight. I assume that the priority given to the high-end storage subsystems is likely to improve overall performance - as long as everything is working.
 

10 More Discussions You Might Find Interesting

1. Red Hat

RHEL Installation issue.

Hi Gurus I am trying to install RHEL linux enterprise Edition v5 on a Dell Optiplex 320 desktop. After the Grub menu,i get the error,Kernel could not be loaded. I am a newbie in linux and would appreciate any help in moving forward. Thanks HG (1 Reply)
Discussion started by: Hari_Ganesh
1 Replies

2. Red Hat

Time Sync issue on RHEL 5.3

Hi, My Linux VMs are running on citrix XEN.Somehow the time drift is too high (going very fast). Even if the correct time configured using date -s command, for every 1 hr it is differing almost 5 - 10 mins. Tried configuring ntpd (local NTP server) and still it is not synchronized. Machine... (2 Replies)
Discussion started by: uday123
2 Replies

3. UNIX for Dummies Questions & Answers

How to change ethernet cable of iscsi initiator (iscsi client)?

how to change ethernet cable of iscsi initiator (iscsi client) (0 Replies)
Discussion started by: pankajd
0 Replies

4. Red Hat

Does RHEL 5 provide a command to collect RHEL system log in single compress file?

Hi, I heard a command that can collect all RHEL 5 log in a single compress file before I forget. Does any body know...What the command is ? Thanks. (4 Replies)
Discussion started by: nnnnnnine
4 Replies

5. Red Hat

Error throwing while installing vsftpd package in rhel 6. using rhel 6 dvd.

Hi all, Im studying rhcsa as of now, so yum installation and dependencies are messing me to not workit out. i have dual os, win 7 & rhel 6. i have tried this installation of vsftpd package with rhel 6 dvd in VM rhel 6 in win 7 as well as host rhel 6.still the same issue. below error... (6 Replies)
Discussion started by: redhatlbug
6 Replies

6. Red Hat

Logrotate issue in RHEL

Hi, I have a requirement to compress logs that generate daily. Below is the logrotate configuration for this But for some kind of reason the logs are not getting compressed daily. When tried to execute the logrotate manually with -d it says "log does not need rotating." The path... (0 Replies)
Discussion started by: EmbedUX
0 Replies

7. Linux

Installation issue RHEL 6.5 on SAN

HI All I am facing issue in installation of Red Hat 6.5 on SAN.. i am having a partion of 2 TB in my SAN and i am installing the operating system after completed the installation it not booting and giving the error. (2 Replies)
Discussion started by: deviltech
2 Replies

8. Red Hat

RHEL Memory issue

Friends, I have 8CPU/24gb RAM in my Linux servers around 30+. All the servers are running with Java based application with NFS mounted. The problem what I have is, When I reboot servers the server memory looks decent. But after some time the "cached" memory increasing and "free" memory... (5 Replies)
Discussion started by: baluchen
5 Replies

9. Shell Programming and Scripting

AIX to RHEL migration - awk treating 0e[0-9]+ as 0 instead of string issue

Greetings Experts, We are migrating from AIX to RHEL Linux. I have created a script to verify and report the NULLs and SPACEs in the key columns and duplicates on key combination of "|" delimited set of big files. Following is the code that was successfully running in AIX. awk -F "|" 'BEGIN {... (5 Replies)
Discussion started by: chill3chee
5 Replies

10. UNIX for Advanced & Expert Users

RHEL 5.11 - gid issue

Hello folks, I am facing a weird issue on one of the red hat linux 5 box. There is an oracle id which i want to change the primary group. Currently the primary group id is 16600, I did changed it to 16601 using below command: usermod -g 16601 oracle But it did not changed. I have tried... (3 Replies)
Discussion started by: snchaudhari2
3 Replies
ISCSID(8)						      System Manager's Manual							 ISCSID(8)

NAME
iscsid - establish iSCSI connections SYNOPSIS
iscsid [ -b bindingfile ] [ -d ] [ -f configfile ] [ -l basedir ] [ -m mode ] [ -n ] DESCRIPTION
iscsid establishes connections with iSCSI targets defined in /etc/iscsi.conf. Once the Linux iSCSI driver is activated, a discovery process for iSCSI storage devices will proceed as follows: - The iSCSI daemon requests available iSCSI targets from the iSCSI target, and passes the information discovered to the iSCSI kernel module. - The iSCSI kernel module establishes connections to the targets. - Linux queries targets for device information. - Linux creates a mapping from SCSI device nodes to iSCSI targets. iscsid should be started after networking is configured and stopped after all iSCSI devices have been unmounted. Warning: Data corruption can occur if you do not unmount iSCSI devices before disabling network interfaces! DEVICE NAMES
Because Linux assigns SCSI device nodes dynamically whenever a SCSI logical unit is detected, the mapping from device nodes (e.g /dev/sda, /dev/sdb) to iSCSI targets and logical units may vary. Variations in process scheduling and network delay may result in iSCSI targets being mapped to different SCSI device nodes every time the driver is started. Because of this variability, configuring applications or operating system utilities to use the standard SCSI device nodes to access iSCSI devices may result in SCSI commands being sent to the wrong target or logical unit. To provide a more reliable namespace, the iSCSI driver will scan the system to determine the mapping from SCSI device nodes to iSCSI tar- gets, and then create a tree of directories and symbolic links under /dev/iscsi to make it easier to use a particular iSCSI target's logi- cal units. TARGET BINDINGS
The iSCSI driver automatically maintains a bindings file /var/iscsi/bindings. This file contains persistent bindings to ensure that the same iSCSI bus and target id number are used for every iSCSI session to a particular iSCSI TargetName, no matter how many times the driver is restarted. This feature ensures that the SCSI numbers in the device symlinks described above will always map to the same iSCSI target. Note that because of the way Linux dynamically allocates SCSI device nodes as SCSI devices are found, the driver does not and can not ensure that any particular SCSI device node (e.g. /dev/sda) will always map to the same iSCSI TargetName. The symlinks described in the section on Device Names are intended to provide a persistent device mapping for use by applications and fstab files, and should be used instead of direct references to particular SCSI device nodes. If the bindings file grows too large, lines for targets that no longer exist may be manually removed by editing the file. Manual editing should not normally be needed, since the driver can maintain up to 65535 different bindings. OPTIONS
-b bindingfile Specify an alternative bindings file instead of /var/iscsi/bindings, which is the default. -d Turns on debug mode. Each occurence of -d will increment the debug level by one. The default is zero (off). -f configfile Specify an alternative configuration file instead of /etc/iscsi.conf, which is the default. -l basedir Specify the base directory under which to build a tree of directories containing symlinks to SCSI device nodes, in a manner similar to the devfs Linux kernel option. Using these symlinks hides variations in the mapping from SCSI device nodes to SCSI device id numbers. -m mode Specify the directory permission mode (in octal) to use when creating directories. -n Avoid auto-backgrounding. -v Print version and exit. SIGNALS
iscsid reacts to a set of signals. You may easily send a signal to iscsid using the following: kill -SIGNAL `cat /var/run/iscsid.pid` SIGTERM The daemon and all of it's children will die. SIGHUP sent to the main daemon process will restart all discovery processes and reprobe LUNs on all targets. iscsid and all of it's chil- dren will die after shutting down all of the kernel's iSCSI sessions. SIGCHLD Wait for children. NOTES
The iSCSI Driver for Linux provides IP access to a maximum of sixteen remote SCSI targets. Each target will be probed for up to 256 LUNs, until the Linux kernel's limit of SCSI devices has been reached. The iSCSI drivers, README files, and example configuration files are available on the Linux-iSCSI homepage at: http://linux-iscsi.sourceforge.com/ <http://linux-iscsi.sourceforge.com/> FILES
/etc/iscsi.conf target address and LUN configuration /var/run/iscsi.pid the process id of the running daemon /var/iscsi/bindings persistent bus and target id bindings for iSCSI TargetNames /proc/scsi/iscsi information about iSCSI devices /dev/iscsi a directory tree containing symlinks to iSCSI device nodes. SEE ALSO
iscsi.conf(5) $Revision: 1.8 $ $Date: 2002/09/20 19:27:32 $ ISCSID(8)
All times are GMT -4. The time now is 02:15 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy