Shell Programming and Scripting

I typically pull a bunch of data via SQL that lists a bunch of users and the server on which they want to access, as well as other attributes, in one row of an excel sheet and the number of rows is directly proportionate to the number of users.

I'm trying to write a loop to read each line of the excel sheet (exported to a text file), store the column containing the server name into a variable called HOST, and the column containing the username into a variable called HUMAN_ID. Obviously I'm not getting the desired result.

The excel export to text puts all columns separated by quotes. I can get the column I want by doing the following:

Code:

cat txt | awk -F\" {'print $18'}

Ok, that's fine - so now I want to write a loop and store that value into HUMAN_ID for each iteration so it can be run in conjunction with the HOST variable. I'm doing this and adding the "echo $HUMAN_ID" to test that this is working:

Code:

for i in $(cat txt);do HUMAN_ID=`echo $i | awk -F\" {'print $18'}`;done

It appears to be printing all the other cell contents before column 18 at each iteration. The excel sheet may be inserting control characters like a carriage return and I can look into that but I'm puzzled because the basic cat / awk works fine.

I eventually want to write a loop doing:

Connect to the value stored in HOST at the time of iteration, grep the /etc/passwd file for the value stored in HUMAN_ID at the time of the iteration, and append the results to a log file for viewing until the value of HUMAN_ID is null (end of the file).

What am I doing wrong?

That's a useless use of cat and dangerous use of backticks. the for-loop is splitting on spaces, not just lines, which ruins your data. Also, Never use external utilities to process individual lines if you can possibly help it; externals are fast, but also have a fixed cost for loading and quitting time. Running them repeatedly for small quantities of data is like making 9,000 phone calls to say 9,000 words.Shell has a way to do all of these things natively without using awk or cat, which will be hundreds of times faster.

Code:

while IFS="\"" read C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 C16 C17 C18 REST
do
        echo "Column 18 is ${C18}"
done < txt

I'll take a look at the links you provided but I got it to work using the aforementioned methods you advised not to use. Appreciate your suggestion in any event.