Shell Programming and Scripting

Wow man thanks!

Storm was way worse than anyone thought it was going to be, killed my home net connection so I could not get back to this until now.

Thank you so much for the script, and it would work fine if their files were consistent, but upon further examination I have now realized they are not.

The DBA's here need to be kicked in the head, or maybe they already have been, that would explain a lot...

ANYWAY

File2's format is the most consistent so I am going to work off of it, just like in the script you have written. Since I can't assume the 3rd field will be an email, since there are some middle names, some have two last names, some have no spaces, some are Jr's, the list goes on and on... I have no intention of trying to write a condition for all of those situations.

So here is my idea:
1. I am going to try to script it to search for an email in each line of File2 (ie ":*@*:" or something like that).

2. Once I've got an email I will search in the other file for a line that has that email somewhere in it.

3. Once I have that line I will get the 7 digit number (which is the ID #) out of it.

4. Then I will also search this same line for the email that ends in the specified string.

5. Then take those two items and add the ID number to the front of the line from file2 and the email ending in the specified string to the end of it and write it back out.

I think this is the best way (and maybe only way) to do it....

I apologize for not realizing that ht was inconsistent before, the DBA said it was and I took their word for it (my mistake there).

Thanks again for trying, I think I have learned what I needed to know from looking at your script, to get this done.

I'll post what I come up with in case it may help someone else in the future.

OK I was wrong I don't know all I need to know to do this...

I can use this:

Code:

while read F2Line
   do
         .....
            search var F2line for the string that is the email
         .....

done<${FILE2}

to look at file two line by line. At that point I need to search that line (stored in variable name F2Line) which will be an undetermined bunch of strings separated by :'s iin one line for the string containing the email address.

How do I do that?

Just line I did, show some of the data in an irregular format for the two files.
Something like the following will make it easier to understand:

Code:

> cat file151
bob:johnson:3343:bjohn@email_SB.org,bjohn@emailARG.org,3343@email.org
sol:admin:3344:sadmin@email_SB.org,sadmin@emailARG.org,3344@email.org
joe:sample:3345:jsample@email_SB.org,jsample@emailARG.org,3345@email.org
> cat file152
bob:johnson:bjohn@email_SB.org
sol:admin:3344@email.org
joe:sample:jsample@email_SB.org

OK I will try to give some good examples here. With the occasional extra crap bolded.

file151:
bob:johnson:3343:bjohn@email_SB.org,bjohn@emailARG.org,3343@email.org
sol:admin:3344:sadmin@email_SB.org,sadmin@emailARG.org,3344@email.org
joe:sample:JR:3345:jsample@email_SB.org,jsample@emailARG.org,3345@email.org
john:De:Salva:3346:jds@email.com,jDesalva@emailRAG.edu,jSalva@email.com

Now for file152 what you have there is correct but sometimes you get an extra name field so I will add a few for example:
bob:johnson:bjohn@email_SB.org
sol:admin:3344@email.org
joe:sample:jsample@email_SB.org
bill:jones:jr:bjones@email.edu
bill:jones:john:jr:sampson:bjs@emailARG.edu

Things that are safe to assume:
file152:
This will have an email (and only one email) in each line.
This is the email I will need to trim out and use to get the corresponding line out of file151 to work with...

file151:
This will have at least one line with an email in it that will match the email from the file152.
Each line in this file will have a unique 4 digit number that is the Id number somewhere in it. (I need to get that out and into a variable)
Also somewhere in each line of this file there will be an email ending in "something@ARG.edu" (I need to get that out too)

SO once I get the ID and the something@ARG.edu email out of each line in file 151 I will write the line back out to a new file like this:

IDfrom151:entire line from file 152:something@ARG.edu email from 151

follow me? Or do I need to give more examples lines?

Where the other script fails is in the cut commands, they assume field 3 is email when in fact field 4 could be email, if there is a Jr. in the mix. Or field 5 if there is a Jr. and 2 middle names...

Something I was thinking about was to insert another character to help delimit the first file. It is not all done, and doesn't yet accomplish everything. But, while I had a few spare moments, thought I'd pass this along as a possible approach.
Experiment with that sed command and see what it does to you input file, and then it will be clear how you can cut with the additional delimiter of ~.
Unsure yet if the same type of approach can be done with the 2nd file.


If my first file now looks like:

Code:

> cat file151a
bob:johnson:3343:bjohn@email_SB.org,bjohn@emailARG.org,3343@email.org
sol:admin:3344:sadmin@email_SB.org,sadmin@emailARG.org,3344@email.org
joe:sample:JR:3345:jsample@email_SB.org,jsample@emailARG.org,3345@email.org
john:De:Salva:3346:jds@email.com,jDesalva@emailRAG.edu,jSalva@email.com

Then I can use sed to place a ~ before the first number, and that can help me grab data.
See code below:

Code:

> cat make153a.sh
#! /usr/bin/bash

FILE1="file151a"      #1 input file
FILE1x="file151ax"    #1 input file fixed
FILE2="file152a"      #2 input file
FILE2x="file152ax"    #2 input file fixed
FILE3="file153a"      #output file

MATCH_EM="@emailARG.org"

sed "s/:[0-9]/~&/" ${FILE1} >${FILE1x}

savifs=$IFS
IFS=":"

rm ${FILE3} 2>/dev/null   #delete the file, if it exists

while read FNAME LNAME EMAIL
   do
   EMAIL1=`grep "${EMAIL}" ${FILE1x} | cut -d"~" -f2 | cut -d":" -f3`
   EMAIL2=`echo ${EMAIL1} | tr "," "\n" | grep "${MATCH_EM}" `
   ID=`grep "${EMAIL}" ${FILE1x} | cut -d"~" -f2 | cut -d":" -f2`

   echo "${ID}:${FNAME}:${LNAME}:${EMAIL}:${EMAIL2}"
#   echo "${ID}:${FNAME}:${LNAME}:${EMAIL}:${EMAIL2}" >>${FILE3}
done<${FILE2}

I follow you. Looking at this I now see how you are using the "tr" command, that will help out a lot too..

I have some ideas I am trying too, let me get something together and I will post it and see what you think.

This may be a different direction than you are going but can you help me out here:

Code:

#!/bin/bash

var="bob:johnson:3454:jb@email.edu:test:fu.org::"
echo $var
found=`echo ${var} | tr ":" "\n" | grep ####`
echo $found

I am trying to get 3454 onto the found var. Am I close?