Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Shell/Perl logic for loop

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell/Perl logic for loop
# 1  
Old 05-11-2012
Shell/Perl logic for loop
Hi,
I have a requirement as follows. Have 3 files. Need to match up the data in each one of them and sum up the data by a field and display it. example given below.

File 1 : Name, Emp id
File 2 : Empid, Subject,
File 3 : Subject, Score, Class

Match Emp id in File 1 and File 2 and then match up Subject from File 2 and File 3 and print the results.

O/p 1 - Name, Emp id, Subject , Score, Class.

Now group the details by the Empid and print the data. One added feature needed is the score can be negative. So if we could print the +ve and -ve seperately at Emp id is betetr.

Currently I am using a combination of Shell, Database join and macro to do this wrork. Want to eliminate the DB and macro. Please assist.

PS: This is NOT an assignment.

Thanks and Regards,
Gideon.
# 2  
Old 05-11-2012
Are the entries are unique in each file?
# 3  
Old 05-11-2012
Yes. I clean up the the dupes before this step.

But point is that multiple guys will have the same subject.

Thanks and Regards,
Gideon.

---------- Post updated at 07:19 PM ---------- Previous update was at 07:14 PM ----------

I put a piece of code like this. But it's taking pretty long...perf wise the DB seems to be much faster. Need inputs.

Code:
rm -f /tmp/Cont.csv
for line in `cat /tmp/File1.csv`
do
    curr=`echo $line | cut -f2 -d "|"`
    rate=`grep $curr /tmp/File2 | cut -f1,2 -d "|"`
    if [ "$rate" != "" ]
    then
           echo $line "|" $rate >> /tmp/Cont.csv
    fi
done

---------- Post updated at 07:20 PM ---------- Previous update was at 07:19 PM ----------

Also I would need to repeat the same cycle for abother file as well which would taken eternity to complete.

Thanks and Regards,
Gideon.
Last edited by Scrutinizer; 05-11-2012 at 09:53 AM.. Reason: code tags
# 4  
Old 05-11-2012
You could give this a try:
Code:
awk -F, 'FNR==1{f++} f==1{E[$2]=$1} f==2{S[$1]=$0} f==3{print E[$1],$1,S[$2]}' OFS=, file1 file3 file2

These 2 Users Gave Thanks to Scrutinizer For This Post:
clx preethgideon
# 5  
Old 05-11-2012
Are you looking for something like this?

Code:
$ awk -F, 'BEGIN {while (( getline < "2" ) > 0 ){a[$1]=$2}} NR==FNR {b[$1]=$2FS$3;next} {print $1,$2,a[$2],b[a[$2]]}' OFS=, 3 1
name1,11,sub1,sc1,c1
name2,22,sub1,sc1,c1
name3,33,sub3,sc3,c3

Code:
$ cat 1
name1,11
name2,22
name3,33
$ cat 2
11,sub1
22,sub1
33,sub3
$ cat 3
sub1,sc1,c1
sub2,sc2,c2
sub3,sc3,c3

This User Gave Thanks to clx For This Post:
preethgideon
# 6  
Old 05-11-2012
Thanks Anchal and Scrutinizer. I tried to modify your code, It's throwing some bad rows. If you don't mind can you please modify the row according to this requirement?


File 1

Code:
900 |PINE  |00000003

----------------------------------------------------------------------
File 2

Code:
C|00000003|AAAAAAA.1
C|00000004|BBBBBBB.1

----------------------------------------------------------------------

File3 - /tmp/ContractsDetails.txt
Code:
123|AAAAAAA.1|XYZ|123.56|Yes
123|BBBBBBB.1|ABC|456.67|NO
333|BCDCDC.2|DBC|456.67|YES

Last edited by Scrutinizer; 05-11-2012 at 11:32 AM.. Reason: code tags
# 7  
Old 05-11-2012
The field separator differs from your original samples, and the number of field differs from your original samples. We do not know on what fields you want the joining to occur and we do not know what the final output should look like (and you have not use code tags)
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell or Perl Loop Screenshot URLs

I am trying to capture screenshots from a huge list of URLs. I am able to manually capture images of individual pages; that is, I simply run the following command to get a screenshot of Foo.com $ python /path/to/screencapture.sh http://www.foo.com I want to modify the script so that instead of... (2 Replies)
Discussion started by: chipperuga
2 Replies

2. Shell Programming and Scripting

while loop logic

Hi, Here I am trying to query database and check a value, if the value not matches then I wants to re-query the database.Once the value matches, I want to email the reqidstatus_log.txt file. Database query produces a file reqidstatus_log.txt which contains result. But the query not working as... (3 Replies)
Discussion started by: rajsp217
3 Replies

3. UNIX for Dummies Questions & Answers

if then else logic with while loop problem

Hi Friends, I have to do write a shell file based on one flag.If that flag value is 'N' then process look in $DATA are and the normal process continue.If vaule is 'P' then it check for the files in different location $CONV and move those file in $DATA area and rest of the process... (2 Replies)
Discussion started by: Param0073
2 Replies

4. Shell Programming and Scripting

Perl - pass shell-vars into perl for input loop

I need to process a file line-by-line using some value from a shell variable Something like:perl -p -e 's/$shell_srch/$shell_replace/g' input.txt I can't make the '-s' work in the '-p' or '-n' input loop (or couldn't find a syntaxis.) I have searched and found... (4 Replies)
Discussion started by: alex_5161
4 Replies

5. Shell Programming and Scripting

loop logic inside of an inline redirect?

i need to log the feedback from the ftp server as i'm performing some deletes. the only way i know of to do this is with the inline redirect << EOF ... but from there to the closing EOF, it's like i'm at the ftp command prompt, so I don't know how to have ksh script logic in there I have an... (3 Replies)
Discussion started by: tlavoie
3 Replies

6. Shell Programming and Scripting

for loop logic with multiple parameters

hi, unix wizards, i have a question about the logic of my inner for loop below. first, what i am trying to do is to write a script called create_account that automatically creates mysql accounts. the user can provide a user_name or a group_id as an argument (and the script can take multiple... (1 Reply)
Discussion started by: ankimo
1 Replies

7. Shell Programming and Scripting

While Loop Logic

I would need to with making while loop logic working in shell program when I am new into the shell programing 1) I would need to try to get the file from the remote side ----need to try 15 mins apart for 4 times and terminate the program if file is not available.... I would need to know how I... (4 Replies)
Discussion started by: sambakamba
4 Replies

8. Shell Programming and Scripting

Shell or PERL Logic

Hello All, Currently i am working on a logic in PERL scripting ... But i am sort of stuck up, can any one please help. Here goes. 1. Search for a pattern in a file 2. If the pattern matched lets say 10 lines 2.1 Reterive the first line and check for another pattern 2.1.1 if... (1 Reply)
Discussion started by: maxmave
1 Replies
Login or Register to Ask a Question