Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

shell loop and awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting shell loop and awk
# 1  
Old 10-17-2012
shell loop and awk
Hello, everyone!

I merge two files using awk like following, and there are two files involved in this command.

Code:
awk 'NR==FNR{A[$1]=$0; next} $1=A[$1]' file1 FS=, OFS=',' file2 > tempfile.txt

Further, I need to deal with hundreds of files at the same time.

I want to make a loop like this:
Code:
#!/bin/bash
FILES=*
for f in $FILES
do
   awk    
done

But I don't know how to deal with files in order, cause I need to put the file name instead of file2.
# 2  
Old 10-17-2012
For a tmp file for each file:

Code:
for fl in *
do
   awk 'NR==FNR{A[$1]=$0; next} $1=A[$1]' file1 FS=, OFS=',' "$fl" > "$fl".tmp
done

# 3  
Old 10-17-2012
Quote:
Originally Posted by rdrtx1
For a tmp file for each file:

Code:
for fl in *
do
   awk 'NR==FNR{A[$1]=$0; next} $1=A[$1]' file1 FS=, OFS=',' "$fl" > "$fl".tmp
done

Sorry, I didn't express my need clearly.

What I want to do is as follows:

Code:
awk 'NR==FNR{A[$1]=$0; next} $1=A[$1]' file1 FS=, OFS=',' file2 > file2.tmp

Code:
awk 'NR==FNR{A[$1]=$0; next} $1=A[$1]' file2.tmp FS=, OFS=',' file3 > file3.tmp

Code:
awk 'NR==FNR{A[$1]=$0; next} $1=A[$1]' file3.tmp FS=, OFS=',' file4 > file4.tmp

.
.
.
.
.
# 4  
Old 10-17-2012
What results do you get with the last post? file1 should reside in another directory as file#, or use /dir/file1 and /dir/file#
# 5  
Old 10-17-2012
Quote:
Originally Posted by rdrtx1
What results do you get with the last post? file1 should reside in another directory as file#, or use /dir/file1 and /dir/file#
I will get records in all these files with the same string in $1 of each file.
# 6  
Old 10-17-2012
something like,

Code:
for f in *; do
  [ -n "$last" ] && awk '...code...' "$f" "$prev" > "$f".tmp
  prev=${f}.tmp
done

edit: using bash can replace [ -n "$last" ] with [[ $last ]]
# 7  
Old 10-17-2012
Quote:
Originally Posted by neutronscott
something like,

Code:
for f in *; do
  [ -n "$last" ] && awk '...code...' "$f" "$prev" > "$f".tmp
  prev=${f}.tmp
done

edit: using bash can replace [ -n "$last" ] with [[ $last ]]
It seems doesn't work. The code as follows:

Code:
#!/bin/bash
for f in *;
  do
    echo $f
    [ -n "$last"] && awk -F, 'NR==FNR{A[$1]=$0; next} $1=A[$1]' "$f" FS=, OFS=, "$prev" > "$f".tmp
  prev=${f}.tmp
done
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk for loop help

Hello, I have an input file that looks like so: 1 2 3 4 5 6 7 8 9 and I just want to print the first and third column (note: my actual file contains many many more fields so I don't want to use '{ print $NF }' for each field I want. I tried using: awk 'BEGIN {FS=" "} { for (i=1;... (13 Replies)
Discussion started by: Rabu
13 Replies

2. Shell Programming and Scripting

awk programming -Passing variable to awk for loop

Hi All, I am new to AWK programming. I have the following for loop in my awk program. cat printhtml.awk: BEGIN -------- <some code here> END{ ----------<some code here> for(N=0; N<H; N++) { for(M=5; M<D; M++) print "\t" D ""; } ----- } ... (2 Replies)
Discussion started by: ctrld
2 Replies

3. UNIX for Dummies Questions & Answers

Basic loop awk/shell script question..

Hi, Sorry if this is a newbie question. I guess you can use either awk or shell script for this sequence of operations, but knowing very little about either of them I'm not sure how I should try to write this. The basic objective is to copy certain files that are scattered all over my... (10 Replies)
Discussion started by: pc2001
10 Replies

4. Shell Programming and Scripting

awk loop using array:wish to store array values from loop for use outside loop

Here's my code: awk -F '' 'NR==FNR { if (/time/ && $5>10) A=$2" "$3":"$4":"($5-01) else if (/time/ && $5<01) A=$2" "$3":"$4-01":"(59-$5) else if (/time/ && $5<=10) A=$2" "$3":"$4":0"($5-01) else if (/close/) { B=0 n1=n2; ... (2 Replies)
Discussion started by: klane
2 Replies

5. Shell Programming and Scripting

If else condition inside for loop of awk command in UNIX shell scripting

Hi , Please excuse me for opening a new thread i am unable to find out the syntax error in my if else condition inside for loop in awk command , my actual aim is to print formatted html td tag when if condition (True) having string as "failed", could anyone please advise what is the right... (2 Replies)
Discussion started by: karthikram
2 Replies

6. UNIX for Dummies Questions & Answers

Print each output of loop in new column using awk or shell

I have this output from a loop a11 1,2 3,4 5,6 7,8 12,8 5,4 3,6 a12 10,11 12,13 15,18 20,22 a13 ... (3 Replies)
Discussion started by: maryre89
3 Replies

7. Shell Programming and Scripting

awk loop and using shell in awk

Hi, everyone! I have a file, when I print its $1 out it show several strings like this: AABBCC AEFJKLFG FALEF FAIWEHF What I want to do is that, after output of each record, search the string in all files in the same folder, print out the record and file name. This is what I want... (4 Replies)
Discussion started by: xshang
4 Replies

8. Shell Programming and Scripting

Comparison and editing of files using awk.(And also a possible bug in awk for loop?)

I have two files which I would like to compare and then manipulate in a way. File1: pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2: pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2... (1 Reply)
Discussion started by: linuxkid
1 Replies

9. Shell Programming and Scripting

Shell script / Grep / Awk to variable and Loop

Hi, I have a text file with data in that I wish to extract, assign to a variable and process through a loop. Kind of the process that I am after: 1: Grep the text file for the values. Currently using: cat /root/test.txt | grep TESTING= | awk -F"=" '{ a = $2 } {print a}' | sort -u ... (0 Replies)
Discussion started by: Spoonless
0 Replies

10. Shell Programming and Scripting

Passing a variable to awk while in a shell for loop

I am a newbie to awk and c programming, however am not a unix newbie. However, I do need help with a kshell script I am writing. It is almost complete, the last step is killing me. Any help would be greatly appreciated. What I am trying to do is cat a text file that has usernames. Then, using... (2 Replies)
Discussion started by: synergy_texas
2 Replies
Login or Register to Ask a Question