newbie needs help batching awk, tabitha


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting newbie needs help batching awk, tabitha
# 1  
Old 08-27-2012
newbie needs help batching awk, tabitha

Hi guys,

I need a little help learning to batch an awk script. All the examples I found on line are too complicated for me. Here's the awk command that I want to run on lots of files.

Code:
awk 'NR==FNR{a[$1]=$0;next$3 in a{print $0 a[$1] " " a[$3]}' inputfile_A_1.out  inputfile_B_1.out  >  outputfile1.txt

then on
inputfile_A_2.out inputfile_B_2.out > outputfile2.txt
and
inputfile_A_3.out inputfile_B_3.out > outputfile3.txt
like that

thanks sooo much, tabitha
# 2  
Old 08-27-2012
first of all, you have an issue with your awk - missing '}':
Code:
awk 'NR==FNR{a[$1]=$0;next} $3 in a{print $0 a[$1] " " a[$3]}' inputfile_A_1.out  inputfile_B_1.out  >  outputfile1.txt

Then... something to start with:
Code:
#!/bin/ksh

for i in 1 2 3 4 5
do
  awk 'NR==FNR{a[$1]=$0;next} $3 in a{print $0,a[$1], a[$3]}' "inputfile_A_${i}.out"  "inputfile_B_${i}.out" > "outputfile${i}.txt"
done

# 3  
Old 08-27-2012
thanks! you are correct, I forgot the curly brace in my post - ooops

will it work using bash, or just ksh, I have bash?

the file names are more complex than I first showed, sorry, here's some examples of how the files are named:

inputfile_A_1 --> CFD-2012-10-7_RTC_P09_L000058.txt
inputfile_B_1 --> CFD-2012-10-7_RTC_P09_L000102.txt

inputfile_A_2 --> CFD-2012-10-4_RTC_P11_L000058.txt
inputfile_B_2 --> CFD-2012-10-4_RTC_P11_L000102.txt

inputfile_A_2 --> CFD-2012-11-4_RTC_P02_L000058.txt
inputfile_B_2 --> CFD-2012-11-4_RTC_P02_L000102.txt

the _L000058.txt and the _L000102.txt are always the same but the rest of the file name will change like that but many many more

thanks tabitha
# 4  
Old 08-27-2012
He's not using any ksh-specific features.
# 5  
Old 08-27-2012
Quote:
Originally Posted by Corona688
He's not using any ksh-specific features.
thank you!

what should I do about the complex filenames to batch over them?
# 6  
Old 08-28-2012
try this
Code:
i=1
for j in $(ls *_L000058.txt)
do 

b_file=$(echo $j | awk -F"_" 'BEGIN{OFS=FS}{print $1,$2,$3"_L000102.txt"}')

awk 'NR==FNR{a[$1]=$0;next} $3 in a{print $0,a[$1], a[$3]}' $j $b_file  > "outputfile${i}.txt"

i=$(expr $i + 1)

done


Last edited by raj_saini20; 08-28-2012 at 03:35 AM..
# 7  
Old 08-28-2012
Something shorter and elegant:
Code:
for i in *_L000058.txt
do
 awk 'NR==FNR{a[$1]=$0;next} $3 in a{print $0,a[$1], a[$3]}' $i ${i%_*}_L000102.txt > outputfile$((++j)).txt
done

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Basic awk...newbie quetion

Hi, I was trying to change the value of the 4th column (put '1' in the 4th column of each row). My awk command is: awk -F, '{$3=1;}1' OFS= input.txt > ./test_out.txt My input file is: a 1 2 31 b 4 5 61 c 7 8 91 My output file (test_out.txt)is: a 1 2 31 b 4 5 61 c 7 8 91 What... (4 Replies)
Discussion started by: pc2001
4 Replies

2. Shell Programming and Scripting

Newbie AWK Question

$ awk -f awk1.awk Where awk1.awk reads: BEGIN { printf ("Name = Smith, Richard" > "awk1" \n) printf ("Name = Johnson, Waylan" > "awk1" \n) printf ("Name = Brown, Pat" > "awk1" \n) } If I get rid of "\n", awk1 file gets created but all the names... (3 Replies)
Discussion started by: ora_umair
3 Replies

3. Shell Programming and Scripting

Help with awk for a newbie

Hi there, greetingt to everybody. I have configured syslog-ng to get messages over UDP saving logs into a text file, it works fine. I need to store the content of this file in several files depending from some criteria that I try to explain you with some examples : Suppose the content of my log... (5 Replies)
Discussion started by: capnino
5 Replies

4. Shell Programming and Scripting

Help a newbie please with awk if else statements

Hi, Despite reading the Conditional Statements chapter in the O'Reilly Sed & Awk book several times and looking at numerous examples, I cannot for the life of me get any kind of if ... else statement to work in my awk scripts! My scripts work perfectly (as they are written at least) and do what... (4 Replies)
Discussion started by: jonathanm
4 Replies

5. Shell Programming and Scripting

Grep causing long delay (batching) whilst piping

Hi all. I have a problem at work which I have managed to break down into a simple test scenario: I have written a monitoring script that outputs every second the status of various processes, but for now, lets just print the date input.sh: while true do date sleep 1 done This... (9 Replies)
Discussion started by: spudtheimpaler
9 Replies

6. Shell Programming and Scripting

Need help with AWK (newbie)

Hello all I have a file with below data, I need to add a third column as XXXX if the first coloumn (H180620081) last number is "1" and YYYY if the first coloumn last number is "2" Can it be done with AWK, I would appreciate if anyone could help me ou with this. H180620081 31310 ... (11 Replies)
Discussion started by: udaybo
11 Replies

7. Shell Programming and Scripting

Need Help with AWK or SED (Newbie)

Hello, I have a text file in below format, how do I put a header and assign field names to the file with either AWK or SED STRT~ VA ~23606 ~TM14~8506~1485 (page 1) STRT~ VA ~23662 ~TM17~8362~1783 (page 2) STRT~ VA ~23662 ~TM17~8362~1783 STRT~ VA ~23662 ~TM17~8362~1783 STRT~ VA ~23662... (1 Reply)
Discussion started by: udaybo
1 Replies

8. Shell Programming and Scripting

awk newbie

I would like to parse a file and use each record to create a unique file where the name of the file created is taken from the 3rd field. Example: here are 3 records. 000007 0110 07-0001583 20060802132213 20060801112601 000007 0110 07-0001584 ... (4 Replies)
Discussion started by: zoo591
4 Replies

9. Shell Programming and Scripting

batching files together

I'm looking after part of an application which batches up files concatanates them together and then moves the concatanated files to another directory. The problem I have is that the box runs Windows 2003 and UNIX shell emulation is provided by MKS Toolkit, when 10,000 or more files arive in the... (1 Reply)
Discussion started by: pookie
1 Replies

10. UNIX for Dummies Questions & Answers

batching files

HI, Suppose I have thousand files in one directory and they can keep increasing ie /home/rooh Then I want to create a batch of 100 files and then compress that batch file and ftp them somewhere else. And morever the files in the batch will be moved to somewhere else. Can you please help... (21 Replies)
Discussion started by: rooh
21 Replies
Login or Register to Ask a Question