Loop over files and use awk


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Loop over files and use awk
# 1  
Old 03-21-2013
Loop over files and use awk

Hi, I have a large number of files which are numbered numerically, i.e. of the type

Code:
1.usr, 2.usr, 3.usr ...

This is what I'd like to do:

1. In ascending order, use awk to read a value from each file.

2. Write this value to another file (say data.txt). This file, 'data.txt' should be appended.

The problem is when I use the following command

Code:
awk '{print $2 }' *.usr > data.txt

It does write out all the data I need into a file but awk doesn't arrange the files in ascending order when it does it.

Thanks!
# 2  
Old 03-21-2013
Code:
ls *.usr | xargs awk '{print $2 }' > data.txt

# 3  
Old 03-21-2013
Quote:
Originally Posted by lost.identity
awk doesn't arrange the files in ascending order when it does it.
Don't blame awk. It's your shell doing the globbing in lexicographical order.

One way:
Code:
printf "%s\n" *.usr|sort -n|xargs awk '...'

This User Gave Thanks to elixir_sinari For This Post:
# 4  
Old 03-21-2013
Thanks!
# 5  
Old 03-21-2013
Supposing that you have 1-3 digit user numbers, you could also use:
Code:
awk '{print $2 }' ?.usr ??.usr ???.usr > data.txt

without needing to resort to sort and xargs. Obviously, you'll need more terms in the list of files passed to awk if some of your *.usr names contain more than 3 digits; but the pattern should be obvious with this example.
These 2 Users Gave Thanks to Don Cragun For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Awk: Performing "for" loop within text block with two files

I am hoping to pull multiple strings from one file and use them to search within a block of text within another file. File 1PS001,001 HLK PS002,004 MWQ PS004,002 RXM PS004,006 DBX PS004,006 SBR PS005,007 ML PS005,009 DBR PS005,011 MR PS005,012 SBR PS006,003 RXM PS006,003 >SJ PS006,010... (11 Replies)
Discussion started by: jvoot
11 Replies

2. UNIX for Dummies Questions & Answers

Loop awk command on files in a folder

Hi, I'd like to loop an action over all files with given extension within a folder. The "main" action is: awk -F "\t" 'BEGIN{OFS="\t"}{if ($10=="S") print$0; }' input.txt > output.txt The input.txt should be every file in the folder with *.subVCF extension; and the output should be a file... (3 Replies)
Discussion started by: dovah
3 Replies

3. Shell Programming and Scripting

awk - 2 files comparison without for loop - multi-line issue

Greetings Experts, I need to handle the views created over monthly retention tables for which every new table in YYYYMMDD format, there is equivalent view created and the older table which might be dropped, the view over it has to be re-created over a dummy table so that it doesn't fail.... (2 Replies)
Discussion started by: chill3chee
2 Replies

4. Shell Programming and Scripting

How to use a loop for multiple files in a folder to run awk command?

Dear folks I have two data set which there names are "final.map" and "1.geno" and look like this structures: final.map: gi|358485511|ref|NC_006088.3| 2044 gi|358485511|ref|NC_006088.3| 2048 gi|358485511|ref|NC_006088.3| 2187 gi|358485511|ref|NC_006088.3| 17654 ... (2 Replies)
Discussion started by: sajmar
2 Replies

5. Shell Programming and Scripting

For loop inside awk to read and print contents of files

Hello, I have a set of files Xfile0001 - Xfile0021, and the content of this files (one at a time) needs to be printed between some line (lines start with word "Generated") that I am extracting from another file called file7.txt and all the output goes into output.txt. First I tried creating a for... (5 Replies)
Discussion started by: jaldo0805
5 Replies

6. Shell Programming and Scripting

FOR loop with multiple files as input and awk

Hi all , i want to pass multiple files as input to a for loop for i in file1 file2 file3 do some awk action < $i >> $i.out done but im getting error in that for loop is the way i use to pass files to awk using for correct and 2.we can directly pass multiple files to awk as... (7 Replies)
Discussion started by: zozoo
7 Replies

7. Shell Programming and Scripting

awk: too many output files created from while loop

I am using awk to read lines from a CSV file then put data into other files. These other files are named using the value of a certain column. Column 7 is a name such as "att" or "charter" . I want to end up with file names with the value of column 7 appended to them, like this: ... (5 Replies)
Discussion started by: dodgerfan78
5 Replies

8. Shell Programming and Scripting

Loop for row-wise averaging of multiple files using awk

Hello all, I need to compute a row-wise average of files with a single column based on the pattern of the filenames. I really appreciate any help on this. it would just be very difficult to do them manually as the rows are mounting to 100,000 lines. the filenames are as below with convention as... (2 Replies)
Discussion started by: ida1215
2 Replies

9. Shell Programming and Scripting

Get values from 2 files - Complex "for loop and if" awk problem

Hi everyone, I've been thinking and trying/changing all day long the below code, maybe some awk expert could help me to fix the for loop I've thought, I think I'm very close to the correct output. file1 is: <boxes content="Grapes and Apples"> <box No.="Box MT. 53"> <quantity... (8 Replies)
Discussion started by: Ophiuchus
8 Replies

10. Shell Programming and Scripting

Comparison and editing of files using awk.(And also a possible bug in awk for loop?)

I have two files which I would like to compare and then manipulate in a way. File1: pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2: pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2... (1 Reply)
Discussion started by: linuxkid
1 Replies
Login or Register to Ask a Question