awk, multiple files input and multiple files output

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk, multiple files input and multiple files output
# 1  
Old 05-28-2012
awk, multiple files input and multiple files output

I'm new in awk and I need some help.
I have a folder with a lot of files and I need that awk do something in each file and print a new file with the output. The input file name should be modified when I print the outpu files.

Thanks in advance for help!
# 2  
Old 05-28-2012
what is this 'something' you need awk to do?

In what way must the input file be modified?

What does this input and output look like?

We need details.
# 3  
Old 05-28-2012
the 'something' is the elimination of duplicates considering only the first column (i try '!_[$1]++' and works good for me).
I want to eliminate the duplicates (as written above) from each input file in the folder and I want an ouput file (one for each input file) that should have the same name of the input file with added something. For example: input file name is "333.e" and I would like an output file name like "333_mod.e".
The input files have two columns and I need to eliminate duplicates agreeing with the elements of the first column.
They are like:

pb_04 0,1
pb_05 0.3
pb_05 0.2
pb_12 0.85
... ...

in this case i need to eliminate one row with pb_05.

# 4  
Old 05-29-2012
How about this ?
awk '!a[$1]{a[$1]++;print $0 > FILENAME"_mod"}' File.txt

# 5  
Old 05-29-2012
Thank you Pravin!! it works perfectly!!
MAy be you know also how can I apply it for a list of files in a folder?
All the files have some equal words in the name and their name look like 3254_blabla.e
# 6  
Old 05-29-2012
Don't forget to close the files in awk, or it will break if there are many output files..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep strings on multiple files and output to multiple files

Hi All, I want to use egrep on multiple files and the results should be output to multiple files. I am using the below code in my shell script(working in Ksh shell). However with this code I am not attaining the desired results. #!/bin/ksh ( a="/path/file1" b="path/file2" for file in... (4 Replies)
Discussion started by: am24
4 Replies

2. Shell Programming and Scripting

[Solved] Multiple input files and output files

Hi, I have many test*.ft1 files to which I want to read as input for a script called pipe2txt.tcl and print the output in each separate file. For example, pipe2txt.tcl < test001.ft1 > test001.txt How can I read many files in this maner? thank you very much, Best, Pahuja (5 Replies)
Discussion started by: Pahuja
5 Replies

3. Shell Programming and Scripting

Create Multiple UNIX Files for Multiple SQL Rows output

Dear All, I am trying to write a Unix Script which fires a sql query. The output of the sql query gives multiple rows. Each row should be saved in a separate Unix File. The number of rows of sql output can be variable. I am able save all the rows in one file but in separate files. Any... (14 Replies)
Discussion started by: Rahul_Bhasin
14 Replies

4. Shell Programming and Scripting

FOR loop with multiple files as input and awk

Hi all , i want to pass multiple files as input to a for loop for i in file1 file2 file3 do some awk action < $i >> $i.out done but im getting error in that for loop is the way i use to pass files to awk using for correct and 2.we can directly pass multiple files to awk as... (7 Replies)
Discussion started by: zozoo
7 Replies

5. Shell Programming and Scripting

Split: File into multiple and keeping the same 3 lines from input into all output files

The following code will split the infile into multiple files. However, I need it to insert the same first 3 lines from the original input file into each splitted file. How do I modify my script below to do so: print -n "Enter file name to split? " ; read infile if then echo "Invalid file... (4 Replies)
Discussion started by: mrn6430
4 Replies

6. UNIX for Dummies Questions & Answers

Use awk to pipe output from one file into multiple files

Hi All. Thanks for your help in advance. I have a requirement to examine the number of delimiters in each record of a file. If the record has the expected number of delimiters it should be passed into a 'good' file. If it does not, the record should be passed into a 'bad' file. I have been able... (8 Replies)
Discussion started by: codestar1
8 Replies

7. Shell Programming and Scripting

how to give multiple csv files as input in awk

Hi All, I am new to shell scripting..My problem is i want to give multiple csv files as input to awk script and process the data into one file.. My input file is File1 File2 File3 Product Location Period SalesPrice A x 8/11/2010 ... (7 Replies)
Discussion started by: kvth
7 Replies

8. UNIX for Dummies Questions & Answers

Using AWK: Extract data from multiple files and output to multiple new files

Hi, I'd like to process multiple files. For example: file1.txt file2.txt file3.txt Each file contains several lines of data. I want to extract a piece of data and output it to a new file. file1.txt ----> newfile1.txt file2.txt ----> newfile2.txt file3.txt ----> newfile3.txt Here is... (3 Replies)
Discussion started by: Liverpaul09
3 Replies

9. Shell Programming and Scripting

building output file from multiple input files

Hi there, I am trying to figure out a way to combine multiple sources with different data on a single file, and I am trying to find the best way to do it. I have multiple files, let's say A, B, C and D. A has a field in common with B, B has a field in common with C, and C has a field in... (2 Replies)
Discussion started by: ppucci
2 Replies

10. Shell Programming and Scripting

Splitting input files into multiple files through AWK command

Hi, I needs to split *.txt files from single directory depends on the some mutltiple input values. i have wrote the code like below for file in *.txt do grep -i -h "value1|value2" $file > $file; done. My requirment is more input values needs to be given in grep; let us say 50... (3 Replies)
Discussion started by: arund_01
3 Replies
Login or Register to Ask a Question

Featured Tech Videos