Using AWK: Extract data from multiple files and output to multiple new files


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Using AWK: Extract data from multiple files and output to multiple new files
# 1  
Old 10-12-2010
Question Using AWK: Extract data from multiple files and output to multiple new files

Hi,
I'd like to process multiple files. For example:
file1.txt
file2.txt
file3.txt

Each file contains several lines of data. I want to extract a piece of data and output it to a new file.
file1.txt ----> newfile1.txt
file2.txt ----> newfile2.txt
file3.txt ----> newfile3.txt

Here is an example of what an input file and an output file could look like:
file1.txt
---------
line1
line2
line3

newfile1.txt
------------
line2

So, in this case, the awk script would extract line2 from each input file. That part is ok. I need to know how to create the multiple output files which contain the extracted data (ie. line2).

Any help would be greatly appreciatedSmilie
# 2  
Old 10-12-2010
I'm still a newbie at this but what about...

(Assuming all your files are in the same directory)

Code:
for oldfile in *;
do
   egrep 'line2' $oldfile > new-$oldfile;
done;


Last edited by calitiggr; 10-12-2010 at 04:39 AM..
# 3  
Old 10-12-2010
Quote:
Originally Posted by calitiggr
I'm still a newbie at this but what about...

(Assuming all your files are in the same directory)

Code:
for oldfile in *;
do;
   egrep 'line2' $oldfile > new-$oldfile;
done;

I'm a newbie aswellSmilie

Thanks for the reply but I actually solved the problem while messing around with itSmilie.

I wrote the following function:
Code:
function save_file()
{
printf ("\n%-35s%-35s%-35s", uwi, curve, FILENAME) > "new" FILENAME;
}

The variables I created are: uwi, curve
FILENAME is an AWK standard variable.

Hope this helps someone out there. If there's a better way to do this, please feel free to post it here, afterall, I'm a beginner. Otherwise, consider this problem solvedSmilie.

Last edited by Liverpaul09; 10-12-2010 at 10:15 AM..
# 4  
Old 10-12-2010
Code:
 
 awk 'FNR==2{print $0 > "new"FILENAME}' file[123].txt

This User Gave Thanks to malcomex999 For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep strings on multiple files and output to multiple files

Hi All, I want to use egrep on multiple files and the results should be output to multiple files. I am using the below code in my shell script(working in Ksh shell). However with this code I am not attaining the desired results. #!/bin/ksh ( a="/path/file1" b="path/file2" for file in... (4 Replies)
Discussion started by: am24
4 Replies

2. Shell Programming and Scripting

Create Multiple UNIX Files for Multiple SQL Rows output

Dear All, I am trying to write a Unix Script which fires a sql query. The output of the sql query gives multiple rows. Each row should be saved in a separate Unix File. The number of rows of sql output can be variable. I am able save all the rows in one file but in separate files. Any... (14 Replies)
Discussion started by: Rahul_Bhasin
14 Replies

3. Shell Programming and Scripting

Extract data in tabular format from multiple files

Hi, I have directory with multiple files from which i need to extract portion of specif lines and insert it in a new file, the new file will contain a separate columns for each file data. Example: I need to extract Value_1 & Value_3 from all files and insert in output file as below: ... (2 Replies)
Discussion started by: belalr
2 Replies

4. UNIX for Dummies Questions & Answers

Extract common data out of multiple files

I am trying to extract common list of Organisms from different files For example I took 3 files and showed expected result. In real I have more than 1000 files. I am aware about the useful use of awk and grep but unaware in depth so need guidance regarding it. I want to use awk/ grep/ cut/... (7 Replies)
Discussion started by: macmath
7 Replies

5. Shell Programming and Scripting

awk, multiple files input and multiple files output

Hi! I'm new in awk and I need some help. I have a folder with a lot of files and I need that awk do something in each file and print a new file with the output. The input file name should be modified when I print the outpu files. Thanks in advance for help! :-) ciao (5 Replies)
Discussion started by: gabrysfe
5 Replies

6. UNIX for Dummies Questions & Answers

Use awk to pipe output from one file into multiple files

Hi All. Thanks for your help in advance. I have a requirement to examine the number of delimiters in each record of a file. If the record has the expected number of delimiters it should be passed into a 'good' file. If it does not, the record should be passed into a 'bad' file. I have been able... (8 Replies)
Discussion started by: codestar1
8 Replies

7. UNIX for Dummies Questions & Answers

awk, extract last line of multiple files

Hi, I have a directory full of *.txt files. I would like to print the last line of every file to screen. I know you can use FNR for printing the first line of each file, but how do I access the last line of each file? This code doesn't work, it only prints the last line of the last file:BEGIN... (5 Replies)
Discussion started by: Liverpaul09
5 Replies

8. UNIX for Dummies Questions & Answers

AWK, extract data from multiple files

Hi, I'm using AWK to try to extract data from multiple files (*.txt). The script should look for a flag that occurs at a specific position in each file and it should return the data to the right of that flag. I should end up with one line for each file, each containing 3 columns:... (8 Replies)
Discussion started by: Liverpaul09
8 Replies

9. Shell Programming and Scripting

need help with post:extract multiple columns from multiple files

hello, I will would be grateful if anyone can help me reply to my post extract multiple cloumns from multiple files; skip rows and include filenames; awk Please see this thread. Thanks manishabh (0 Replies)
Discussion started by: manishabh
0 Replies

10. Shell Programming and Scripting

extract multiple cloumns from multiple files; skip rows and include filenames; awk

Hello, I am trying to write a bash shell script that does the following: 1.Finds all *.txt files within my directory of interest 2. reads each of the files (25 files) one by one (tab-delimited format and have the same data format) 3. skips the first 10 rows of the file 4. extracts and... (4 Replies)
Discussion started by: manishabh
4 Replies
Login or Register to Ask a Question