Extracting data from one file, based on another file (splitting)


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Extracting data from one file, based on another file (splitting)
# 8  
Old 02-08-2014
Code:
FILES="data/*"
for Z in $FILES
do
	name=$(basename $Z) 
	awk -F\| 'FNR==NR{A[$1];next}FNR==1{if(f)close(f);split(FILENAME,F,".");f=F[1]"-testing.txt"}($1 in A){print $0>f}' $Z file2
done

data directory is where I placed my file1
interestingly I get no errors but nothing happens either
# 9  
Old 02-08-2014
Please try bartus11's approach
This User Gave Thanks to Akshay Hegde For This Post:
# 10  
Old 02-08-2014
Thanks for the help
I sorted it out by taking the directory out ... because then the results get combined...
Code:
FILES="*"
for Z in $FILES
do
	name=$(basename $Z) 
	awk -F"|" 'NR==FNR{a[$1]=1;next}$1 in a{print > $1"-testing.txt"}' file1 file2
done

I dont know how to put the results in the another directory as I would like to add more files to my data file
# 11  
Old 02-09-2014
Quote:
Originally Posted by A-V
Thanks for the help
I sorted it out by taking the directory out ... because then the results get combined...
Code:
FILES="*"
for Z in $FILES
do
	name=$(basename $Z) 
	awk -F"|" 'NR==FNR{a[$1]=1;next}$1 in a{print > $1"-testing.txt"}' file1 file2
done

I dont know how to put the results in the another directory as I would like to add more files to my data file
I'm confused and lost.
What directory did you take out?
The above code will run the command:
Code:
	awk -F"|" 'NR==FNR{a[$1]=1;next}$1 in a{print > $1"-testing.txt"}' file1 file2

once for every file in the current directory. And, since the values stored in Z and name have no effect on the behavior of the command:
Code:
	awk -F"|" 'NR==FNR{a[$1]=1;next}$1 in a{print > $1"-testing.txt"}' file1 file2

the entire code sequence shown above could be replaced by the single command:
Code:
awk -F"|" 'NR==FNR{a[$1]=1;next}$1 in a{print > $1"-testing.txt"}' file1 file2

In message #2 in this thread bartus11 proposed using:
Code:
awk -F"|" 'NR==FNR{a[$1]=1;next}$1 in a{print > $1"-testing.txt"}' file1 file2

to solve your problem. The above two lines look the same to me, but you said it didn't work. What changed?

As far as what you need to do to put the results in a different directory, change:
Code:
awk -F"|" 'NR==FNR{a[$1]=1;next}$1 in a{print > $1"-testing.txt"}' file1 file2

to:
Code:
awk -F"|" 'NR==FNR{a[$1]=1;next}$1 in a{print > "/destination/directory/here/"$1"-testing.txt"}' file1 file2


Last edited by Don Cragun; 02-09-2014 at 09:36 PM.. Reason: Add missing /
This User Gave Thanks to Don Cragun For This Post:
# 12  
Old 02-09-2014
I would create a korn shell script as below, and then pass file1 and file2 as parameters to it:

Code:
#!/bin/ksh
infile1=$1
infile2=$2
for name in $(cat ${infile1})
do
   egrep ${name} ${infile2} > ${name}.txt
done

..... I named it as script.ksh
I run it as:
script.ksh file2 file1

It will create three files for Tom, David and Ellen each ... if that's what you wanted?

thanks
This User Gave Thanks to juzz4fun For This Post:
# 13  
Old 02-19-2014
Thank you very much for help Smilie Don Cragun thank you so much for explaining things
and Jazz4fun, yes I managed to make it word at the end.. .thanks
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Splitting the file based on two fields - Fixed length file

Hi , I am having a scenario where I need to split the file based on two field values. The file is a fixed length file. ex: AA0998703000000000000190510095350019500010005101980301 K 0998703000000000000190510095351019500020005101480 ... (4 Replies)
Discussion started by: saj
4 Replies

2. Shell Programming and Scripting

Script for extracting data from csv file based on column values.

Hi all, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 5 columns having values say column 1,column 2.....column 5 as below along with their valuesm.... (3 Replies)
Discussion started by: Vivekit82
3 Replies

3. Shell Programming and Scripting

Help me pls : splitting single file in unix into different files based on data

I have a file in unix with sample data as follows : -------------------------------------------------------------- -------------------------------------------------------------- {30001002|XXparameter|Layout|$ I want this file to be splitted into different files and corresponding to the sample... (54 Replies)
Discussion started by: Ravindra Swan
54 Replies

4. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

5. Shell Programming and Scripting

Extracting data based on the list file

Hi there, Can you help. I need to extract data based on the list file(list.txt) from item.txt as shown below. Please note the actual files are enormous in size. Thank you. item.txt nokia1100 123,000 nokia2100 66,000 samsung123 11,000 samsung456 23,000 iphone432 234,000... (12 Replies)
Discussion started by: shtobias
12 Replies

6. Shell Programming and Scripting

Extracting data between tags based on search string from unix file

Input file is on Linux box and the input file has data in just one line with 1699741696 characters. Sample Input: <xxx><document coll="uspatfull" version="0"><CMSdoc>xxxantivirus</CMSdoc><tag1>1</tag1></document><document coll="uspatfull"... (5 Replies)
Discussion started by: gaya
5 Replies

7. UNIX for Dummies Questions & Answers

Splitting a file based on record sin another file

All, We receive a file with a large no of records (records can vary) and we have to split it into two files based on another file. e.g. File1: UHDR 2008112 "25187","00000022","00",21-APR-1991,"" ,"D",-000000519,+0000000000,"C", ,+000000000,+000000000,000000000,"2","" ,21-APR-1991... (7 Replies)
Discussion started by: er_ashu
7 Replies

8. Shell Programming and Scripting

Splitting a file based on the records in another file

All, We receive a file with a large no of records (records can vary) and we have to split it into two files based on another file. e.g. File1: UHDR 2008112 "25187","00000022","00",21-APR-1991,"" ,"D",-000000519,+0000000000,"C", ,+000000000,+000000000,000000000,"2","" ... (2 Replies)
Discussion started by: er_ashu
2 Replies

9. Shell Programming and Scripting

Extracting data from text file based on configuration set in config file

Hi , a:) i have configuration file with pattren <Range start no>,<Range end no>,<type of records to be extracted from the data file>,<name of the file to store output> eg: myfile.confg 9899000000,9899999999,DATA,b.dat 9899000000,9899999999,SMS,a.dat b:) Stucture of my data file is... (3 Replies)
Discussion started by: suparnbector
3 Replies

10. Shell Programming and Scripting

[Splitting file] Extracting group of segments from one file to others

Hi there, I need to split one huge file into separate files if the condition is fulfilled according to that the position between 97 and 98 matches with “IT” at the segment MAS. There is no delimiter file is fix-width with varous line length. Could you please help me how I do split the file... (1 Reply)
Discussion started by: ozgurgul
1 Replies
Login or Register to Ask a Question