Visit Our UNIX and Linux User Community


Need some Help for file filteration and saving the output in other directory


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need some Help for file filteration and saving the output in other directory
# 1  
Old 02-19-2012
Data Need some Help for file filteration and saving the output in other directory

Hi all........
Plss do help me.......in a big trouble... SmilieSmilieSmilie
I have 3 directories named as :1. /home/shuchi/source
2./home/shuchi/destination
3./home/shuchi/filter
now the problem is /home/shuchi/source has say 2 files with extension .txt as given below :
A.txt
Code:
msisdn             | imsi | name | subscription | product | date | recharge type
09311424354 | 09488439820 | ekta | yes | VAS | 2012-12-07 | IVR
09128734473 | 83974738734 | priya | no | Non VAS | 2011-08-11 | SMS
09376536263 | 63473627632 | shubhra | yes | VAS | 2011-07-07 | WAP
09655423434 | 13312353343 | shubhi | no | non VAS | 2011-05-04 | SMS
09473874383 | 36267267372 | amita | yes | VAS | 2011-04-05 | IVR

B.txt
Code:
msisdn             | imsi | name | subscription | product | date | recharge type
09311424354 | 09488439820 | ekta | yes | VAS | 2012-12-07 | IVR
09128734473 | 83974738734 | priya | no | Non VAS | 2011-08-11 | SMS
09376536263 | 63473627632 | shubhra | yes | VAS | 2011-07-07 | WAP
09302117719 | 13312353343 | shubhi | no | non VAS | 2011-05-04 | SMS
08295708671 | 36267267372 | amita | yes | VAS | 2011-04-05 | IVR

/home/shuchi/filter has one file as below :
msisdn
--------------
Code:
09311424354
09128734473
09487736472
09302117719
08295708671

now on the basis of file present in /home/shuchi/filter i need to have 2 subfiles with the same name as A.txt and B.txt in /home/dstst/destination folder as given below :
A.txt
Code:
msisdn             | imsi | name | subscription | product | date | recharge type
09311424354 | 09488439820 | ekta | yes | VAS | 2012-12-07 | IVR
09128734473 | 83974738734 | priya | no | Non VAS | 2011-08-11 | SMS:o

B.txt
Code:
msisdn             | imsi | name | subscription | product | date | recharge type
09302117719 | 13312353343 | shubhi | no | non VAS | 2011-05-04 | SMS
08295708671 | 36267267372 | amita | yes | VAS | 2011-04-05 | IVR

i have to write a script which will read files present in /home/shuchi/source one by one i.e. A.txt and B.txt and match them with file present in /home/shuchi/filter and on the basis of msisdn and store the result in /home/shuchi/destination directory.I have to use grep or fgrep as some crappy version of unix being used and awk and sed are not supported ............it would be a grttttttttt help...........SmilieSmilieSmilieSmilie

~Priya...........

Last edited by Franklin52; 02-20-2012 at 04:46 AM.. Reason: Please use code tags for code and data samples, thank you
# 2  
Old 02-19-2012
No awk, that's very odd. Assuming you have Kshell or bash, have a go with this. Replace the directory names with yours (I used the ones here to test).

Code:
#!/usr/bin/env bash
# change the directory names to be your source, destination and filter
sdir=src     
ddir=dest
fdir=filter

# use find to prevent issues if there are a lot of *.txt files
find $sdir -name "*.txt" | while read file
do
    grep -f $fdir/filter $file >$ddir/${file##*/}
done

You might want to remove the line of dashes from your filter file since that is not a potential pattern in one of your source files you'll waste time with each line in each grep trying to match it.
This User Gave Thanks to agama For This Post:
# 3  
Old 02-21-2012
Data

Quote:
Originally Posted by agama
No awk, that's very odd. Assuming you have Kshell or bash, have a go with this. Replace the directory names with yours (I used the ones here to test).

Code:
#!/usr/bin/env bash
# change the directory names to be your source, destination and filter
sdir=src     
ddir=dest
fdir=filter

# use find to prevent issues if there are a lot of *.txt files
find $sdir -name "*.txt" | while read file
do
    grep -f $fdir/filter $file >$ddir/${file##*/}
done

You might want to remove the line of dashes from your filter file since that is not a potential pattern in one of your source files you'll waste time with each line in each grep trying to match it.
Thanks a lot buddy....it worked....thx again....

---------- Post updated at 07:21 AM ---------- Previous update was at 03:38 AM ----------

Quote:
Originally Posted by agama
No awk, that's very odd. Assuming you have Kshell or bash, have a go with this. Replace the directory names with yours (I used the ones here to test).

Code:
#!/usr/bin/env bash
# change the directory names to be your source, destination and filter
sdir=src     
ddir=dest
fdir=filter

# use find to prevent issues if there are a lot of *.txt files
find $sdir -name "*.txt" | while read file
do
    grep -f $fdir/filter $file >$ddir/${file##*/}
done

You might want to remove the line of dashes from your filter file since that is not a potential pattern in one of your source files you'll waste time with each line in each grep trying to match it.
frnd...........SmilieSmilieSmilie
d above script is writing the file to destination directory but no data in it...SmilieSmilie
all d files are 0 byte files....my manager is makin my lyf hell to complete dis...SmilieSmilie
plz plz do help me.................Smilie

~Priya............
# 4  
Old 02-21-2012
If you could paste the script you are using it would help. Possibly a typo in the script. You could also try running a grep by hand to see what it yields.

Assuming one of the files that find will list is foo.txt, try running this to see what happens:


Code:
grep -f filter-file-name foo.txt|more

If you see output from this command, then your script likely has a bug. No output could mean that there was nothing in foo.txt to find, or the filter file is wrong.

Hard to say more without seeing your script.
# 5  
Old 02-22-2012
Data

Quote:
Originally Posted by agama
If you could paste the script you are using it would help. Possibly a typo in the script. You could also try running a grep by hand to see what it yields.

Assuming one of the files that find will list is foo.txt, try running this to see what happens:


Code:
grep -f filter-file-name foo.txt|more

If you see output from this command, then your script likely has a bug. No output could mean that there was nothing in foo.txt to find, or the filter file is wrong.

Hard to say more without seeing your script.
Code:
#!/usr/bin/env bash
# change the directory names to be your source, destination and filter
sdir=/data1/T2EDW/R1D1_SIT/initial_files/00800010
ddir=/home/dstst/ekta
fdir=/home/dstst/ekta/xp

# use find to prevent issues if there are a lot of *.txt files
find $sdir -name "*.EDW" | while read file
do
    grep -f $fdir $file >> $ddir/${file##*/}
done

above is the script with proper directories for source and destinations...plz plz help me......my manager is at top of my head plz............SmilieSmilieSmilieSmilie

Last edited by Franklin52; 02-22-2012 at 05:42 AM.. Reason: Please use code tags for code and data samples, thank you
# 6  
Old 02-22-2012
I don't see any problems with your script. You probably should change the double redirection (>>) to a signle (>), but that's not causing the files to be empty.

The only thing I can think of is that you might have trailing blanks and/or tabs in your list of patterns (the filter file: /home/dstst/ekta/xp). Check it to be sure that there isn't any leading or trailing white space (spaces or tabs) as that might be the cause of nothing being found.

Previous Thread | Next Thread
Test Your Knowledge in Computers #467
Difficulty: Easy
In computer networking, telecommunication and information theory, broadcasting is a method of transferring a message to all recipients using reverse polish notation.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed command is saving output as blank file

Hi, I am working on a script where I am adding adding colors to few of the info in the output. Now , after that is done , I see colour codes in log files which I don't want to see.:mad::mad::mad::mad: So , I tried using sed command in script as below which gives me o/p (new.log) as blank file... (7 Replies)
Discussion started by: Dream4649
7 Replies

2. Shell Programming and Scripting

Saving files with file name as output

Hi, i need help with a file creation of an output program. I've got a program that with #find creates an output for each files in a directory. If i give this command : -o spec$(date -u +%Y%m%dt%H%M) it creates just one file, overwriting all the others since it is the creation date .... (2 Replies)
Discussion started by: Board27
2 Replies

3. Shell Programming and Scripting

Saving nohup output to a file other than nohup.out

Shell : bash OS : Oracle Linux 6.4 I want to save the ouput of a nohup command to file other than nohup.out . Below are my 3 attempts. For both Attempt1 and Attempt2 , the redirection logs the output correctly to the output file. But I get the error "ignoring input and redirecting stderr to... (7 Replies)
Discussion started by: kraljic
7 Replies

4. Shell Programming and Scripting

Saving svn log in a separate directory

Hi Folks, I have a directory at /usr/local/aa and there is other directory where I have checkout the code through svn checkout command ( /opt/app/fgh) now I can apply svn log command here as shown below $ cd /opt/app/fgh svn checkout <url> svn log shows me the log on console . please... (3 Replies)
Discussion started by: punpun66
3 Replies

5. Shell Programming and Scripting

Need some Help for file filteration and saving the output in other directory using grep....plz ...

Hi all........ Plss do help me.......in a big trouble... :wall::wall::wall: I have 3 directories named as :1. /home/shuchi/source 2./home/shuchi/destination 3./home/shuchi/filter now the problem is /home/shuchi/source has say 2 files with extension .txt as given below : A.txt Code: ... (0 Replies)
Discussion started by: ektubbe
0 Replies

6. Shell Programming and Scripting

Need help for file filteration script......

Hiii all... Pls help me out wid below prblm : i have 5 files A,B,C,D and E located at /home/anubha I have a file F located at /home/anubha/ed File F has some records which can be matched in A,B,C,D and E and another set of files on the basis of these filteration should be created i.e.... (5 Replies)
Discussion started by: ektubbe
5 Replies

7. Shell Programming and Scripting

saving output from bash into a file

I am ssh to many servers to get some information... however sometimes the server is unreacheable and i am getting an error. I want to save that output to a file but I am not able to do so... I want to be able to save output of bash into a file.. so when I run this command on a script ssh... (5 Replies)
Discussion started by: eponcedeleonc
5 Replies

8. UNIX for Dummies Questions & Answers

Saving a temporary output within a script

Good morning everyone, i am looking to know how to save the output of a command and reuse it again within a script i already tired this one but it didn't work TEMPDIR=/dir1/dir2 My_command> $TEMPDIR/$TEMPFILE rm $TEMPDIR/$TEMPFILE* it keeps saying "cannot write to a... (15 Replies)
Discussion started by: Portabello
15 Replies

9. UNIX for Dummies Questions & Answers

saving command output to a variable

Hello, I have a shell script containing a command string in the following format: command1 | command2 | cut -c9-16 The output from this is a record number (using characters 9-16 of the original output string) e.g. ORD-1234 I wish to save this value to a variable for use in later commands... (4 Replies)
Discussion started by: philjo
4 Replies

10. Shell Programming and Scripting

Saving output from awk into a perl variable

How would I pass awk output to a perl variable? For example, I want to save the value in the 4th column into the variable called test. My best guess is something as follow, but I am sure this isn't correct. $test = system("awk '/NUMBER/{print \$4}' $_"); (8 Replies)
Discussion started by: userix
8 Replies

Featured Tech Videos