Visit Our UNIX and Linux User Community


Need help for file filteration script......


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need help for file filteration script......
# 1  
Old 02-16-2012
Data Need help for file filteration script......

Hiii all...

Pls help me out wid below prblm :
i have 5 files A,B,C,D and E located at /home/anubha
I have a file F located at /home/anubha/ed
File F has some records which can be matched in A,B,C,D and E
and another set of files on the basis of these filteration should be created i.e. G,H,I,J and K to be stored at /home/anubha/priya
Plsss help me out....pls pls pls..pls pls pls...SmilieSmilie

I created below script for the same but it is just creating the files at locations not having any data in it....can you ppl plz lemme kno whr am i making mistakE???SmilieSmilieSmilieSmilieSmilieSmilie
Code:
##filter.sh
#!/bin/bash
. anusha.sh
#we will use shell script to create these filter files as well
for file in `ls -1 ${SRC_DIR}/*.EDW`
do
       filename=`basename $file`
#echo "FILE NAME -" $filename
grep -e ${IMSI_FILTER}  ${file} >> ${TEMP_LOAD_DIR}
done

##anusha.sh
Code:
#!/bin/bash
export SRC_DIR="/home/anubha"
export LND_DIR="/home/anubha/priya"
export TEMP_LOAD_DIR="/home/anubha/priya"
export IMSI_FILTER="/home/anubha/ed"

Moderator's Comments:
Mod Comment How to use code tags

Last edited by Franklin52; 02-16-2012 at 05:10 AM.. Reason: Please use code tags for code and data samples, thank you
# 2  
Old 02-16-2012
1) Please give an exemple :
What do contains A B and E ?
How should look like G H and K ?

2) If you use redirection like ... >> ${MY_FILE}
${MY_FILE} should be a file (not a directory!)

3) Just to take some good habits : don't use "file" as a name of a variable since it already is a name of a unix command.
# 3  
Old 02-16-2012
Power

Quote:
Originally Posted by ctsgnb
1) Please give an exemple :
What do contains A B and E ?
How should look like G H and K ?

2) If you use redirection like ... >> ${MY_FILE}
${MY_FILE} should be a file (not a directory!)

3) Just to take some good habits : don't use "file" as a name of a variable since it already is a name of a unix command.
A,B,E are having 10 columns one of which is msisdn...
filter files F is having a list of msisdn...on the basis of F;A,B,C,D and E will be picked up with relevant rows and should be written as G,H and K..

---------- Post updated at 04:17 AM ---------- Previous update was at 04:14 AM ----------

Quote:
Originally Posted by ctsgnb
1) Please give an exemple :
What do contains A B and E ?
How should look like G H and K ?

2) If you use redirection like ... >> ${MY_FILE}
${MY_FILE} should be a file (not a directory!)

3) Just to take some good habits : don't use "file" as a name of a variable since it already is a name of a unix command.
A,B,E are having 10 columns one of which is msisdn...
filter files F is having a list of msisdn...on the basis of F;A,B,C,D and E will be picked up with relevant rows and should be written as G,H and K..

also below are the examples of the files i am taking into consideration :
F
Code:
msisdn
--------------
09311424354
09128734473
09487736472

A/B/C/D/E
Code:
msisdn             | imsi | name | subscription | product | date | recharge type
09311424354 | 09488439820 | ekta | yes | VAS | 2012-12-07 | IVR
09128734473 | 83974738734 | priya | no | Non VAS | 2011-08-11 | SMS
09376536263 | 63473627632 | shubhra | yes | VAS | 2011-07-07 | WAP
09655423434 | 13312353343 | shubhi | no | non VAS | 2011-05-04 | SMS
09473874383 | 36267267372 | amita | yes | VAS | 2011-04-05 | IVR

G/H/I/J/K
Code:
msisdn             | imsi | name | subscription | product | date | recharge type
09311424354 | 09488439820 | ekta | yes | VAS | 2012-12-07 | IVR
09128734473 | 83974738734 | priya | no | Non VAS | 2011-08-11 | SMS:o


Last edited by Franklin52; 02-16-2012 at 07:34 AM.. Reason: Please use code tags for code and data samples, thank you
# 4  
Old 02-16-2012
Code:
awk 'NR==FNR{a[$1];next}FNR==1{f=FILENAME "_new"}($1 in a){print $0 > f}' F A B C D E

Your results will be in the files
A_new
B_new
C_new
...

Of course you can specify abolute path.
Code:
awk 'NR==FNR{a[$1];next}FNR==1{f=FILENAME "_new"}($1 in a){print $0 > f}' /home/anubha/ed/F /home/anubha/A /home/anubha/B /home/anubha/C /home/anubha/D /home/anubha/E

The awk code can also easyly be adapted to handle better the naming and the location of the target file i just did a quick shot here.


I haven't tried it so it may require some fix but you can give a try to the following

Code:
cd /home/anubha
awk 'BEGIN{n=split("ABCDEGHIJK",b,"");d="/home/anubha/priya"}
NR==FNR{a[$1];next}
FNR==1{for(i=0;++i<=n/2;) if(b[i]==FILENAME) {t=d "/" b[i+5];break}}($1 in a){print $0 > t}' ./ed/F A B C D E


Last edited by ctsgnb; 02-16-2012 at 07:28 AM..
# 5  
Old 02-16-2012
Data

Quote:
Originally Posted by ctsgnb
Code:
awk 'NR==FNR{a[$1];next}FNR==1{f=FILENAME "_new"}($1 in a){print $0 > f}' F A B C D E

Your results will be in the files
A_new
B_new
C_new
...

Of course you can specify abolute path.
Code:
awk 'NR==FNR{a[$1];next}FNR==1{f=FILENAME "_new"}($1 in a){print $0 > f}' /home/anubha/ed/F /home/anubha/A /home/anubha/B /home/anubha/C /home/anubha/D /home/anubha/E

The awk code can also easyly be adapted to handle better the naming and the location of the target file i just did a quick shot here.
i use some crap version of linux....which doesnt hv sed/awk plugin installed in it....i dunno bt code doesnt wrk wid sed/awk...cn u suggst wid grep or fgrep... Smilie
# 6  
Old 02-16-2012
Code:
ksh
cd /home/anubha
for i in A B C D E
do
t=$(echo "$i" | tr 'ABCDE' 'GHIJK')
sed 's/./^&/' /home/anubha/ed/F | grep -f - $i >$t
done

Why not to process them all at once and gather the result in a single file ?

Code:
cd /home/anubha
cat A B C D E >all
fgrep ./ed/F all >result


Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

ksh Script, Reading A File, Grepping A File Contents In Another File

So I'm stumped. First... APOLOGIES... my work is offline in an office that has zero internet connectivity, as required by our client. If need be, I could print out my script attempts and retype them here. But on the off chance... here goes. I have a text file (file_source) of terms, each line... (3 Replies)
Discussion started by: Brusimm
3 Replies

2. Shell Programming and Scripting

Shell script (sh file) logic to compare contents of one file with another file and output to file

Shell script logic Hi I have 2 input files like with file 1 content as (file1) "BRGTEST-242" a.txt "BRGTEST-240" a.txt "BRGTEST-219" e.txt File 2 contents as fle(2) "BRGTEST-244" a.txt "BRGTEST-244" b.txt "BRGTEST-231" c.txt "BRGTEST-231" d.txt "BRGTEST-221" e.txt I want to get... (22 Replies)
Discussion started by: pottic
22 Replies

3. Shell Programming and Scripting

What is the function of the following lines at the top of a shell script file: Directory and Script?

The file starts like this: Directory: <path to the script> Script: <script fife name> #!bin/ksh ##Comments <actual script> What is the use of the first two lines in the script? What if I save the file without them? What will be the effect? They are not comments. Im very new to this,... (4 Replies)
Discussion started by: remytom
4 Replies

4. Shell Programming and Scripting

Need output of script on screen and file with correct return status of the called script.

Hi, I am trying to capture logs of the script in the file as well as on the screen. I have used exec and tee command for this. While using exec command I am getting the correct output in the file but, script output is not getting displayed on the screen as it get executed. Below is my sample... (14 Replies)
Discussion started by: Prathmesh
14 Replies

5. Shell Programming and Scripting

Call a Perl script within a bash script and store the ouput in a .txt file

I'm attempting to write a bash script that will create a network between virtual machines. It accepts three arguments: an RSpec that describes the network topology, and two list of machines (servers and clients). I have a (working) Perl script that I want to call. This Perl script takes an RSpec... (6 Replies)
Discussion started by: mecaka
6 Replies

6. Shell Programming and Scripting

Script to call a menu script and redirect each option to a text file

Hello, I want to design a script that will call an existing menu script and select options one by one and redirict the out put to a file. For example;- In the script MENU.sh there are 10 options i want to design a script MENU2.sh that will select option 2 3 4 6 7 10 and redirict the output... (4 Replies)
Discussion started by: spradha
4 Replies

7. Shell Programming and Scripting

Script to read a log file and run 2nd script if the dates match

# cat /tmp/checkdate.log SQL*Plus: Release 11.2.0.1.0 Production on Mon Sep 17 22:49:00 2012 Copyright (c) 1982, 2009, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production FIRST_TIME NEXT_TIME... (1 Reply)
Discussion started by: SarwalR
1 Replies

8. Shell Programming and Scripting

Need some Help for file filteration and saving the output in other directory

Hi all........ Plss do help me.......in a big trouble... :wall::wall::wall: I have 3 directories named as :1. /home/shuchi/source 2./home/shuchi/destination 3./home/shuchi/filter now the problem is /home/shuchi/source has say 2 files with extension .txt as given below : A.txt msisdn ... (5 Replies)
Discussion started by: ektubbe
5 Replies

9. Shell Programming and Scripting

Need some Help for file filteration and saving the output in other directory using grep....plz ...

Hi all........ Plss do help me.......in a big trouble... :wall::wall::wall: I have 3 directories named as :1. /home/shuchi/source 2./home/shuchi/destination 3./home/shuchi/filter now the problem is /home/shuchi/source has say 2 files with extension .txt as given below : A.txt Code: ... (0 Replies)
Discussion started by: ektubbe
0 Replies

10. Shell Programming and Scripting

Reading file names from a file and executing the relative file from shell script

Hi How can i dynamically read files names from a list file and execute them from a single shell script. Please help its urgent Thanks in Advance (4 Replies)
Discussion started by: anushilrai
4 Replies

Featured Tech Videos