Visit Our UNIX and Linux User Community


Need help in filters


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need help in filters
# 1  
Old 11-06-2009
Question Need help in filters

Hi,

I have input data.
9214919702; B5; 1;20070216;
9231590437; BY; 1;20070215;9;20091022;12;20091022;
9211765888; AZ; 1;20080802;1;20080802;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091026;
9252412219; MM; 1;20070217;
9214917135; MM; 1;20070215;
9214917056; B5; 1;20070215;
9219381040; E2; 1;20070215;
9253112376; NM; 1;20071222;2;20071222;9;20091030;12;20091030;

Can I get output like those rows have duplicated columns (red color)

output required
9211765888; AZ; 1;20080802;1;20080802;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091026;

Thanks in Advance.
Suresh
# 2  
Old 11-06-2009
Code:
$
$ cat f1
9214919702; B5; 1;20070216;
9231590437; BY; 1;20070215;9;20091022;12;20091022;
9211765888; AZ; 1;20080802;1;20080802;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091026;
9252412219; MM; 1;20070217;
9214917135; MM; 1;20070215;
9214917056; B5; 1;20070215;
9219381040; E2; 1;20070215;
9253112376; NM; 1;20071222;2;20071222;9;20091030;12;20091030;
$
$ perl -F";" -lane '$F[2]=~s/^\s+//;
                    print if (("$F[2]:$F[3]" eq "$F[4]:$F[5]" || "$F[4]:$F[5]" eq "$F[6]:$F[7]") && ($F[4] ne ""))' f1
9211765888; AZ; 1;20080802;1;20080802;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091026;
$
$

tyler_durden
# 3  
Old 11-06-2009
Code:
while(<DATA>){
	my @tmp=$_=~/([0-9];[0-9]+)/g;
	my %hash = map {$_,1} @tmp;
	my @t=keys %hash;
	print if $#tmp != $#t;
}
__DATA__
9214919702; B5; 1;20070216;
9231590437; BY; 1;20070215;9;20091022;12;20091022;
9211765888; AZ; 1;20080802;1;20080802;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091026;
9252412219; MM; 1;20070217;
9214917135; MM; 1;20070215;
9214917056; B5; 1;20070215;
9219381040; E2; 1;20070215;
9253112376; NM; 1;20071222;2;20071222;9;20091030;12;20091030;

# 4  
Old 11-06-2009
Code:
nawk ' {
        ln = $0
        gsub(" ", "")
        for( i =1 ; i <= NF ; i++ ){
                if( ++f[$i] > 1 && f[$(i - 1)] > 1 )                              
                        printf("%s\n", ln)
        }
        split("", f)
} ' FS=";" OFS=";"


Last edited by steadyonabix; 11-06-2009 at 12:47 PM..
# 5  
Old 11-10-2009
Hi Durden,
I have tried below command, working very fine. But I have columns more than 20. then what is the value in -- $F[4] ne ""))

$ perl -F";" -lane '$F[2]=~s/^\s+//;
print if (("$F[2]:$F[3]" eq "$F[4]:$F[5]" || "$F[4]:$F[5]" eq "$F[6]:$F[7]") && ($F[4] ne ""))' f1

---------- Post updated 11-09-09 at 11:11 PM ---------- Previous update was 11-08-09 at 11:12 PM ----------

Anybody ... plz help
# 6  
Old 11-10-2009
Quote:
Originally Posted by suresh3566
...
But I have columns more than 20. then what is the value in...
...
Sorry this line is not clear.
Post your actual input data and expected output data.

tyler_durden
# 7  
Old 11-10-2009
Hi Durden,

Input file data is like

9211765888; AZ; 1;20080802;1;20080802;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091027;
9252412219; MM; 1;20070217;
9211765888; AZ; 1;20080802;1;20080802;2;20080909;2;20080909;4;20070809;5;20091001;5;20091001;12;20080509;13;20090101 ;14;20091027;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091026;20;20091010;20;20091111

expected OUTPUT - only these rows have duplicate columns.
9211765888; AZ; 1;20080802;1;20080802;14;20091027;
9211765888; AZ; 1;20080802;1;20080802;2;20080909;2;20080909;4;20070809;5;20091001;5;20091001;12;20080509;13;20090101 ;14;20091027;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091026;20;20091010;20;20091111

Previous Thread | Next Thread
Test Your Knowledge in Computers #7
Difficulty: Easy
Linux was invented by Linux Torvalds.
True or False?

8 More Discussions You Might Find Interesting

1. Programming

Creating filters with Python on excel

Hello, I have an excel sheet with 11 tabs. I need to take data from the first tab and write the output to the second tab. The first tab looks like this, starting from Row 3 The filters that needs to be created are 1) keep anything greater than 'POS' 5 and less than 160 AND 2)... (2 Replies)
Discussion started by: nans
2 Replies

2. Programming

Applying filters

I have a value X, a value DX and an odd integer N (say N=9) and want to create an array such that let X = 10, DX = 2 and N = 9 DIST(1) = X - 4 * DX DIST(2) = X - 3 * DX DIST(3) = X - 2 * DX DIST(4) = X - DX DIST(5) = X DIST(6) = X + DX DIST(7) = X + 2 * DX DIST(8) = X + 3 * DX DIST(9)... (2 Replies)
Discussion started by: kristinu
2 Replies

3. UNIX for Advanced & Expert Users

Unix- filters ppt

Hello.. i want a ppt on unix filters.. can anybody gv me d link 4 that?? (1 Reply)
Discussion started by: shweta_babbar
1 Replies

4. UNIX for Dummies Questions & Answers

Difference between filters of ps

Hi I am a newbie to Unix . I am just trying to understand the difference between various filters for ps. Can someoen pelase explain me whta is the difference between using /usr/bin/ps -ef | grep <PID> or <Process name> and /usr/bin/ps -auxwww| grep <PID> or <Process Name>. (1 Reply)
Discussion started by: sillybirdie123
1 Replies

5. UNIX for Dummies Questions & Answers

filters

how to filter one particular row from one text file and copy it in another? (1 Reply)
Discussion started by: rajanandhini
1 Replies

6. UNIX for Dummies Questions & Answers

How can I use filters to extract infos?

I encountered some complicated problems course studies. take this for example: under /home/data/stockdata we have 1999,2000,2001,.......2004 these sub-dirs and, each sub-dir has its mothly(for Jan~Dec) transaction records, i.e. they are all named "foo.txt", like this: Date ... (2 Replies)
Discussion started by: virii
2 Replies

7. Solaris

Solaris lp printer filters

I don't have much experience with solaris printers. I've created a printer using lpadmin. When I tried to print something, I got a message that there a no filters on the server. I went to docs.sun.com and got a script to add a filter (below) for filter in *.fd;do >name=`basename $filter... (1 Reply)
Discussion started by: soliberus
1 Replies

8. UNIX for Dummies Questions & Answers

IP Filters

Anyone know where I can find good documentation for IPF on the Internet? Thanks, Chuck (1 Reply)
Discussion started by: 98_1LE
1 Replies

Featured Tech Videos