Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

sed and egrep in perl

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sed and egrep in perl
# 1  
Old 03-11-2013
sed and egrep in perl
Hi i have a data file whcih contains the data as follows :

Code:
FH332OY86|AAABBB CCCC DDDA FHLMC 30 8.000|FHLMC|3|30|8.00000000|1986|26.29164289|3.29544844|0.00000000|10.05940539|107.50704264|Mar  8 2013 12:00AM|20130311|D|DA|DAA|DAAC|201302
FH332OY87|AAABBB CCCC DDDA FHLMC 30 8.000|FHLMC|3|30|8.00000000|1987|25.59725593|4.03243180|0.00000000|8.60684509|108.96588429|Mar  8 2013 12:00AM|20130311|D|DA|DAA|DAAC|201302
FH332OY88|AAABBB CCCC DDDA FHLMC 30 8.000|FHLMC|3|30|8.00000000|1988|24.91666667|5.08333333|0.00000000|9.16500000|110.99727000|Mar  8 2013 12:00AM|20130311|D|DA|DAA|DAAC|201302
FN112OY12|AAABBB CCCC DDDA FNMA 10 3.000|FNMA|23|10|3.00000000|2012|0.93679640|8.87642584|0.00000000|3.41606507|105.27375086|Mar  8 2013 12:00AM|20130311|D|DA|DAA|DAAC|201302
FN112OY12|AAABBB CCCC DDDA FNMA 15 3.000|FNMA|5|15|3.00000000|2012|0.73128584|14.05295511|0.00000000|3.43542813|105.18244536|Mar  8 2013 12:00AM|20130311|D|DA|DAA|DAAE|201302
FN114OY10|AAABBB CCCC DDDA FNMA 15 3.500|FNMA|5|15|3.50000000|2010|2.28459632|12.23354007|0.00000000|3.92717106|106.09882219|Mar  8 2013 12:00AM|20130311|D|DA|DAA|DAAE|201302
FN114OY10|AAABBB CCCC DDDA FNMA 10 3.500|FNMA|23|10|3.50000000|2010|2.61217598|7.01039864|0.00000000|4.00370780|105.86328000|Mar  8 2013 12:00AM|20130311|D|DA|DAA|DAAE|201302

now i need to filter out the lines which have the same first column and remove one line out of them which contains "FNMA 15" .in other words i need the final file to be

Code:
FH332OY86|AAABBB CCCC DDDA FHLMC 30 8.000|FHLMC|3|30|8.00000000|1986|26.29164289|3.29544844|0.00000000|10.05940539|107.50704264|Mar  8 2013 12:00AM|20130311|D|DA|DAA|DAAC|201302
FH332OY87|AAABBB CCCC DDDA FHLMC 30 8.000|FHLMC|3|30|8.00000000|1987|25.59725593|4.03243180|0.00000000|8.60684509|108.96588429|Mar  8 2013 12:00AM|20130311|D|DA|DAA|DAAC|201302
FH332OY88|AAABBB CCCC DDDA FHLMC 30 8.000|FHLMC|3|30|8.00000000|1988|24.91666667|5.08333333|0.00000000|9.16500000|110.99727000|Mar  8 2013 12:00AM|20130311|D|DA|DAA|DAAC|201302
FN112OY12|AAABBB CCCC DDDA FNMA 10 3.000|FNMA|23|10|3.00000000|2012|0.93679640|8.87642584|0.00000000|3.41606507|105.27375086|Mar  8 2013 12:00AM|20130311|D|DA|DAA|DAAC|201302
FN114OY10|AAABBB CCCC DDDA FNMA 10 3.500|FNMA|23|10|3.50000000|2010|2.61217598|7.01039864|0.00000000|4.00370780|105.86328000|Mar  8 2013 12:00AM|20130311|D|DA|DAA|DAAE|201302

could you please suggest how to do this ?

The above is only sample data .
# 2  
Old 03-11-2013
try sth like this..

Code:
awk -F\| '!/FNMA 15/{if(!X[$1]++){print}}' file

# 3  
Old 03-11-2013
Code:
cat file | grep -v "FNMA 15" | sort -u -k 1

I was thinking Smilie
Last edited by Franklin52; 04-02-2013 at 08:47 AM.. Reason: Please use code tags
# 4  
Old 03-11-2013
Quote:
Originally Posted by PikK45
cat file | grep -v "FNMA 15" | sort -u -k 1

I was thinking Smilie
useless use of cat

Code:
grep -v "FNMA 15" file | sort -u -k 1

# 5  
Old 03-11-2013
Ah yes!! Thanks there Smilie

Used to it!! Just started to chuck out all the "useless uses" Smilie
# 6  
Old 03-13-2013
I believe i am not clear .

the reuqirement is

1.first to find out which are column 1 values getting repeated .

Code:
 
cat cre_fmr_gen.temp_data_file_gen.dat.CP.CP.prabhu | cut -d "|" -f 1,1 | sort | uniq -c | awk '{if ($1 > 1) print $2;}'
 
2 FN110OY12
2 FN112OY10
2 FN112OY11
2 FN112OY12
2 FN114OY03
2 FN114OY09
2 FN114OY10
2 FN114OY11
2 FN114OY12
2 FN116OY03
2 FN116OY04
2 FN116OY05
2 FN118OY03
2 FN118OY04
2 FN118OY05
2 FN118OY08
2 FN118OY09
2 FN118OY10
2 FN120OY08
2 FN122OY05
2 FN122OY07
2 FN124OY08

2. after this i need to find the reocrd which

1. contain above values as first column and
2. which does not contain the string "FNMA 15"
and delete them from the file.

ex : if the file contains below line it should get deleted.

Code:
FN112OY12|AAABBB CCCC DDDA FNMA 15 3.000|FNMA|5|15|3.00000000|2012|0.73128584|14.05295511|0.00000000|3.43542813|105.18244536|Mar  8 2013 12:00AM|20130311|D|DA|DAA|DAAE|201302

# 7  
Old 03-13-2013
So you want to delete rows having "FNMA 15" only if the first column value is duplicated?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Rewrite sed to perl or run sed in perl

I am having trouble re-writing this sed code sed -nr 's/.*del(+)ins(+).*NC_0{4}(+).*g\.(+)_(+).*/\3\t\4\t\5\t\1\t\2/p' C:/Users/cmccabe/Desktop/Python27/out_position.txt > C:/Users/cmccabe/Desktop/Python27/out_parse.txt in perl Basically, what the code does is parse text from two fields... (12 Replies)
Discussion started by: cmccabe
12 Replies

2. Shell Programming and Scripting

sed and egrep question

Its really 2 questions, but both are pretty basic. Linux Redhat 1. Need to do a search and replace on a file. I need to append '--' (comment out the line) to specific lines based on a wildcard search. So if I Have GRANT SOME_ROLE_OR_USER ... I dont care what comes after that.... (2 Replies)
Discussion started by: guessingo
2 Replies

3. Shell Programming and Scripting

Perl match multiple numbers from a variable similar to egrep

I want to match the number exactly from the variable which has multiple numbers seperated by pipe symbol similar to search in egrep.below is the code which i tried #!/usr/bin/perl my $searchnum = $ARGV; my $num = "148|1|0|256"; print $num; if ($searchnum =~ /$num/) { print "found"; }... (2 Replies)
Discussion started by: kar_333
2 Replies

4. Shell Programming and Scripting

Patterns with egrep/sed/awk?

I have an array with characters, what I want is if there are other characters in the array which I am looking for than take action that is print BAD ARRAY. So far my code just finds characters but instead I want that it should look for other characters. echo "A B C D F" | egrep -o "D | F" O/P... (5 Replies)
Discussion started by: dixits
5 Replies

5. Shell Programming and Scripting

Formatting problem with cat, egrep and perl

Hi guys I'm using the following script to change input file format to another format. some where I'm getting the error. Could you please let me know if you find out? cat input.txt|egrep -v ‘^#'|\ perl -ane ‘if (@F>3){$_=~/(chr.+):(\d+)\ s()/;print $1,”\t”,$2,”\t”,($2+35),”\n”}'\ > output.bed ... (1 Reply)
Discussion started by: repinementer
1 Replies

6. Shell Programming and Scripting

running egrep in perl script ?

Hi there if i run this from the BASH command line, i get a good result # FS="my-box23/account" # zfs list -t filesystem -H | cut -f1 |egrep "^ZPpool1/$FS$" ZP0pool1/my-box23/account which is great, however if I try to run in a perl script populating an array with the result/s, i get... (4 Replies)
Discussion started by: rethink
4 Replies

7. Shell Programming and Scripting

Perl or awk/egrep from big files??

Hi experts. In one thread i have asked you how to grep the string from the below sample file- Unfortunately the script did not gave proper output (it missed many strings). It happened may be i did gave you the proper contents of the file That was the script- "$ perl -00nle'print join... (13 Replies)
Discussion started by: thepurple
13 Replies

8. Shell Programming and Scripting

[Perl] Accessing array elements within a sed command in Perl script

I am trying to use a script to replace the header of each file, whose filename are stored within the array $test, using the sed command within a Perl script as follows: $count = 0; while ( $count < $#test ) { `sed -e 's/BIOGRF 321/BIOGRF 332/g' ${test} > 0`; `cat 0 >... (2 Replies)
Discussion started by: userix
2 Replies

9. UNIX for Dummies Questions & Answers

Egrep cheat sheet anywhere? Looking for meaning of egrep -c

Hi I've been searching google and have not found what egrep -c means. Does anyone know where I can get a cheat sheet or what that -c means? thanks, Linda (2 Replies)
Discussion started by: leelm
2 Replies

10. Shell Programming and Scripting

egrep vs. sed backreference

My egrep outputs this: $ cat html.out|sed -n '/bluetext/s/&nbsp;/ /gp'|egrep '{5}' <span class="bluetext"><b> Lexington Park, MD 20653</b></span> But my backreference \1 is empty. I dont understand why. Can someone clarify? $ cat html.out|sed -n '/bluetext/s/&nbsp;/ /gp'|sed -n... (1 Reply)
Discussion started by: r0sc0
1 Replies
Login or Register to Ask a Question