Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

extract unique pattern from large text file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting extract unique pattern from large text file
# 15  
Old 07-27-2009
Sorry bro

Just omit -v option from

only use
grep -w "[0-9]\{5\}-" filename
# 16  
Old 07-27-2009
done , and this is the output

linux-kmy7:/home/shiju.joseph/Desktop # grep -wv "[0-9]\{5\}-" text11.txt
123-
123456-
123-
12-
212222-
linux-kmy7:/home/shiju.joseph/Desktop #

I wanted 12345- and 23456- to come in the result,which dint come.



Quote:
Originally Posted by pritish.sas

123-
123456-
12345-
23456-
123-
12-
212222-
Put all these thing in a file and then
grep -wv "[0-9]\{5\}-" filename

Try to fire it again in ur shell
# 17  
Old 07-27-2009
Quote:
Originally Posted by pritish.sas

123-
123456-
12345-
23456-
123-
12-
212222-
Put all these thing in a file and then
grep -wv "[0-9]\{5\}-" filename

Try to fire it again in ur shell
Why do you need to increase the font size? If that is intentional, its not encouraged here.
# 18  
Old 07-27-2009
yes , it worked for the set of data you gave

linux-kmy7:/home/shiju.joseph/Desktop # grep -w "[0-9]\{5\}-" text11.txt
12345-
23456-
linux-kmy7:/home/shiju.joseph/Desktop #

But dint work with my test file with this contents

12345-123213sdfsdfsdsdfsdfsd
65433-2342342342342342342342
45454-4353453453453453453453
34534-34534534534534534534534
576457-32542352345235235234523
42343241-2314234234234234234
2345234523-4523523523452345234523453
23452345234-52345324532452345235
234523452345-234523523452345234523
2345342523452-35234523534252345234
32452345324532-45324523453452345234

and I wanted
12345-123213sdfsdfsdsdfsdfsd
65433-2342342342342342342342
45454-4353453453453453453453
34534-34534534534534534534534 to appear in the result

Whether the tips are working or not I am really happy to see the helping minds from the community , the real power of community , thanks to everyone who responded.

Thanks
Shiju

Quote:
Originally Posted by pritish.sas
Sorry bro

Just omit -v option from

only use
grep -w "[0-9]\{5\}-" filename
# 19  
Old 07-27-2009
Try it , and be cool
I am using opensuse, i tried it's working fine. Just add one * after hypen(-)

grep -w "[0-9]\{5\}-*" filename
# 20  
Old 07-27-2009
Hey..that syntax worked...

linux-kmy7:/home/shiju.joseph/Desktop # grep -w "[0-9]\{5\}-*" text.txt
12345-123213sdfsdfsdsdfsdfsd
65433-2342342342342342342342
45454-4353453453453453453453
34534-34534534534534534534534
linux-kmy7:/home/shiju.joseph/Desktop #

thanks Pritish , thanks for your support.

Shiju

Quote:
Originally Posted by pritish.sas
I am using opensuse, i tried it's working fine. Just add one * after hypen(-)

grep -w "[0-9]\{5\}-*" filename
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

sed awk: split a large file to unique file names

Dear Users, Appreciate your help if you could help me with splitting a large file > 1 million lines with sed or awk. below is the text in the file input file.txt scaffold1 928 929 C/T + scaffold1 942 943 G/C + scaffold1 959 960 C/T +... (6 Replies)
Discussion started by: kapr0001
6 Replies

2. Shell Programming and Scripting

Extract pattern from text

Hi all, I got a txt here and I need to extract all D 8888 44 and D 8888 43 + next field =",g("en")];f._sn&&(f._sn= "og."+f._sn);for(var n in f)l.push("&"),l.push(g(n)),l.push("="),l.push(g(f));l.push("&emsg=");l.push(g(d.name+":"+d.message));var m=l.join("");Ea(m)&&(m=m.substr(0,2E3));c=m;var... (5 Replies)
Discussion started by: stinkefisch
5 Replies

3. Shell Programming and Scripting

Extract all the sentences from a text file that matches a pattern list

Hi I have a big text file. I want to extract all the sentences that matches at least 70% (seventy percent) of the words from each sentence based on a word list called A. Say the format of the text file is as given below: This is the first sentence which consists of fifteen words... (4 Replies)
Discussion started by: my_Perl
4 Replies

4. Shell Programming and Scripting

Extract specific line in an html file starting and ending with specific pattern to a text file

Hi This is my first post and I'm just a beginner. So please be nice to me. I have a couple of html files where a pattern beginning with "http://www.site.com" and ending with "/resource.dat" is present on every 241st line. How do I extract this to a new text file? I have tried sed -n 241,241p... (13 Replies)
Discussion started by: dejavo
13 Replies

5. Shell Programming and Scripting

Extract UNIque records from File

Hi, I have a file with 20GB Pipe Delimited file where i have too many duplicate records. I need an awk script to extract the unique records from the file and put it into another file. Kindly help. Thanks, Arun (1 Reply)
Discussion started by: Arun Mishra
1 Replies

6. UNIX for Dummies Questions & Answers

Extract unique combination of rows from text files

Hi Gurus, I have 100 tab-delimited text files each with 21 columns. I want to extract only 2nd and 5th column from each text file. However, the values in both 2bd and 5th column contain duplicate values but the combination of these values in a row are not duplicate. I want to extract only those... (3 Replies)
Discussion started by: Unilearn
3 Replies

7. UNIX for Dummies Questions & Answers

Extract Unique Values from file

Hello all, I have a file with following sample data 2009-08-26 05:32:01.65 spid5 Process ID 86:214 owns resources that are blocking processes on Scheduler 0. 2009-08-26 05:32:01.65 spid5 Process ID 86:214 owns resources that are blocking processes on Scheduler 0. 2009-08-26... (5 Replies)
Discussion started by: simonsimon
5 Replies

8. Shell Programming and Scripting

sed: Find start of pattern and extract text to end of line, including the pattern

This is my first post, please be nice. I have tried to google and read different tutorials. The task at hand is: Input file input.txt (example) abc123defhij-E-1234jslo 456ujs-W-abXjklp From this file the task is to grep the -E- and -W- strings that are unique and write a new file... (5 Replies)
Discussion started by: TestTomas
5 Replies

9. Shell Programming and Scripting

Need to extract 7 characters immediately after text '19' from a large file.

Hi All!! I have a large file containing millions of record. My purpose is to extract 7 characters immediately after text '19' from this file (including text '19') and save the result in new file. So, my OUTPUT would be as under : 191234561 194567894 192789005 198839408 and so on..... ... (7 Replies)
Discussion started by: parshant_bvcoe
7 Replies

10. Shell Programming and Scripting

Extract pattern from text line

Hi, the text line looks like this: "test1" " " "test2" "test3" "test4" "10" "test 10 12" "00:05:58" "filename.bin" "3.3MB" "/dir/name" "18459" what's the best way to select any of it? So I can for example get only the time or size and so on. I was trying awk -F""" '{print $N}' but... (3 Replies)
Discussion started by: TehOne
3 Replies
Login or Register to Ask a Question