Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Finding some records with sed command
Top Forums Shell Programming and Scripting Finding some records with sed command Post 302494328 by Aia on Monday 7th of February 2011 01:01:36 AM
Old 02-07-2011
Quote:
Originally Posted by shizik
thanks, but this file very BIG(>20 GB)
I know grep command, but it takes too much time
do you know how get these records by sed command?
I don't know what gave you the impression that sed would do a faster job than grep or awk for the same matter.

The syntax for what you want is:
Code:
sed -n '/matching_pattern/ p' filename

Which it is the same that
Code:
grep 'matching_pattern' filename

awk it is a field processor, that's what it does best and what you want.

instead of:
Code:
awk '{if($6~/(2588300|2580000|2587021)$/)print}'

I would:
Code:
awk '$6 ~ /(2588300|2580000|2587021)/' filename

The if is not necessary and the end of line $ is extra, since you want that pattern in 6th field.

Perhaps, you have to get to terms that 20Gb+ is a lot of data to process, and search takes time, regardless.
Last edited by Aia; 02-07-2011 at 02:15 AM..
This User Gave Thanks to Aia For This Post:
shizik
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

finding null records in data file

I am having a "|" delimited flat file and I have to pick up all the records with the 2nd field having null value. Please suggest. (3 Replies)
Discussion started by: dsravan
3 Replies

2. Programming

Finding number of records in SAS dataset

I am running the following Korn shell script: #!/usr/bin/ksh num_records=`sas "select count(*) from /users/abc/123/sasdata.sas7bdat"` echo "$num_records" The script keeps returning an invalid file error even though I am certain that the file really exists. Does anyone see anything wrong... (1 Reply)
Discussion started by: sasaliasim
1 Replies

3. Shell Programming and Scripting

awk script required for finding records in 1 file with corresponding another file.

Hi, I have a .txt file (uniqfields.txt) with 3 fields separated by " | " (pipe symbol). This file contains unique values with respect to all these 3 fields taken together. There are about 40,000 SORTED records (rows) in this file. Sample records are given below. 1TVAO|OVEPT|VO... (2 Replies)
Discussion started by: RRVARMA
2 Replies

4. Shell Programming and Scripting

How to replicated records using sed

I need to generate exactly 102 duplicates in a file using sed. Here's an example of my input: Grid-ref = 1, 148 Grid-ref = 1, 311 Grid-ref = 1, 428 I need: Grid-ref = 1, 148 Grid-ref = 1, 148 Grid-ref = 1, 148 Grid-ref = 1, 148 ... repeated 102 times, then Grid-ref = 1, 311... (2 Replies)
Discussion started by: Russ
2 Replies

5. UNIX for Dummies Questions & Answers

logging deleted records by sed

Hi, I want to use the sed command to delete some lines in a file and I was wondering whether there is a possibility of knowing which lines are deleted, or at least which line numbers. Thanks (4 Replies)
Discussion started by: vanagreg
4 Replies

6. Shell Programming and Scripting

deleting text records with sed (sed paragraphs)

Hi all, First off, Thank you all for the knowledge I have gleaned from this site! Deleting Records from a text file... sed paragraphs The following code works nearly perfect, however each time it is run on the log file it adds a newline at the head of the file, run it 5 times, it'll have 5... (1 Reply)
Discussion started by: Festus Hagen
1 Replies

7. Shell Programming and Scripting

Finding the records with a specified length

I have a sample txt file which has different variable lengths of 2,10,3,15. What is the command that I need use in order to get the record count that has length '3' Thanks (3 Replies)
Discussion started by: bobby1015
3 Replies

8. Shell Programming and Scripting

Finding missing records and Dups

I have a fixed width file. The records looks something similar to below: Type ID SSN NAME .....AND SOME MORE FIELDS A1 1234 ..... A1 1234 ..... B1 1234 ..... M2 4567 ..... M2 4567 ..... N2 4567 ..... N2 4567 ..... A1 9999 N2 9999 Now if A1 is present then B1 has to be present.... (2 Replies)
Discussion started by: Saanvi1
2 Replies

9. Shell Programming and Scripting

UNIX scripting for finding duplicates and null records in pk columns

Hi, I have a requirement.for eg: i have a text file with pipe symbol as delimiter(|) with 4 columns a,b,c,d. Here a and b are primary key columns.. i want to process that file to find the duplicates and null values are in primary key columns(a,b) . I want to write the unique records in which... (5 Replies)
Discussion started by: praveenraj.1991
5 Replies

10. UNIX for Beginners Questions & Answers

Finding records NOT on another file

I have three files named ALL, MATCH, and DIFF. Match and diff have completely different records included in the "all" file, but the "all" file also has records not in either the Match or Diff files. I know I can sort all three files together, one unique and one without that option to show which... (5 Replies)
Discussion started by: wbport
5 Replies

LEARN ABOUT PLAN9

zgrep

ZGREP(1)						      General Commands Manual							  ZGREP(1)

NAME

       zgrep - search possibly compressed files for a regular expression

SYNOPSIS

       zgrep [ grep_options ] [ -e ] pattern filename...

DESCRIPTION

       Zgrep  invokes  grep  on  compressed  or  gzipped  files.   These  grep	options  will  cause  zgrep  to  terminate  with  an  error  code:
       (-[drRzZ]|--di*|--exc*|--inc*|--rec*|--nu*).  All other options specified are passed directly to grep.  If no file is specified,  then  the
       standard input is decompressed if necessary and fed to grep.  Otherwise the given files are uncompressed if necessary and fed to grep.

       If the GREP environment variable is set, zgrep uses it as the grep program to be invoked.

EXIT CODE

       2 - An option that is not supported was specified.

AUTHOR

       Charles Levert (charles@comm.polymtl.ca)

SEE ALSO

       grep(1), gzexe(1), gzip(1), zdiff(1), zforce(1), zmore(1), znew(1)

																	  ZGREP(1)
All times are GMT -4. The time now is 02:35 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy