Sponsored Content
Top Forums Shell Programming and Scripting Grep matched records from huge file Post 302338873 by lathavim on Wednesday 29th of July 2009 06:35:36 AM
Old 07-29-2009
Code:
while read record
do
a=`echo $record | cut -c11-20`
echo $a | grep 1 1>/dev/null
if [ $? -eq 0 ]
then
echo $record >>output
fi
done<input



---------- Post updated at 05:35 AM ---------- Previous update was at 05:30 AM ----------

Or


Code:
 
 awk '{ if( substr($0, 11, 10) ~ /1/ ) { print } }' inputfile

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed, grep, awk, regex -- extracting a matched substring from a file/string

Ok, I'm stumped and can't seem to find relevant info. (I'm not even sure, I might have asked something similar before.): I'm trying to use shell scripting/UNIX commands to extract URLs from a fairly large web page, with a view to ultimately wrapping this in PHP with exec() and including the... (2 Replies)
Discussion started by: ropers
2 Replies

2. Shell Programming and Scripting

deleting multiple records from a huge file at one time

I have a very big file of 5gb size and there are about 50 million records in there. I have to delete the records based on recrord number that I know fromoutside with out opening the file. The record numbers are very random like 5000678, 7890005 etc. Can somebody let me know how i can... (5 Replies)
Discussion started by: dsravan
5 Replies

3. Shell Programming and Scripting

grep all records in a file and get a word count -perl

Hi, I have a file .. file.txt .. i need to get a total record count in the files into a $variable.. im using perl script thanks (4 Replies)
Discussion started by: meghana
4 Replies

4. UNIX for Dummies Questions & Answers

Parsing out records from one huge record

Hi, I have one huge record and know that each record in the file is 550 bytes long. How do I parse out individual records from the single huge record. Thanks, (4 Replies)
Discussion started by: bwrynz1
4 Replies

5. UNIX for Dummies Questions & Answers

Grep specific records from a file of records that are separated by an empty line

Hi everyone. I am a newbie to Linux stuff. I have this kind of problem which couldn't solve alone. I have a text file with records separated by empty lines like this: ID: 20 Name: X Age: 19 ID: 21 Name: Z ID: 22 Email: xxx@yahoo.com Name: Y Age: 19 I want to grep records that... (4 Replies)
Discussion started by: Atrisa
4 Replies

6. HP-UX

Performance issue with 'grep' command for huge file size

I have 2 files; one file (say, details.txt) contains the details of employees and another file (say, emp.txt) has some selected employee names. I am extracting employee details from details.txt by using emp.txt and the corresponding code is: while read line do emp_name=`echo $line` grep -e... (7 Replies)
Discussion started by: arb_1984
7 Replies

7. Shell Programming and Scripting

Grep records out of file

Hi, I have a file where there "Tab" seperated values are present.I need to identify duplicate entries based on column 1 & 6 only . For e.g : I tried using uniq ..but the output is only having one duplicate entry, instead of both the entries.I need both the above entries . uniq -f5... (2 Replies)
Discussion started by: appu2176
2 Replies

8. UNIX for Dummies Questions & Answers

What is the faster way to grep from huge file?

Hi All, I am new to this forum and this is my first post. My requirement is like to optimize the time taken to grep the file with 40000 lines. There are two files FILEA(40000 lines) FILEB(40000 lines). The requirement is like this, both the file will be in the format below... (11 Replies)
Discussion started by: mad man
11 Replies

9. Shell Programming and Scripting

Extract Matched Records from XML

Hi All, I have a requirement to extract para in XML file on the basis of another list file having specific parameters. I will extract these para from XML and import in one scheduler tool. file2 <FOLDER DATACENTER="ControlMserver" VERSION="800" PLATFORM="UNIX" FOLDER_NAME="SH_AP_INT_B01"... (3 Replies)
Discussion started by: looney
3 Replies

10. Shell Programming and Scripting

How to fetch matched records from files between two different directory?

awk 'NR==FNR{arr;next} $0 in arr' /tmp/Data_mismatch.sh /prd/HK/ACCTCARD_20160115.txt edit by bakunin: seems that one CODE-tag got lost somewhere. i corrected that, but please check your posts more carefully. Thank you. (5 Replies)
Discussion started by: suresh_target
5 Replies
Mail::Box::Search::Grep(3pm)				User Contributed Perl Documentation			      Mail::Box::Search::Grep(3pm)

NAME
Mail::Box::Search::Grep - select messages within a mail box like grep does INHERITANCE
Mail::Box::Search::Grep is a Mail::Box::Search is a Mail::Reporter SYNOPSIS
use Mail::Box::Manager; my $mgr = Mail::Box::Manager->new; my $folder = $mgr->open('Inbox'); my $filter = Mail::Box::Search::Grep->new ( label => 'selected' , in => 'BODY', match => qr/abc?d*e/ ); my @msgs = $filter->search($folder); my $filter = Mail::Box::Search::Grep->new ( field => 'To' , match => $my_email ); if($filter->search($message)) {...} DESCRIPTION
Try to find some text strings in the header and footer of messages. Various ways to limit the search to certain header fields, the whole header, only the body, the whole message, but even binary multiparts, are provided for. The name grep is derived from the UNIX tool grep, which means: "Get Regular Expression and Print". Although you can search using regular expressions (the Perl way of them), you do not have to print those as result. METHODS
Constructors Mail::Box::Search::Grep->new(OPTIONS) Create a UNIX-grep like search filter. -Option --Defined in --Default binaries Mail::Box::Search <false> decode Mail::Box::Search <true> delayed Mail::Box::Search <true> deleted Mail::Box::Search <false> deliver undef field undef in Mail::Box::Search <$field ? 'HEAD' : C<'BODY'>> label Mail::Box::Search undef limit Mail::Box::Search 0 log Mail::Reporter 'WARNINGS' logical Mail::Box::Search 'REPLACE' match <required> multiparts Mail::Box::Search <true> trace Mail::Reporter 'WARNINGS' binaries => BOOLEAN decode => BOOLEAN delayed => BOOLEAN deleted => BOOLEAN deliver => undef|CODE|'DELETE'|LABEL|'PRINT'|REF-ARRAY Store the details about where the match was found. The search may take much longer when this feature is enabled. When an ARRAY is specified it will contain a list of references to hashes. Each hash contains the information of one match. A match in a header line will result in a line with fields "message", "part", and "field", where the field is a Mail::Message::Field object. When the match is in the body the hash will contain a "message", "part", "linenr", and "line". In case of a CODE reference, that routine is called for each match. The first argument is this search object and the second a reference to same hash as would be stored in the array. The "PRINT" will call printMatchedHead() or printMatchedBody() when any matching header resp body line was found. The output is minimized by not reprinting the message info on multiple matches in the same message. "DELETE" will flag the message to be deleted in case of a match. When a multipart's part is matched, the whole message will be flagged for deletion. field => undef|STRING|REGEX|CODE Not valid in combination with "in" set to "BODY". The STRING is one full field name (case-insensitive). Use a REGEX to select more than one header line to be scanned. CODE is a routine which is called for each field in the header. The CODE is called with the header as first, and the field as second argument. If the CODE returns true, the message is selected. in => 'HEAD'|'BODY'|'MESSAGE' label => STRING limit => NUMBER log => LEVEL logical => 'REPLACE'|'AND'|'OR'|'NOT'|'AND NOT'|'OR NOT' match => STRING|REGEX|CODE The pattern to be search for can be a REGular EXpression, or a STRING. In both cases, the match succeeds if it is found anywhere within the selected fields. With a CODE reference, that function will be called each field or body-line. When the result is true, the details are delivered. The call formats are $code->($head, $field); # for HEAD searches $code->($body, $linenr, $line); # for BODY searches The $head resp $body are one message's head resp. body object. The $field is a header line which matches. The $line and $linenr tell the matching line in the body. Be warned that when you search in "MESSAGE" the code must accept both formats. multiparts => BOOLEAN trace => LEVEL Searching $obj->inBody(PART, BODY) See "Searching" in Mail::Box::Search $obj->inHead(PART, HEAD) See "Searching" in Mail::Box::Search $obj->search(FOLDER|THREAD|MESSAGE|ARRAY-OF-MESSAGES) See "Searching" in Mail::Box::Search $obj->searchPart(PART) See "Searching" in Mail::Box::Search The Results $obj->printMatch([FILEHANDLE], MATCH) $obj->printMatchedBody(FILEHANDLE, MATCH) $obj->printMatchedHead(FILEHANDLE, MATCH) Error handling $obj->AUTOLOAD() See "Error handling" in Mail::Reporter $obj->addReport(OBJECT) See "Error handling" in Mail::Reporter $obj->defaultTrace([LEVEL]|[LOGLEVEL, TRACELEVEL]|[LEVEL, CALLBACK]) Mail::Box::Search::Grep->defaultTrace([LEVEL]|[LOGLEVEL, TRACELEVEL]|[LEVEL, CALLBACK]) See "Error handling" in Mail::Reporter $obj->errors() See "Error handling" in Mail::Reporter $obj->log([LEVEL [,STRINGS]]) Mail::Box::Search::Grep->log([LEVEL [,STRINGS]]) See "Error handling" in Mail::Reporter $obj->logPriority(LEVEL) Mail::Box::Search::Grep->logPriority(LEVEL) See "Error handling" in Mail::Reporter $obj->logSettings() See "Error handling" in Mail::Reporter $obj->notImplemented() See "Error handling" in Mail::Reporter $obj->report([LEVEL]) See "Error handling" in Mail::Reporter $obj->reportAll([LEVEL]) See "Error handling" in Mail::Reporter $obj->trace([LEVEL]) See "Error handling" in Mail::Reporter $obj->warnings() See "Error handling" in Mail::Reporter Cleanup $obj->DESTROY() See "Cleanup" in Mail::Reporter $obj->inGlobalDestruction() See "Cleanup" in Mail::Reporter DIAGNOSTICS
Error: Package $package does not implement $method. Fatal error: the specific package (or one of its superclasses) does not implement this method where it should. This message means that some other related classes do implement this method however the class at hand does not. Probably you should investigate this and probably inform the author of the package. SEE ALSO
This module is part of Mail-Box distribution version 2.105, built on May 07, 2012. Website: http://perl.overmeer.net/mailbox/ LICENSE
Copyrights 2001-2012 by [Mark Overmeer]. For other contributors see ChangeLog. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See http://www.perl.com/perl/misc/Artistic.html perl v5.14.2 2012-05-07 Mail::Box::Search::Grep(3pm)
All times are GMT -4. The time now is 05:16 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy