01-26-2011
big file
1. Always check that you are using the correct tools. Sed is a stream editor, and is wonderful for making on-the-fly edits. What you want, though, is to select or deselect lines to retain. Finding and filtering lines this way is a 'grep' kinda thing. Used in the form 'fgrep' it finds strings, rather then regular expressions, and is a tool better suited to your purpose.
2. Perl is also capable, but may be overkill for what you want to do.
3. AWK could be made to do the same job, but would be awkward compared to grep. (pun intended: sorry)
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hello All,
I have a .csv file and I have to delete the selcted records stored in a vairable
e.g
echo $lname
7 88 91 94 97 100 103 106
I dont know how to pass the variable name to "sed" for deleting the $lname from a file
can any one help as this is very urgent.
$lname is changing the... (3 Replies)
Discussion started by: 009satya
3 Replies
2. Shell Programming and Scripting
Hi
I have a file & always I need to remove or delete last 2 lines from that file. So in a file if I have 10 lines then it should return me first 8 lines.
Can someone help me? (4 Replies)
Discussion started by: videsh77
4 Replies
3. Shell Programming and Scripting
How do I delete all the lines after the line containing text ***DISCLOSURES*** . I want to delete this line too.
Thank you (2 Replies)
Discussion started by: reachsamir
2 Replies
4. UNIX for Dummies Questions & Answers
I have a file with 65 sets of 35 coordinates, and would like to isolate these coordinates so that I can easily copy the coordinates to another file. The problem is, I've got a 9 line header before each set of coordinates (so each set is 44 lines long). There are a zillion threads out there about... (3 Replies)
Discussion started by: red baron
3 Replies
5. UNIX for Advanced & Expert Users
How I can delete 100 lines anywhere in a file without opening a file and without renaming the file. (11 Replies)
Discussion started by: Nirgude07
11 Replies
6. Shell Programming and Scripting
We have a server that logs transactions to a file. I want to write a script that will delete the first 50 lines of the file daily without renameing the file or moving the file. (8 Replies)
Discussion started by: daveisme
8 Replies
7. Shell Programming and Scripting
I have a command which prints #lines after and before the search string in the huge file
nawk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r;print;c=a}b{r=$0}' b=0 a=10 s="STRING1" FILE
The file is 5 gig big.
It works great and prints 10 lines after the lines which contains search string in... (8 Replies)
Discussion started by: prash184u
8 Replies
8. UNIX for Advanced & Expert Users
Hi,
I need a unix command to delete first n (say 100) lines from a log file. I need to delete some lines from the file without using any temporary file. I found sed -i is an useful command for this but its not supported in my environment( AIX 6.1 ). File size is approx 100MB.
Thanks in... (18 Replies)
Discussion started by: unohu
18 Replies
9. Shell Programming and Scripting
Hi All,
I am trying to get some lines from a file i did it with while-do-loop. since the files are huge it is taking much time. now i want to make it faster.
The requirement is the file will be having 1 million lines.
The format is like below.
##transaction, , , ,blah, blah... (38 Replies)
Discussion started by: mad man
38 Replies
10. UNIX for Beginners Questions & Answers
Dear all,
I have stuck with this problem for some days.
I have a very big file, this file can not open by vi command.
There are 200 loops in this file, in each loop will have one line like this:
GWA quasiparticle energy with Z factor (eV)
And I need 98 lines next after this line.
Is... (6 Replies)
Discussion started by: phamnu
6 Replies
grep(1) General Commands Manual grep(1)
Name
grep, egrep, fgrep - search file for regular expression
Syntax
grep [option...] expression [file...]
egrep [option...] [expression] [file...]
fgrep [option...] [strings] [file]
Description
Commands of the family search the input files (standard input default) for lines matching a pattern. Normally, each line found is copied
to the standard output.
The command patterns are limited regular expressions in the style of which uses a compact nondeterministic algorithm. The command patterns
are full regular expressions. The command uses a fast deterministic algorithm that sometimes needs exponential space. The command pat-
terns are fixed strings. The command is fast and compact.
In all cases the file name is shown if there is more than one input file. Take care when using the characters $ * [ ^ | ( ) and in the
expression because they are also meaningful to the Shell. It is safest to enclose the entire expression argument in single quotes ' '.
The command searches for lines that contain one of the (new line-separated) strings.
The command accepts extended regular expressions. In the following description `character' excludes new line:
A followed by a single character other than new line matches that character.
The character ^ matches the beginning of a line.
The character $ matches the end of a line.
A . (dot) matches any character.
A single character not otherwise endowed with special meaning matches that character.
A string enclosed in brackets [] matches any single character from the string. Ranges of ASCII character codes may be abbreviated
as in `a-z0-9'. A ] may occur only as the first character of the string. A literal - must be placed where it can't be mistaken as
a range indicator.
A regular expression followed by an * (asterisk) matches a sequence of 0 or more matches of the regular expression. A regular
expression followed by a + (plus) matches a sequence of 1 or more matches of the regular expression. A regular expression followed
by a ? (question mark) matches a sequence of 0 or 1 matches of the regular expression.
Two regular expressions concatenated match a match of the first followed by a match of the second.
Two regular expressions separated by | or new line match either a match for the first or a match for the second.
A regular expression enclosed in parentheses matches a match for the regular expression.
The order of precedence of operators at the same parenthesis level is the following: [], then *+?, then concatenation, then | and new
line.
Options
-b Precedes each output line with its block number. This is sometimes useful in locating disk block numbers by context.
-c Produces count of matching lines only.
-e expression
Uses next argument as expression that begins with a minus (-).
-f file Takes regular expression (egrep) or string list (fgrep) from file.
-i Considers upper and lowercase letter identical in making comparisons and only).
-l Lists files with matching lines only once, separated by a new line.
-n Precedes each matching line with its line number.
-s Silent mode and nothing is printed (except error messages). This is useful for checking the error status (see DIAGNOSTICS).
-v Displays all lines that do not match specified expression.
-w Searches for an expression as for a word (as if surrounded by `<' and `>'). For further information, see only.
-x Prints exact lines matched in their entirety only).
Restrictions
Lines are limited to 256 characters; longer lines are truncated.
Diagnostics
Exit status is 0 if any matches are found, 1 if none, 2 for syntax errors or inaccessible files.
See Also
ex(1), sed(1), sh(1)
grep(1)