Your suggestion is really cool, it does like what you said where it skips the ones that has been parsed before as per the filesdone control file. I tested it and rename one of the file and re-run the same awk and it only does the one that it has not work on before.
It works on Linux but not on Solaris. On Solaris it gives error
Code:
grep: illegal option -- f
Usage: grep [-c|-l|-q] -bhinsvw pattern file . . .
awk: syntax error near line 9
awk: bailing out near line 9
I also tried using /usr/xpg4/bin/grep
Code:
awk: syntax error near line 9
awk: bailing out near line 9
The only problem with this approach is that while most of the alert_${sid}* are final, one of them isn't. So there will be several alert_${sid}.log.YYYYMMDDHHMM and one current log that is named alert_${sid}.log. So
should parse the others once but should always be parsing alert_${sid}.log. If such is the case, then the search for the current log may or may not always be a duplicate since the CORRUPT string may or may not appear. Not sure if am explaining it correctly, sorry
Hi,
I am trying to remove duplicate lines from a file. For example the contents of example.txt is:
this is a test
2342
this is a test
34343
this is a test
43434
and i want to remove the "this is a test" lines only and end up with the numbers in the file, that is, end up with:
2342... (4 Replies)
Hi,
I need to concatenate three files in to one destination file.In this if some duplicate data occurs it should be deleted.
eg:
file1:
-----
data1 value1
data2 value2
data3 value3
file2:
-----
data1 value1
data4 value4
data5 value5
file3:
-----
data1 value1
data4 value4 (3 Replies)
Hello,
I am in need of removing duplicate lines from within a file per section.
File:
ABC1 012345 header
ABC2 7890-000
ABC3 012345 Header Table
ABC4
ABC5 593.0000 587.4800
ABC5 593.5000 587.6580 <= dup need to remove
ABC5 593.5000 ... (5 Replies)
So I have two files. The first file, file1.txt, has lines of numbers separated by commas.
file1.txt
10,2,30,50
22,6,3,15,16,100
73,55
78,40,33,30,11
73,55
99,82,85
22,6,3,15,16,100
The second file, file2.txt, has sentences.
file2.txt
"the cat is fat"
"I like eggs"
"fish live in... (6 Replies)
Hi,
I have attached an output file which is some kind of database file mapping. It is basically like an allocation mapping of a tablespace and its datafile/s.
The output is generated by the SQL script that I found from 401 Authorization Required
Excerpts of the file are as below:
... (2 Replies)
Discussion started by: newbie_01
2 Replies
LEARN ABOUT V7
grep
GREP(1) General Commands Manual GREP(1)NAME
grep, egrep, fgrep - search a file for a pattern
SYNOPSIS
grep [ option ] ... expression [ file ] ...
egrep [ option ] ... [ expression ] [ file ] ...
fgrep [ option ] ... [ strings ] [ file ]
DESCRIPTION
Commands of the grep family search the input files (standard input default) for lines matching a pattern. Normally, each line found is
copied to the standard output; unless the -h flag is used, the file name is shown if there is more than one input file.
Grep patterns are limited regular expressions in the style of ed(1); it uses a compact nondeterministic algorithm. Egrep patterns are full
regular expressions; it uses a fast deterministic algorithm that sometimes needs exponential space. Fgrep patterns are fixed strings; it
is fast and compact.
The following options are recognized.
-v All lines but those matching are printed.
-c Only a count of matching lines is printed.
-l The names of files with matching lines are listed (once) separated by newlines.
-n Each line is preceded by its line number in the file.
-b Each line is preceded by the block number on which it was found. This is sometimes useful in locating disk block numbers by con-
text.
-s No output is produced, only status.
-h Do not print filename headers with output lines.
-y Lower case letters in the pattern will also match upper case letters in the input (grep only).
-e expression
Same as a simple expression argument, but useful when the expression begins with a -.
-f file
The regular expression (egrep) or string list (fgrep) is taken from the file.
-x (Exact) only lines matched in their entirety are printed (fgrep only).
Care should be taken when using the characters $ * [ ^ | ? ' " ( ) and in the expression as they are also meaningful to the Shell. It is
safest to enclose the entire expression argument in single quotes ' '.
Fgrep searches for lines that contain one of the (newline-separated) strings.
Egrep accepts extended regular expressions. In the following description `character' excludes newline:
A followed by a single character matches that character.
The character ^ ($) matches the beginning (end) of a line.
A . matches any character.
A single character not otherwise endowed with special meaning matches that character.
A string enclosed in brackets [] matches any single character from the string. Ranges of ASCII character codes may be abbreviated
as in `a-z0-9'. A ] may occur only as the first character of the string. A literal - must be placed where it can't be mistaken as
a range indicator.
A regular expression followed by * (+, ?) matches a sequence of 0 or more (1 or more, 0 or 1) matches of the regular expression.
Two regular expressions concatenated match a match of the first followed by a match of the second.
Two regular expressions separated by | or newline match either a match for the first or a match for the second.
A regular expression enclosed in parentheses matches a match for the regular expression.
The order of precedence of operators at the same parenthesis level is [] then *+? then concatenation then | and newline.
SEE ALSO ed(1), sed(1), sh(1)DIAGNOSTICS
Exit status is 0 if any matches are found, 1 if none, 2 for syntax errors or inaccessible files.
BUGS
Ideally there should be only one grep, but we don't know a single algorithm that spans a wide enough range of space-time tradeoffs.
Lines are limited to 256 characters; longer lines are truncated.
GREP(1)
04-26-2020
