Visit Our UNIX and Linux User Community

Top Forums UNIX for Beginners Questions & Answers Advise to print lines before and after patterh match and checking and removing duplicate files Post 303046169 by newbie_01 on Sunday 26th of April 2020 08:43:24 AM
Old 04-26-2020
Hi Rudic

Your suggestion is really cool, it does like what you said where it skips the ones that has been parsed before as per the filesdone control file. I tested it and rename one of the file and re-run the same awk and it only does the one that it has not work on before.

It works on Linux but not on Solaris. On Solaris it gives error

grep: illegal option -- f
Usage: grep [-c|-l|-q] -bhinsvw pattern file . . .
awk: syntax error near line 9
awk: bailing out near line 9

I also tried using /usr/xpg4/bin/grep

awk: syntax error near line 9
awk: bailing out near line 9

The only problem with this approach is that while most of the alert_${sid}* are final, one of them isn't. So there will be several alert_${sid}.log.YYYYMMDDHHMM and one current log that is named alert_${sid}.log. So
$(ls $DIR_PATH/alert_${sid}* | grep -vf filesdone) /dev/null

should parse the others once but should always be parsing alert_${sid}.log. If such is the case, then the search for the current log may or may not always be a duplicate since the CORRUPT string may or may not appear. Not sure if am explaining it correctly, sorry Smilie

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Removing duplicate lines ignore case

hi, I have the following input in file: abc ab a AB b c a C B When I use uniq -u file,the out put file is: abc ab AB c v B C (17 Replies)
Discussion started by: hellsd
17 Replies

2. UNIX for Dummies Questions & Answers

removing duplicate lines from a file

Hi, I am trying to remove duplicate lines from a file. For example the contents of example.txt is: this is a test 2342 this is a test 34343 this is a test 43434 and i want to remove the "this is a test" lines only and end up with the numbers in the file, that is, end up with: 2342... (4 Replies)
Discussion started by: ocelot
4 Replies

3. Shell Programming and Scripting

removing duplicate blank lines

Hi, how to remove the blank lines from the file only If we have more than one blank line. thanks rameez (8 Replies)
Discussion started by: rameezrajas
8 Replies

4. Shell Programming and Scripting

removing the duplicate lines in a file

Hi, I need to concatenate three files in to one destination file.In this if some duplicate data occurs it should be deleted. eg: file1: ----- data1 value1 data2 value2 data3 value3 file2: ----- data1 value1 data4 value4 data5 value5 file3: ----- data1 value1 data4 value4 (3 Replies)
Discussion started by: Sharmila_P
3 Replies

5. Shell Programming and Scripting

Removing duplicates from string (not duplicate lines)

please help me in getting following: Input Desired output x="foo" foo x="foo foo" foo x="foo foo" foo x="foo abc foo" foo abc x="foo foo1 foo2" foo foo1 foo2 I need to remove duplicated from string.. (8 Replies)
Discussion started by: vickylife
8 Replies

6. Shell Programming and Scripting

Removing Duplicate Lines per Section

Hello, I am in need of removing duplicate lines from within a file per section. File: ABC1 012345 header ABC2 7890-000 ABC3 012345 Header Table ABC4 ABC5 593.0000 587.4800 ABC5 593.5000 587.6580 <= dup need to remove ABC5 593.5000 ... (5 Replies)
Discussion started by: petersf
5 Replies

7. Shell Programming and Scripting

removing duplicate lines while maintaing coherence with second file

So I have two files. The first file, file1.txt, has lines of numbers separated by commas. file1.txt 10,2,30,50 22,6,3,15,16,100 73,55 78,40,33,30,11 73,55 99,82,85 22,6,3,15,16,100 The second file, file2.txt, has sentences. file2.txt "the cat is fat" "I like eggs" "fish live in... (6 Replies)
Discussion started by: adrunknarwhal
6 Replies

8. Shell Programming and Scripting

Removing a block of duplicate lines from a file

Hi all, I have a file with the data 1 abc 2 123 3 ; 4 rao 5 bell 6 ; 7 call 8 abc 9 123 10 ; 11 rao 12 bell 13 ; (10 Replies)
Discussion started by: raosr020
10 Replies

9. UNIX for Dummies Questions & Answers

Removing a set of Duplicate lines from a file

Hi, How do i remove a set of duplicate lines from a file. My file contains the lines: abc def ghi abc def ghi jkl mno pqr jkl mno (1 Reply)
Discussion started by: raosr020
1 Replies

10. UNIX for Beginners Questions & Answers

Advise on how to print range of lines above and below a number?

Hi, I have attached an output file which is some kind of database file mapping. It is basically like an allocation mapping of a tablespace and its datafile/s. The output is generated by the SQL script that I found from 401 Authorization Required Excerpts of the file are as below: ... (2 Replies)
Discussion started by: newbie_01
2 Replies
strace(1M)						  System Administration Commands						strace(1M)

strace - print STREAMS trace messages SYNOPSIS
strace [ mid sid level...] DESCRIPTION
strace without arguments writes all STREAMS event trace messages from all drivers and modules to its standard output. These messages are obtained from the STREAMS log driver (see log(7D)). If arguments are provided, they must be in triplets of the form mid, sid, level, where mid is a STREAMS module ID number, sid is a sub-ID number, and level is a tracing priority level. Each triplet indicates that tracing messages are to be received from the given module/driver, sub-ID (usually indicating minor device), and priority level equal to, or less than the given level. The token all may be used for any member to indicate no restriction for that attribute. The format of each trace message output is: <seq> <time> <ticks> <level> <flags> <mid> <sid> <text> <seq> trace sequence number <time> time of message in hh:mm:ss <ticks> time of message in machine ticks since boot <level> tracing priority level <flags> E : message is also in the error log F : indicates a fatal error N : mail was sent to the system administrator (hardcoded as root) <mid> module ID number of source <sid> sub-ID number of source <text> formatted text of the trace message Once initiated, strace will continue to execute until terminated by the user. EXAMPLES
Example 1: A sample output of the strace command: The following example outputs all trace messages from the module or driver whose module ID is 41: strace 41 all all The following example outputs those trace messages from driver or module ID 41 with sub-IDs 0, 1, or 2: strace 41 0 1 41 1 1 41 2 0 Messages from sub-IDs 0 and 1 must have a tracing level less than or equal to 1. Those from sub-ID 2 must have a tracing level of 0. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ SEE ALSO
attributes(5), log(7D) STREAMS Programming Guide NOTES
o There is no restriction to the number of strace processes opening the STREAMS log driver at a time. o The log-driver records the list of the triplets specified in the command invocation, and compares each potential trace message against this list to decide if it should be formatted and sent up to the strace process. Hence, long lists of triplets will have a greater impact on overall STREAMS performance. Running strace will have the most impact on the timing of the modules and drivers generating the trace messages that are sent to the strace process. If trace messages are generated faster than the strace process can handle them, some of the messages will be lost. This last case can be determined by examining the sequence numbers on the trace messages output. SunOS 5.10 4 Oct 1994 strace(1M)

Featured Tech Videos

All times are GMT -4. The time now is 08:00 PM.
Unix & Linux Forums Content Copyright 1993-2021. All Rights Reserved.
Privacy Policy