Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Remove duplicate line starting with a pattern
Top Forums Shell Programming and Scripting Remove duplicate line starting with a pattern Post 302878638 by varun22486 on Saturday 7th of December 2013 04:13:54 PM
Old 12-07-2013
Remove duplicate line starting with a pattern
HI,

I have the below input file

Code:
  /* ----------------- cmdsDlyStartFWJ -----------------*/
UNIX_JOB CMDS065J
  RUN ANY
  CMDNAME sleep 5
  AGENT CMDSHP
  USER proddata
  RUN MON,TUE,WED,THU,FRI
  DELAYSUB 02:00
  /* "Triggers daily file watcher jobs" */
  ENVAR STDOUT=/home3/cmdsprod/stdout/cmdsDlyStartFWJ
ENDJOB
 
  /* ----------------- cmdsDlyStartFWJ -----------------*/
UNIX_JOB CMDS065J
  RUN ANY
  CMDNAME sleep 5
  AGENT CMDSHP
  USER proddata
  DELAYSUB 02:00
  /* "Triggers daily file watcher jobs" */
  ENVAR STDOUT=/home3/cmdsprod/stdout/cmdsDlyStartFWJ
ENDJOB
 
  /* ----------------- cmdsDlyStartFWJ -----------------*/
 
FILE_TRIGGER CMDS065J
  RUN ANY
  CMDNAME sleep 5
  AGENT CMDSHP
  USER proddata
  RUN MON,TUE,WED,THU,FRI
  DELAYSUB 02:00
  /* "Triggers daily file watcher jobs" */
  ENVAR STDOUT=/home3/cmdsprod/stdout/cmdsDlyStartFWJ
ENDJOB

Code:
UNIX_JOB

or
Code:
FILE_TRIGGER

marks the beiginning of a job and
Code:
ENDJOB

marks the end. If you see 1st one has two entries for RUN..so what i want is to remove the RUN ANY line if there are two RUN statements

output should look like below

Code:
  /* ----------------- cmdsDlyStartFWJ -----------------*/
UNIX_JOB CMDS065J
  CMDNAME sleep 5
  AGENT CMDSHP
  USER proddata
  RUN MON,TUE,WED,THU,FRI
  DELAYSUB 02:00
  /* "Triggers daily file watcher jobs" */
  ENVAR STDOUT=/home3/cmdsprod/stdout/cmdsDlyStartFWJ
ENDJOB
 
  /* ----------------- cmdsDlyStartFWJ -----------------*/
UNIX_JOB CMDS065J
  RUN ANY
  CMDNAME sleep 5
  AGENT CMDSHP
  USER proddata
  DELAYSUB 02:00
  /* "Triggers daily file watcher jobs" */
  ENVAR STDOUT=/home3/cmdsprod/stdout/cmdsDlyStartFWJ
ENDJOB
 
  /* ----------------- cmdsDlyStartFWJ -----------------*/
FILE_TRIGGER CMDS065J
  CMDNAME sleep 5
  AGENT CMDSHP
  USER proddata
  RUN MON,TUE,WED,THU,FRI
  DELAYSUB 02:00
  /* "Triggers daily file watcher jobs" */
  ENVAR STDOUT=/home3/cmdsprod/stdout/cmdsDlyStartFWJ
ENDJOB

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove Duplicate line

Hi, I have a scenario here where I have created a flatfile with the below mentioned information. File as you can see is dispalyed in three columns 1st column is FileNameString 2nd column is Report_Name (this has spaces) 3rd column is Flag Result file needed is, removal of duplicate... (1 Reply)
Discussion started by: Student37
1 Replies

2. UNIX for Dummies Questions & Answers

modify a particular pattern starting from second line of the search pattern

Hi, I am new to this forum and i would like to get help in this issue. I have a file 1.txt as shown: apple banana orange apple grapes banana orange grapes orange .... Now i would like to search for pattern say apple or orange and then put a # at the beginning of the pattern... (2 Replies)
Discussion started by: imas
2 Replies

3. UNIX for Dummies Questions & Answers

modify a particular pattern starting from second line of the search pattern

Hi, I think you ppl did not get my question correctly, let me explain I have 1.txt with following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433 ** ** ** In file 2.txt I have the following entries as shown: ... (1 Reply)
Discussion started by: imas
1 Replies

4. Shell Programming and Scripting

comment/delete a particular pattern starting from second line of the matching pattern

Hi, I have file 1.txt with following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433 ** ** ** In file 2.txt I have the following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433... (4 Replies)
Discussion started by: imas
4 Replies

5. Shell Programming and Scripting

shell script to remove all lines from a file before a line starting with pattern

hi,, i hav a file with many lines.i need to remove all lines before a line begginning with a specific pattern from the file because these lines are not required. Can u help me out with either a perl script or shell script example:- if file initially contains lines: a b c d .1.2 d e f... (2 Replies)
Discussion started by: raksha.s
2 Replies

6. Shell Programming and Scripting

How to remove all words starting from a matching word in a line

Hi Guys, I have a file like this: wwwe 1 ioie ewew yyy uuu 88 erehrlk 4 ihoiwhe lkjhassad lkhsad yyy mmm 45 jhash lhasdhs lkhsdkjsn ouiyrshroi oihoihswodnw oiyhewe yyy ggg 77 I want to remove everything after "yyy" and including "yyy" from each line in the file. So I want:... (2 Replies)
Discussion started by: npatwardhan
2 Replies

7. Shell Programming and Scripting

Remove line starting from space till end.

Hi, I have a code tag, from which i have the below snippet: intelrpt.GetCMB_FB type=ODBC> intelrpt.GetCMB_FB type=SYBASE> I want the output like: intelrpt.GetCMB_FB intelrpt.GetCMB_FB That is remove the lines starting from WHITESPACE till end. Please help. I am new to... (7 Replies)
Discussion started by: anupdas
7 Replies

8. Shell Programming and Scripting

Remove duplicate occurrences of text pattern

Hi folks! I have a file which contains a 1000 lines. On each line i have multiple occurrences ( 26 to be exact ) of pattern folder#/folder#. # is depicting the line number in the file some text here folder1/folder1 some text here folder1/folder1 some text here folder1/folder1 some text... (7 Replies)
Discussion started by: martinsmith
7 Replies

9. UNIX for Beginners Questions & Answers

Grep file starting from pattern matching line

I have a file with a list of references towards the end and want to apply a grep for some string. text .... @unnumbered References @sp 1 @paragraphindent 0 2017. @strong{Chalenski, D.A.}; Wang, K.; Tatanova, Maria; Lopez, Jorge L.; Hatchell, P.; Dutta, P.; @strong{Small airgun... (1 Reply)
Discussion started by: kristinu
1 Replies

10. Shell Programming and Scripting

Remove new line starting with a numeric value and append it to the previous line

Hi, i have a file with multiple entries. After some tests with sed i managed to get the file output as follows: lsn=X-LINK-IN0,apc=661:0,state=avail,avail/links=1/1, 00,2110597,2094790,0,81,529,75649011,56435363, lsn=TM1ITP1-AM1ITP1-LS,apc=500:0,state=avail,avail/links=1/1,... (5 Replies)
Discussion started by: nms
5 Replies
All times are GMT -4. The time now is 02:10 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy