How to remove content present in between specific pattern ?

Login or Register to Ask a Question and Join Our Community

How to remove content present in between specific pattern ?

Tags

shell scripts, solved

Login to Discuss or Reply to this Discussion in Our Community

Top Forums Shell Programming and Scripting How to remove content present in between specific pattern ?

06-30-2014

Registered User

16, 0

Join Date: May 2014

Last Activity: 4 July 2014, 3:09 AM EDT

Posts: 16

Thanks Given: 8

Thanked 0 Times in 0 Posts

How to remove content present in between specific pattern ?

Hi,

I have a file with following pattern. We are looking to filter out only specific content from this file.

Code:

 
sample
 
BLAdmins    Server.*
LinuxAdmins    Server.*
Policy Name: Recommended Default ACL Policy
Everyone    ACLPushJob.Read
Everyone    ACLTemplate.Read
Everyone    AuditJob.Read 
 
End of Policy: Recommended Default ACL Policy
 
Policy Name: Admin ACL Policy
BLAdmins    ACLPushJob.Read
BLAdmins    ACLTemplate.Read
BLAdmins    AuditJob.Read 
 
End of Policy: Admin ACL Policy

We wish to write content from sample file to another file by removing everything in between "Policy Name:" and "End of Policy:".

so the expected output will be as below

Code:

 
new_sample
BLAdmins    Server.*
LinuxAdmins    Server.*

Regards,
Litu

Litu19

View Public Profile for Litu19

Find all posts by Litu19

06-30-2014

Moderator

3,689, 1,352

Join Date: Jan 2012

Last Activity: 22 August 2020, 11:29 PM EDT

Location: Galactic Empire

Posts: 3,689

Thanks Given: 268

Thanked 1,352 Times in 1,258 Posts

Code:

awk '/Policy Name/{f=1;next}/End of Policy/{f=0;next}!f&&NF' filename

This User Gave Thanks to Yoda For This Post:

Yoda

View Public Profile for Yoda

Visit Yoda's homepage!

Find all posts by Yoda

06-30-2014

Moderator

12,296, 3,792

Join Date: Nov 2008

Last Activity: 1 January 2021, 1:47 AM EST

Location: Amsterdam

Posts: 12,296

Thanks Given: 679

Thanked 3,792 Times in 3,282 Posts

Try range expressions:

Code:

awk '/Policy Name/,/End of Policy/{next}NF' file

or, preserving the empty lines outside the range:

Code:

sed '/Policy Name/,/End of Policy/d' file

These 2 Users Gave Thanks to Scrutinizer For This Post:

Scrutinizer

View Public Profile for Scrutinizer

Find all posts by Scrutinizer

07-01-2014

Registered User

16, 0

Join Date: May 2014

Last Activity: 4 July 2014, 3:09 AM EDT

Posts: 16

Thanks Given: 8

Thanked 0 Times in 0 Posts

Thanks for your response. I tried and seems like it is working. Please explain the below commands.

Quote:

Originally Posted by Scrutinizer

Try range expressions:

Code:

awk '/Policy Name/,/End of Policy/{next}NF' file

or, preserving the empty lines outside the range:

Code:

sed '/Policy Name/,/End of Policy/d' file

Regards,
Litu

---------- Post updated at 02:53 PM ---------- Previous update was at 02:37 PM ----------

Thanks for your quick response. Would you mind to brief the command.

Quote:

Originally Posted by Yoda

Code:

awk '/Policy Name/{f=1;next}/End of Policy/{f=0;next}!f&&NF' filename

Regards,
Litu

Litu19

View Public Profile for Litu19

Find all posts by Litu19

07-01-2014

Registered User

72, 12

Join Date: Oct 2009

Last Activity: 3 January 2019, 2:49 PM EST

Location: The matrix

Posts: 72

Thanks Given: 127

Thanked 12 Times in 10 Posts

Quote:

Originally Posted by Litu19

Thanks for your response. I tried and seems like it is working. Please explain the below commands.

Regards,
Litu

I could help Scrutinizer trying to explain the example with sed.

Code:

sed '/Policy Name/,/End of Policy/d' file

To find text you put it between / /
So if you want to find some text you can do it like this /Policy Name/

Then, if you separate two different text searches between a comma , then these will be used as delimiters, to perform some action for that matching text.

That means /Search from this start point/,/Until you find this coincidence/

The letter d at the end is the action you are telling sed what to do with all that matching text. d comes from "delete".

So that command could be translated into,

Code:

sed '/A/,/B/d' file

"Find this string A and stop when you find this other match B. Then what you have to do is delete all those strings from the output, so I tell sed to perform this with the d at the end."

Since sed works with lines, sed will delete all the lines that finds from A -> to B.

Code:

sed '/A/,/B/d' file

This User Gave Thanks to Kibou For This Post:

Kibou

View Public Profile for Kibou

Find all posts by Kibou

07-02-2014

Moderator

3,105, 1,603

Join Date: May 2013

Last Activity: 31 August 2020, 1:46 AM EDT

Location: Chennai

Posts: 3,105

Thanks Given: 1,269

Thanked 1,603 Times in 1,369 Posts

Hello,

One more solution for same.

Code:

awk '/Policy/ {a++;} {if(a==0){print $0}} {if(a==2){a=0;}}' remove_policy_keyword_check_test121132

Output will be as follows.

Code:

BLAdmins    Server.*
LinuxAdmins    Server.*

NOTE: Where file name remove_policy_keyword_check_test121132 is the Input file name.

Thanks,
R. Singh
"GOD helps those who help themselves."

RavinderSingh13

View Public Profile for RavinderSingh13

Find all posts by RavinderSingh13

07-02-2014

Registered User

16, 0

Join Date: May 2014

Last Activity: 4 July 2014, 3:09 AM EDT

Posts: 16

Thanks Given: 8

Thanked 0 Times in 0 Posts

Thanks Ravinder.
I am very much new to scripting. Initially I was thinking how I am going to do this, but you guys are really champ.

A big thank you to all for your valuable support.

Regards,
Litu

Litu19

View Public Profile for Litu19

Find all posts by Litu19

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk with sed to combine lines and remove specific odd # pattern from line

In the awk piped to sed below I am trying to format file by removing the odd xxxx_digits and whitespace after, then move the even xxxx_digit to the line above it and add a space between them. There may be multiple lines in file but they are in the same format. The Filename_ID line is the last line...

2. Shell Programming and Scripting

awk to remove mutiple values from specific pattern, leaving a single value

In the awk below I am trying to remove all instances after a ; (semi-colon) or , (comma) in the ANN= pattern. I am using gsub to substitute an empty string in these, so that ANN= is a single value (with only one value in it the one right after the ANN=). Thank you :). I have comented my awk and...

3. Red Hat

Moving of file content to another two files after searching with specific pattern

Hello, Please help me with this!! Thanks in advance!! I have a file named file.gc with the content: 1-- Mon Sep 10 08:53:09 CDT 2012 2revoke connect from FR2261; 3delete from mkt_allow where grantee = 'FR2261'; 4grant connect to FR2261 with '******'; 5alter user FR2261 comment...

4. Shell Programming and Scripting

Help with replace the content of specific pattern

Input file: __<name>AWEETET</name> ____<name_evidence="3"_type="2@#">QEWQE</name> __<name>QWE048</name> ____<name_evidence="3"_type="570">@#@$#545</name> ____<name_evidence="2"_type="351">QWE4</name> Desired output: __<tmp>AWEETET</tmp> ____<name_evidence="3"_type="2@#">QEWQE</name>...

5. Shell Programming and Scripting

Help with remove last text of a file that have specific pattern

Input file matrix-remodelling_associated_8_ aurora_interacting_1_ L20 von_factor_A_domain_1 ATP_containing_3B_ . . Output file matrix-remodelling_associated_8 aurora_interacting_1 L20 von_factor_A_domain_1 ATP_containing_3B . .

6. Shell Programming and Scripting

NAWK to remove lines that matches a specific pattern

Hi, I have requirement that I need to split my input file into two files based on a search pattern "abc" For eg. my input file has below content abc defgh zyx I need file 1 with abc and file2 with defgh zyx I can use grep command to acheive this. But with grep I need...

7. Shell Programming and Scripting

Help with remove duplicate content and only keep the first content detail

Input data_10 SSA data_2 TYUE data_3 PEOCV data_6 SSAT data_21 SSA data_19 TYUEC data_14 TYUE data_15 SSA data_32 PEOCV . . Desired Output data_10 SSA data_2 TYUE data_3 PEOCV data_6 SSAT data_19 TYUEC

8. Shell Programming and Scripting

awk/sed/perl command to delete specific pattern and content above it...

Hi, Below is my input file: Data: 1 Length: 20 Got result. Data: 2 Length: 30 No result. Data: 3 Length: 20

9. Shell Programming and Scripting

Remove specific pattern header and its content problem facing

Input file: >TRACK: Position: 1 TYPE: 1 Pos: SVAVPQRHHPGGTVFREPIIIPAIPRLVPGWNKPIIIGRHAFGDQYRATDRVIPGPGKLE LVYTPVNGEPETVKVYDFQGGGIAQTQYNTDESIRGFAHASFQMALLKGLPLYMSTKNTI LKRYDGRFKDIFQEIYESTYQKDFEAKNLWYEHRLIDDMVAQMIKSEGGFVMALKNYDGD >TRACK: Position: 1 TYPE: 2 Pos: FAHASFQMALLKGLPLYMS...

10. Shell Programming and Scripting

Remove specific content in a file

Hi, I have a file called fl_list consists of files i have to archive. I want to create a exception parm called except_parm, so if it finds the directory it will not archive these files and remove from fl_list. $ cat fl_list /apps/dev/ihub/ready/IA003B/IA003B_Deal_Header_yyyymmdd_hhmmss.txt...

Login or Register to Ask a Question