Using sed to execute multiple commands


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using sed to execute multiple commands
# 1  
Old 07-10-2013
Using sed to execute multiple commands

Let's say I have a file called test.out. In this file I want to do the following:
1. Search for DIP-10219 and with this:
2. Remove everything in front of cn=
3. Remove everything after *com
4. Remove duplicate lines
5. Replace ( with \(
6. Replace ) with \)

For 1-3 I have figured out this code:
Code:
sed -rn '/DIP-10219/ s/^.*[^,](cn=.*com).*$/\1/p' test.out

However, I can't figure out how to execeute 4-6 using one sed command. Any thoughts?

Thanks!
# 2  
Old 07-10-2013
You won't be able to do 4. in sed that easily. Duplicate strings, yes, duplicate lines, difficult if not impossible.
5. and 6.:
Code:
sed -rn '/DIP-10219/ {s/^.*[^,](cn=.*com).*$/\1/;s/[(]|[)]/\\&/g;p}' test.out

# 3  
Old 07-10-2013
For multiline problems, you need a looper, a script that has an N, a $ test and a branch t or b, so you can pile up lines in the buffer. Duplicates would need to be sorted to be adjacent, and 'sort -u' or 'uniq' if already sorted, will get them, simpler. You do not have associative arrays like bash, awk, perl in sed to record all and detect dups in an unsorted file.

Don't think of it as one sed command, but one sed instance running a script.

5 and 6 are just this script line: s/[)(]/\\&/g

I find it is better to put looper functionality in a separate sed instance on the pipe. Sometimes, for speed, I chain many sed in a row, so each holds the line the least time.
# 4  
Old 07-10-2013
RudiC - your part works. Thanks!

Another question. If I want to do another search and replace in it, like searching for 'abc' and replacing with 'abc-', how would the code change?

DGPickett - any suggestions for a looper build around RudiC's suggest code? Thanks a bunch!
# 5  
Old 07-10-2013
Add
Code:
; s/abc/&-/g

to above code
# 6  
Old 07-10-2013
Here is a looper to remove more than one consecutive blank lines, which illustrates the concept:
Code:
sed '
  :loop
  $b
  N
  s/^\n$//
  t loop
  P
  s/.*\n//
  b loop
 '

I put my sed script on its own lines for clarity. If you want to check for lines with only spaces and tabs, that scrubbing can be done upstream in one line or inserted here as two lines. That is why I say sed loopers are usually best kept separate from non-loopers. There is no single place to filter and translate single lines in a looper without redundant processing, like substituting on each line twice.

Narrative:
  1. Create a branch target,
  2. if last line branch to end of script (print buffer and exit) as $N usually tosses the last line,
  3. get the next line on end of buffer as '\nLine_2',
  4. if both lines are empty, remove one,
  5. if line was removed branch back to loop,
  6. print the first line,
  7. remove the first line and
  8. branch back to loop.

Last edited by DGPickett; 07-10-2013 at 06:00 PM..
# 7  
Old 07-11-2013
RudiC... That works perfect. Thanks. Didn't know why I didn't figure that out myself Smilie

DGPickett... Thanks for the example. I'm actually trying to remove duplicate lines, not blank lines. Not sure if that makes a difference... How would your code fit in with the code presented here? Would I need to use the first output, send it to file A, and then use a loop outputting to another file?

Here's the code so far:
Code:
sed -rn '/DIP-10219/ {s/^.*[^,](cn=.*com).*$/\1/;s/hbo/&-ns/g;s/[(]|[)]/\\&/g;p}' test.out

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Use sed commands on multiple lines

I have a text file and i want to run 3 sed commands for the lines entered by the user using perl script. I am doing this manually till now. need some help with this The sed commands I have to use are : sed -i "s/{+//" error.txt sed -i "s/+}//" error.txt sed -i "s/\//g" error.txt... (5 Replies)
Discussion started by: utkarshkhanna44
5 Replies

2. Shell Programming and Scripting

Ssh multiple hops to execute commands with arguments

Hi I need to write a script to ssh through several hops (e.g. HostA-HostB-HostC-HostD), where Host A does not have direct assess to HostC ; HostB cannot access HostD directly. when I ssh 3 hops and run command with arg1, arg2 and redirect the output to a file, e.g. HostA> ssh -t HostB ssh -t... (3 Replies)
Discussion started by: chiensh
3 Replies

3. Shell Programming and Scripting

How to execute multiple commands in one shot?

for example: I'm greping the process where i can get the location of the file $ ps -ef | grep LLAWP | awk {'print $9'} | tail -1 /Hostname/ihs/INSTANCE2/conf/WebAgent.conf then I need to display second line of WebAgent.conf file: $ cat /Hostname/ihs/INSTANCE1/conf/WebAgent.conf | head... (2 Replies)
Discussion started by: raghur77
2 Replies

4. Shell Programming and Scripting

connect to multiple servers using SSH and execute commands

Requirement: Run a shell script with below inputs file name checksum path the script should go to multiple servers (around 35) and verify the input cksum and if there is a mismatch display a simple message to the user that cksum verification failed. host details, user id /... (1 Reply)
Discussion started by: amicableperson
1 Replies

5. UNIX for Dummies Questions & Answers

cron used to execute multiple commands

have to run multiple commands at a specified time by the user... (3 Replies)
Discussion started by: hemaa
3 Replies

6. UNIX for Advanced & Expert Users

How to execute multiple unix commands in one session from java

Hi, Iam trying to code in java and wanted to run the commands in the Unix remote servers. I have the following code to run multiple GREP commands in a single session. But when i execute this, the first command executes successfully, whereas from the next line it says "Exception Occured... (1 Reply)
Discussion started by: gravi2020
1 Replies

7. Shell Programming and Scripting

Execute multiple commands in a find

I am checking that a file is older than a reference file that I build with a touch command before processing it. If it is not old enough, I want to sleep for an hour and check again. My problem is if it is old enough to process, I want to exit when I am done, but I cannot find a way to exit... (2 Replies)
Discussion started by: prismtx
2 Replies

8. Shell Programming and Scripting

Can Xargs execute multiple commands of evry input file

Hello , I am trying to print the footer of evry file in the given directory with xargs command like follows ls -1 | xargs -I {} gzcat {} | tail -1 now problem with this is only last file foooter is getting printed as " | tail -1 " is getting executed for the last file. I know this can... (4 Replies)
Discussion started by: nilesrex
4 Replies

9. Shell Programming and Scripting

how do i get my script to execute multiple commands?

New to shell scripting. I can't get my script to execute multiple commands. Here's the code. It's a menu script. #!/bin/ksh clear print "SDE MENU" PS3="SDE MENU, enter choice:" select clean_menu in "tasdedev instance 5151" "orkindev instance 5155" "tasdetst instance 5157" "orkinsys... (1 Reply)
Discussion started by: hvincent
1 Replies

10. Shell Programming and Scripting

multiple sed commands

hello! I have a few sed commands sed '/^$/d' < $1 > tmp.t sed '/^ \{3,\}/d' < tmp.t > tmp1.txt ..... how can I write them in a single line? sed '/^$/d' < $1 > | '/^ \{3,\}/d' < $1 > tmp1.txt any idea? thanks. (5 Replies)
Discussion started by: george_
5 Replies
Login or Register to Ask a Question