Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Combining multiple seds into one awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Combining multiple seds into one awk
# 1  
Old 05-15-2016
Combining multiple seds into one awk
i'm trying to optimize my script. i have a lot of instances where i'm doing something like this:

Code:
echo $blah | sed 's~ ~|~g' | sed 's-_space_- -g' | sed 's-_LP_-(-g' | sed 's-_RP_-)-g'

obviously, this is very inefficient.

i know i can combine into one sed command with something this, using a semi colon:


Code:
sed 's/|/=0,--/g ; s/$/=0/g'

unfortunately, this combining of commands in sed is not available on all unix oses. it bails on systems like AIX. i'm looking for a awk solution that would work on any UNIX os it is run on.

any ideas?
# 2  
Old 05-15-2016
Have you considered using multiple -s options to combine your sed expressions into 1 sed?

eg:

Code:
sed -s 's/|/=0,--/g' -s 's/$/=0/g'

At least then, you are sticking with what you are familiar with, and you won't have to do the work to convert sed replacements to awk.
# 3  
Old 05-16-2016
thank you for the response. the solution should work on all UNIX oses. i think awk is the best way to go here.

i tried the sed suggestion you gave on an aix box and this is what i got:
Code:
$ echo $blah | sed -s 's/|/=0,--/g' -s 's/$/=0/g'
sed: Not a recognized flag: s
Usage:  sed [-n] [-u] Script [File ...]
        sed [-n] [-u] [-e Script] ... [-f Script_file] ... [File ...]
$

# 4  
Old 05-16-2016
You can get rid of those errors by changing:
Code:
echo $blah | sed -s 's/|/=0,--/g' -s 's/$/=0/g'

to:
Code:
echo $blah | sed -e 's/|/=0,--/g' -e 's/$/=0/g'

or, more simply:
Code:
echo $blah | sed 's/|/=0,--/g
s/$/=0/g'

But, this is not at all equivalent to your original pipeline of an echo and four sed commands:
Code:
echo $blah | sed 's~ ~|~g' | sed 's-_space_- -g' | sed 's-_LP_-(-g' | sed 's-_RP_-)-g'

which could be replaced by:
Code:
echo $blah | sed -e 's~ ~|~g' -e 's-_space_- -g' -e 's-_LP_-(-g' -e 's-_RP_-)-g'

or, more simply:
Code:
echo $blah | sed 's~ ~|~g
s-_space_- -g
s-_LP_-(-g
s-_RP_-)-g'

This User Gave Thanks to Don Cragun For This Post:
SkySmart
# 5  
Old 05-16-2016
Quote:
Originally Posted by SkySmart
i'm trying to optimize my script. i have a lot of instances where i'm doing something like this:
Code:
echo $blah | sed 's~ ~|~g' | sed 's-_space_- -g' | sed 's-_LP_-(-g' | sed 's-_RP_-)-g'

obviously, this is very inefficient.
i know i can combine into one sed command with something this, using a semi colon:
Code:
sed 's/|/=0,--/g ; s/$/=0/g'

unfortunately, this combining of commands in sed is not available on all unix oses. it bails on systems like AIX. i'm looking for a awk solution that would work on any UNIX os it is run on.
any ideas?
Hello SkySmart,

As you haven't shown the sample Input_file with expected output so by analyzing your code only I am providing the following solution, please do let me know how it goes on same.
Code:
awk '{gsub(/[[:space:]]/,"|",$0);gsub("_space_"," ",$0);gsub("_LP_","(",$0);gsub("_RP_",")",$0);print}' Input_file

Off course I haven't tested it as Input_file is not shown. Please let me know in case this doesn't meet your requirement with sample Input_file and expected output too, I hope this helps.

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
SkySmart
# 6  
Old 05-16-2016
The multi-liner sed script is best readable if you put each command on a sepearate line
Code:
echo $blah | sed '
s~ ~|~g
s-_space_- -g
s-_LP_-(-g
s-_RP_-)-g
'

Indention can further improve readability
Code:
echo $blah | sed '
  s~ ~|~g
  s-_space_- -g
  s-_LP_-(-g
  s-_RP_-)-g
'

This User Gave Thanks to MadeInGermany For This Post:
SkySmart
# 7  
Old 05-16-2016
Not directly related to your question, but in bash / ksh93 / zsh, parameter expansion would be quicker and more efficient than using external programs to modify a variable..
If those boxes all contain such a shell then that might be a better option.

Code:
blah1=${blah// /|}
blah2=${blah1//_space_/ }
blah3=${blah2//_LP_/(}
blah4=${blah3//_RP_/)}
echo "$blah4"

If you need to repeat that same process in you script, you can also put it in a shell function


--
note: echo $blah should be : echo "$blah"
The quotes are to protect the content from field splitting and globbing by the shell
Last edited by Scrutinizer; 05-16-2016 at 04:58 AM..
This User Gave Thanks to Scrutinizer For This Post:
SkySmart
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Combining multiple greps

I'm trying to learn about regular expressions. Let's say I want to list all the files in /usr/bin beginning with "p", ending with "x", and containing an "a". I know this works:ls | grep ^p | grep x$ | grep abut I'm thinking there must be a way to do it without typing grep three times. Some of my... (9 Replies)
Discussion started by: Xubuntu56
9 Replies

2. UNIX for Beginners Questions & Answers

Combining multiple files into one

Hello Everyone, I have 4 different files (one column in each) that I'm trying to combine into 1 file with four columns. Having issues trying to get the columns to format properly. I have tried the following: paste file1 file2 file3 file4 | column -s $'\t' -t > results.txt paste file1 file2... (1 Reply)
Discussion started by: malk71
1 Replies

3. Shell Programming and Scripting

Combining multiple files

I have 2 files. each having 3 coloums 1st field date as 20130322 2nd field time as 05:55 3rd field numberic value File 2 has entries missing for some date time. FILE1 20130322 05:35 2219 20130322 05:40 1809 20130322 05:45 1617 20130322 05:50 ... (2 Replies)
Discussion started by: sandeepkmehra
2 Replies

4. Shell Programming and Scripting

combining multiple sed statements

I need to run a cronjob that will monitor a directory for files with a certain extension, when one appears I then need to run the below scripts How do I go about combining the following sed statements into one script? and also retain the original filename.? sed 's/71502FSC1206/\n&/g' # add a... (2 Replies)
Discussion started by: firefox2k2
2 Replies

5. Shell Programming and Scripting

Combining multiple rows in single row based on certain condition using awk or sed

Hi, I'm using AIX(ksh shell). > cat temp.txt "a","b",0 "c",bc",0 "a1","b1",0 "cc","cb",1 "cc","b2",1 "bb","bc",2 I want the output as: "a","b","c","bc","a1","b1" "cc","cb","cc","b2" "bb","bc" I want to combine multiple lines into single line where third column is same. Is... (1 Reply)
Discussion started by: samuelray
1 Replies

6. Shell Programming and Scripting

Combining multiple variables into new variable

Hello, I am a new joiner to the forum, and have what i hope is a simple question, however I can't seem to find the answer so maybe it is not available within bash scripting. I intend to use the below script to archive files from multiple directories at once by using a loop, and a variable (n)... (10 Replies)
Discussion started by: dring
10 Replies

7. Shell Programming and Scripting

Combining multiple files into one with the same name/different extension

I've been trying to find information in regard to creating a script that will generate HTML files. I currently have a series of files that contain code I need to surround with a <textarea> tag for easy viewing. I have about a thousand files that contain code, one file that contains the HTML code up... (10 Replies)
Discussion started by: 12o
10 Replies

8. Shell Programming and Scripting

Combining multiple commands

Hi Guys, I am looking to optimze these 5 SSH lines to a single SSH to get my machine to not hang! lol! cat hosts.lst | xargs -n1 -t -i echo 'home/util/timeout 6 0 ssh -q {} top -b > util/{}.top &' >> r_query_info cat hosts.lst | xargs -n1 -t -i echo 'home/util/timeout 6 0 ssh -q {} uname -r... (5 Replies)
Discussion started by: wick3dsunny
5 Replies

9. Shell Programming and Scripting

Sed: Combining Multiple Lines into one

Before I ask my actual question, is it going to be a problem that I want to run this process on a 15 Gig file that is ~140 million rows? What I'm trying to do: I have a file that looks like Color,Type,Count,Day Yellow,Full 5 Tuesday Green,Half 6 Wednesday Purple,Half 8 Tuesday ...... (3 Replies)
Discussion started by: goldfish
3 Replies

10. Shell Programming and Scripting

Combining multiple lines

I am fairly new to scripting. But I have been able to extract and format all of my information required into one file. My issue is that one character is on a separate line. I need to be able to add the character to the previous line. ex. abcdefghi 1 bcdefghij 3 cdefghijk 4 need to... (4 Replies)
Discussion started by: DUST
4 Replies
Login or Register to Ask a Question