Visit Our UNIX and Linux User Community


sed inside awk


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers sed inside awk
# 1  
Old 12-01-2014
sed inside awk

What I want to do is delete everything upto the last underscore (_) in column 2.

Code:
 awk '{ $2=$(echo $2 | sed 's/.*_//'); print $0}'


Sed works fine if I echo the string into it, this doesnt work inside awk. What am I doing wrong?

Similarly, how do I store the substring starting with a keyword KEY,

So if my 2nd column is

Code:
WGGG_TTGSGGDS_KEY12-34


then I want to output
Code:
KEY12-34

Code:
In
23 WGGG_TTGSGGDS_KEY12-34 45
26 WGGG_TTGSG67S_KEY14-34 47

Out
23 KEY12-34 45
26 KEY14-34 47

# 2  
Old 12-01-2014
Why should it work inside awk?
1) your construct can't work; you can't use command substitution inside awk.
2) did you consider the sub/gsub functions in awk?
This User Gave Thanks to RudiC For This Post:
# 3  
Old 12-01-2014
RudiC is correct, here is the long version about process substitution. It is a shell construct which works like that: "$( <command> )" is taken, then "command" is executed in a separate subshell and any output it produces is placed instead of the construct prior to evaluating the rest of the line it is part of, very similar to expanding a variable. The following two examples illustrate this and the similarity of the process (showing the steps in which the shell decodes the respective lines):

Code:
var="my text"
echo "$var"
   |
   V
echo my text
   |
   V
my text


Code:
echo $(echo "my text")
   +---> echo "my text"
   |        |
   |        V
   |     my text
   |        |
   | <------+
   |
   V
echo my text
   |
   V
my text

It is clear that such a syntactic device of the shell will not work outside the shell - like inside awk. For the same reasons PASCALs "BEGIN"-statements do not work in C and curly braces not in PASCAL.

Like RudiC rightfully said, awk can perform "global substitutions" (->gsub()) which do about the same as the "s/.../.../" (substitute) command in sed. It even uses the same regexps so the conversion of your sed statement should be pretty straightforward.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
# 4  
Old 12-02-2014
Thank you for great explainations Rudi and Bakunin.

Hello Senhia83,

Following awk solutions may help you in same.

Code:
i-  awk '{gsub(/.*\_/,X,$2);print $0}' Input_file
ii- awk '{match($2,/KEY.*/);$2=substr($2,RSTART,RLENGTH)} 1' Input_file
####Considering that all your data will have word KEY present in 2nd column.

Output will be as follows in both conditions.
Code:
23 KEY12-34 45
26 KEY14-34 47


Thanks,
R. Singh
# 5  
Old 12-02-2014
Quote:
Originally Posted by bakunin
... ... ...
Like RudiC rightfully said, awk can perform "global substitutions" (->gsub()) which do about the same as the "s/.../.../" (substitute) command in sed. It even uses the same regexps so the conversion of your sed statement should be pretty straightforward.
... ... ...
For the RE used in this example, the RE used by sed and the RE used by awk sub(...) are the same. In general, they aren't quite the same. The REs used by sed are defined to be basic regular expressions and the REs used by awk are defined to be extended regular expressions; so in some cases you can't directly use a sed substitute command BRE as an awk sub() or gsub() ERE.

Quote:
Originally Posted by RavinderSingh13
... ... ...
Code:
awk '{gsub(/.*\_/,X,$2);print $0}' Input_file

... ... ...
There is no need for the backslash in this ERE since underscore is not a special character in an ERE (nor in a BRE). And, there can't be more than one match for the ERE .*_ so gsub() isn't needed here; sub() will work just as well. And print $0 is longhand for print, so this can be simplified to:
Code:
awk '{sub(/.*_/,X,$2);print}' Input_file
        or even
awk '{sub(/.*_/,X,$2)}1' Input_file


Last edited by Don Cragun; 12-02-2014 at 01:22 AM.. Reason: Fix typo s/on/one/.
This User Gave Thanks to Don Cragun For This Post:
 

Previous Thread | Next Thread
Test Your Knowledge in Computers #979
Difficulty: Medium
macOS is based on the Unix operating system and on technologies developed between 1985 and 1997 at NeXT.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

sed inside the awk script to replace a string in the array

The requirement is i need to find an array value matching with pattern {5:{ , replace that with 5: and reassign that to same array index and print it. I write something like below and the issue is sed command is not working. If i replace " with "`" the script gives syntax error.how can i... (8 Replies)
Discussion started by: bhagya123
8 Replies

2. Shell Programming and Scripting

awk statement piped inside sed

Hello folks, I have multiple occurrences of the pattern: ).: where is any digit, in various text context but the pattern is unique as this regex. And I need to turn this decimal fraction into an integer (corresponding percent value: the range of 0-100). What I'm doing is: cat... (1 Reply)
Discussion started by: roussine
1 Replies

3. Shell Programming and Scripting

Using sed inside system call in awk

HI , I am trying to write a code where if a file system has / at the end then replace it with null but it should not affect root file system / for example if ip is /var/opt/home/ then o/p is /var/opt/home but if ip is / then it should remain /. for this i am using below code but no success... (8 Replies)
Discussion started by: Jcpratap
8 Replies

4. Shell Programming and Scripting

Using sed inside shellscript

Hi All, I am trying to use sed command inside a shell script. The same sed command is working on command line, however its not working while using inside a shell script. From various sources i found , it could be done by using -i option, but its not working as well. sed... (11 Replies)
Discussion started by: gotamp
11 Replies

5. Shell Programming and Scripting

Passing awk variable argument to a script which is being called inside awk

consider the script below sh /opt/hqe/hqapi1-client-5.0.0/bin/hqapi.sh alert list --host=localhost --port=7443 --user=hqadmin --password=hqadmin --secure=true >/tmp/alerts.xml awk -F'' '{for(i=1;i<=NF;i++){ if($i=="Alert id") { if(id!="") if(dt!=""){ cmd="sh someScript.sh... (2 Replies)
Discussion started by: vivek d r
2 Replies

6. Shell Programming and Scripting

HELP with AWK one-liner. Need to employ an If condition inside AWK to check for array variable ?

Hello experts, I'm stuck with this script for three days now. Here's what i need. I need to split a large delimited (,) file into 2 files based on the value present in the last field. Samp: Something.csv bca,adc,asdf,123,12C bca,adc,asdf,123,13C def,adc,asdf,123,12A I need this split... (6 Replies)
Discussion started by: shell_boy23
6 Replies

7. Shell Programming and Scripting

Variable inside sed

Problem: Need to read contents from a file and use that value inside sed as avariable. sample code below. THe below code replaces contents inside file123 for matched string with "$x" value only. but we want the value of x which is read from TextFile.txt to go in there. cat TextFile.txt|while... (3 Replies)
Discussion started by: cvsanthosh
3 Replies

8. Shell Programming and Scripting

sed inside sed for replacing string

My need is : Want to change docBase="/something/something/something" to docBase="/only/this/path/for/all/files" I have some (about 250 files)xml files. In FileOne it contains <Context path="/PPP" displayName="PPP" docBase="/home/me/documents" reloadable="true" crossContext="true">... (1 Reply)
Discussion started by: linuxadmin
1 Replies

9. Shell Programming and Scripting

Need help using sed inside the loop.

Hi, i have written a script. it collects data based on the sql queries executed by it. i have multiple output files. after the output file is made i need to do some cosmetic changes in the files and then store them. i am unable to use sed conditions inside the loop. see below code for... (3 Replies)
Discussion started by: dazdseg
3 Replies

10. Shell Programming and Scripting

Appending string (charachters inside the line) to a fixed width file using awk or sed

Source File: abcdefghijklmnop01qrstuvwxyz abcdefghijklmnop02qrstuvwxyz abcdefghijklmnop03qrstuvwxyz abcdefghijklmnop04qrstuvwxyz abcdefghijklmnop05qrstuvwxyz Whatever characters are in 17-18 on each line of the file, it should be concatenated to the same line at the character number... (6 Replies)
Discussion started by: tamahomekarasu
6 Replies

Featured Tech Videos