Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Simplifying awk/sed short pipeline Post 303007684 by Xterra on Monday 20th of November 2017 08:49:53 AM
Old 11-20-2017
Quote:
First of all, you must put in 'ticks' not "quotes" around the embedded g/awk code, because in the latter $1 $2 ... are substituted with positional shell parameters. The shell does this before it passes the code string to g/awk.
I get the following message if I use single quotes:
Code:
gawk: '!/EL/{
gawk: ^ invalid char ''' in expression
sed: couldn't write 72 items to stdout: Invalid argument

It runs ok using double quotes

Quote:
Unfortunately your min works only if the first line in output.txt has non-null value, so with your sample data the rest of the algorithm (which I don't really understand) doesn't work out, always printing the input line followed by a zero.
You right! Let me work a bit more on that. Any suggestions to improve the first script so I dont keep using | to stitch it all together?
Code:
sed "1d" input.txt | gawk -F "," "!/EL/{ if ( $4 > 410 && $4 < 570 && $5 > 0.5 ) print $2, $5; else print $2, 0 }" | gawk "{a[$1]+=$2}END{for(i in a){print i, a[i]}}" | sed "s/\([A-z]\)\([1-9]\) /\10\2 /" | sort > output.txt

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help in simplifying a sed command

I need to do a substitution: CPF to ,C,P,F, CPM to ,C,P,M, SPF to ,S,P,F etc. I can do each of them with separate substitutions e.g. s/CPF/,C,P,F/ but I'd like to know if there is a more elegant solution. In general, how can I use the results of the search in the substitution, ... (3 Replies)
Discussion started by: wvdeijk
3 Replies

2. Shell Programming and Scripting

Simplifying the For loop

Is there a way to simplify stating 1 through to 10, for example in a for loop construct? for x in 1 2 3 4 5 6 7 8 9 10 do .... done I have tried (1-10) with no luck.. thanks (2 Replies)
Discussion started by: sirtrancealot
2 Replies

3. Shell Programming and Scripting

simplifying awk

tcpdump -nr testdump|awk '!/:/;gsub(/^+|+$/,""){print $3};a!~$0;{a=$0};{print $3};!/length/;/./;!/11\:/;!/8 7 6 5 4 3 2 1/;!/UDP/{b=$0} END {for (j=i-1; j>=0; ) print b };{for (i=NF; i>0; i--) printf("%s ",i);printf ("\n")}' *NOTE IN j>=0 the ; ) was given a space since a smiley is showing up...... (1 Reply)
Discussion started by: sil
1 Replies

4. Shell Programming and Scripting

Double Spacing complex sed pipeline

my script: FILE="$1" echo "You Entered $FILE" if ; then tmp=$(cat $FILE | sed '/./!d' | sed -n '/regex/,/regex/{/regex/d;p}'| sed -n '/---/,+2!p' | sed -n '/#/!p' | sed 's/^*//' | sed -e\ s/*:// | sed -n '/==> /!p' | sed -n '/--> /!p' | sed -n '/regex/,+1!p' | sed -n '/======/!p' | sed -n... (1 Reply)
Discussion started by: omgsomuchppl
1 Replies

5. UNIX for Advanced & Expert Users

Simplifying my script

Hi, Is there a way to simplify the below script? Because I am having problems executing this if I added this to CRON. Also, you may notice that its objective is to put all information in one file (rm1.txt). And in addition file "sRMR_6.txt" to sRMR_23.txt" changes its information everyday.... (4 Replies)
Discussion started by: vibora
4 Replies

6. Shell Programming and Scripting

Looking for a short way to summarise many sed commands

Hello, I have a large number of sed commands that I execute one after the other, simply because I don't know if there's a shorter way to do it. I hope someone can help me save some time :-) These are my commands: 1.) remove all " in the file: sed -e 's/\"//g' file 2.) insert ( and... (3 Replies)
Discussion started by: Bloomy
3 Replies

7. Shell Programming and Scripting

need help simplifying an if statement

the code below is a small fragment of the actual line, in fact i have about 20 values i'm comparing and want to know if it can be simplified. other than the x.xx.xx format of the value they have nothing in common if || || ; then do this else do this fiany suggestions? (6 Replies)
Discussion started by: crimso
6 Replies

8. Shell Programming and Scripting

Simplifying sed/tr

Hi all, I don't have much experience with shell scripting and I was wondering if there's a shorter way to write this. Basically, given a list of strings separated by new lines, I want to prepend each string with a prefix and separate the strings with commas i.e. stra strb strc becomes... (3 Replies)
Discussion started by: vshan
3 Replies

9. UNIX for Dummies Questions & Answers

Submitting awk script into cluster short.q

Hello, I want to submit my awk script into cluster queue as my job takes about forty minutes to finish so I can not run it on the main node. My awk script is like the following and I have three files. so, I write : qsub -q short.q Myscript.awk file1 file2 file3 It submits the work into... (1 Reply)
Discussion started by: Homa
1 Replies

10. UNIX for Beginners Questions & Answers

Simplifying awk script using multiple "|"

I have the following script: awk -F "," '{ if ( $4 > 450 && $4 < 550 && $5 > 0.5 ) print $2, $5; else print $2, "0" }' test.txt | awk '{a+=$2}END{for(i in a){print i, a}}' | sort -nk 1.2 | sed 1,2d and a bunch of files that look like the test file attached here. I am outputting all... (2 Replies)
Discussion started by: Xterra
2 Replies
SHELL-QUOTE(1)						User Contributed Perl Documentation					    SHELL-QUOTE(1)

NAME
shell-quote - quote arguments for safe use, unmodified in a shell command SYNOPSIS
shell-quote [switch]... arg... DESCRIPTION
shell-quote lets you pass arbitrary strings through the shell so that they won't be changed by the shell. This lets you process commands or files with embedded white space or shell globbing characters safely. Here are a few examples. EXAMPLES
ssh preserving args When running a remote command with ssh, ssh doesn't preserve the separate arguments it receives. It just joins them with spaces and passes them to "$SHELL -c". This doesn't work as intended: ssh host touch 'hi there' # fails It creates 2 files, hi and there. Instead, do this: cmd=`shell-quote touch 'hi there'` ssh host "$cmd" This gives you just 1 file, hi there. process find output It's not ordinarily possible to process an arbitrary list of files output by find with a shell script. Anything you put in $IFS to split up the output could legitimately be in a file's name. Here's how you can do it using shell-quote: eval set -- `find -type f -print0 | xargs -0 shell-quote --` debug shell scripts shell-quote is better than echo for debugging shell scripts. debug() { [ -z "$debug" ] || shell-quote "debug:" "$@" } With echo you can't tell the difference between "debug 'foo bar'" and "debug foo bar", but with shell-quote you can. save a command for later shell-quote can be used to build up a shell command to run later. Say you want the user to be able to give you switches for a command you're going to run. If you don't want the switches to be re-evaluated by the shell (which is usually a good idea, else there are things the user can't pass through), you can do something like this: user_switches= while [ $# != 0 ] do case x$1 in x--pass-through) [ $# -gt 1 ] || die "need an argument for $1" user_switches="$user_switches "`shell-quote -- "$2"` shift;; # process other switches esac shift done # later eval "shell-quote some-command $user_switches my args" OPTIONS
--debug Turn debugging on. --help Show the usage message and die. --version Show the version number and exit. AVAILABILITY
The code is licensed under the GNU GPL. Check http://www.argon.org/~roderick/ or CPAN for updated versions. AUTHOR
Roderick Schertler <roderick@argon.org> perl v5.16.3 2010-06-11 SHELL-QUOTE(1)
All times are GMT -4. The time now is 08:16 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy