Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Simplifying awk/sed short pipeline Post 303007684 by Xterra on Monday 20th of November 2017 08:49:53 AM
Old 11-20-2017
Quote:
First of all, you must put in 'ticks' not "quotes" around the embedded g/awk code, because in the latter $1 $2 ... are substituted with positional shell parameters. The shell does this before it passes the code string to g/awk.
I get the following message if I use single quotes:
Code:
gawk: '!/EL/{
gawk: ^ invalid char ''' in expression
sed: couldn't write 72 items to stdout: Invalid argument

It runs ok using double quotes

Quote:
Unfortunately your min works only if the first line in output.txt has non-null value, so with your sample data the rest of the algorithm (which I don't really understand) doesn't work out, always printing the input line followed by a zero.
You right! Let me work a bit more on that. Any suggestions to improve the first script so I dont keep using | to stitch it all together?
Code:
sed "1d" input.txt | gawk -F "," "!/EL/{ if ( $4 > 410 && $4 < 570 && $5 > 0.5 ) print $2, $5; else print $2, 0 }" | gawk "{a[$1]+=$2}END{for(i in a){print i, a[i]}}" | sed "s/\([A-z]\)\([1-9]\) /\10\2 /" | sort > output.txt

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help in simplifying a sed command

I need to do a substitution: CPF to ,C,P,F, CPM to ,C,P,M, SPF to ,S,P,F etc. I can do each of them with separate substitutions e.g. s/CPF/,C,P,F/ but I'd like to know if there is a more elegant solution. In general, how can I use the results of the search in the substitution, ... (3 Replies)
Discussion started by: wvdeijk
3 Replies

2. Shell Programming and Scripting

Simplifying the For loop

Is there a way to simplify stating 1 through to 10, for example in a for loop construct? for x in 1 2 3 4 5 6 7 8 9 10 do .... done I have tried (1-10) with no luck.. thanks (2 Replies)
Discussion started by: sirtrancealot
2 Replies

3. Shell Programming and Scripting

simplifying awk

tcpdump -nr testdump|awk '!/:/;gsub(/^+|+$/,""){print $3};a!~$0;{a=$0};{print $3};!/length/;/./;!/11\:/;!/8 7 6 5 4 3 2 1/;!/UDP/{b=$0} END {for (j=i-1; j>=0; ) print b };{for (i=NF; i>0; i--) printf("%s ",i);printf ("\n")}' *NOTE IN j>=0 the ; ) was given a space since a smiley is showing up...... (1 Reply)
Discussion started by: sil
1 Replies

4. Shell Programming and Scripting

Double Spacing complex sed pipeline

my script: FILE="$1" echo "You Entered $FILE" if ; then tmp=$(cat $FILE | sed '/./!d' | sed -n '/regex/,/regex/{/regex/d;p}'| sed -n '/---/,+2!p' | sed -n '/#/!p' | sed 's/^*//' | sed -e\ s/*:// | sed -n '/==> /!p' | sed -n '/--> /!p' | sed -n '/regex/,+1!p' | sed -n '/======/!p' | sed -n... (1 Reply)
Discussion started by: omgsomuchppl
1 Replies

5. UNIX for Advanced & Expert Users

Simplifying my script

Hi, Is there a way to simplify the below script? Because I am having problems executing this if I added this to CRON. Also, you may notice that its objective is to put all information in one file (rm1.txt). And in addition file "sRMR_6.txt" to sRMR_23.txt" changes its information everyday.... (4 Replies)
Discussion started by: vibora
4 Replies

6. Shell Programming and Scripting

Looking for a short way to summarise many sed commands

Hello, I have a large number of sed commands that I execute one after the other, simply because I don't know if there's a shorter way to do it. I hope someone can help me save some time :-) These are my commands: 1.) remove all " in the file: sed -e 's/\"//g' file 2.) insert ( and... (3 Replies)
Discussion started by: Bloomy
3 Replies

7. Shell Programming and Scripting

need help simplifying an if statement

the code below is a small fragment of the actual line, in fact i have about 20 values i'm comparing and want to know if it can be simplified. other than the x.xx.xx format of the value they have nothing in common if || || ; then do this else do this fiany suggestions? (6 Replies)
Discussion started by: crimso
6 Replies

8. Shell Programming and Scripting

Simplifying sed/tr

Hi all, I don't have much experience with shell scripting and I was wondering if there's a shorter way to write this. Basically, given a list of strings separated by new lines, I want to prepend each string with a prefix and separate the strings with commas i.e. stra strb strc becomes... (3 Replies)
Discussion started by: vshan
3 Replies

9. UNIX for Dummies Questions & Answers

Submitting awk script into cluster short.q

Hello, I want to submit my awk script into cluster queue as my job takes about forty minutes to finish so I can not run it on the main node. My awk script is like the following and I have three files. so, I write : qsub -q short.q Myscript.awk file1 file2 file3 It submits the work into... (1 Reply)
Discussion started by: Homa
1 Replies

10. UNIX for Beginners Questions & Answers

Simplifying awk script using multiple "|"

I have the following script: awk -F "," '{ if ( $4 > 450 && $4 < 550 && $5 > 0.5 ) print $2, $5; else print $2, "0" }' test.txt | awk '{a+=$2}END{for(i in a){print i, a}}' | sort -nk 1.2 | sed 1,2d and a bunch of files that look like the test file attached here. I am outputting all... (2 Replies)
Discussion started by: Xterra
2 Replies
GREP(1) 						      General Commands Manual							   GREP(1)

NAME
grep, g - search a file for a pattern SYNOPSIS
grep [ option ... ] pattern [ file ... ] g [ option ... ] pattern [ file ... ] DESCRIPTION
Grep searches the input files (standard input default) for lines that match the pattern, a regular expression as defined in regexp(7) with the addition of a newline character as an alternative (substitute for |) with lowest precedence. Normally, each line matching the pattern is `selected', and each selected line is copied to the standard output. The options are -c Print only a count of matching lines. -h Do not print file name tags (headers) with output lines. -e The following argument is taken as a pattern. This option makes it easy to specify patterns that might confuse argument parsing, such as -n. -i Ignore alphabetic case distinctions. The implementation folds into lower case all letters in the pattern and input before interpre- tation. Matched lines are printed in their original form. -l (ell) Print the names of files with selected lines; don't print the lines. -L Print the names of files with no selected lines; the converse of -l. -n Mark each printed line with its line number counted in its file. -s Produce no output, but return status. -v Reverse: print lines that do not match the pattern. -f The pattern argument is the name of a file containing regular expressions one per line. -b Don't buffer the output: write each output line as soon as it is discovered. Output lines are tagged by file name when there is more than one input file. (To force this tagging, include /dev/null as a file name argument.) Care should be taken when using the shell metacharacters $*[^|()= and newline in pattern; it is safest to enclose the entire expression in single quotes '...'. An expression starting with '*' will treat the rest of the expression as literal characters. G invokes grep with -n and forces tagging of output lines by file name. If no files are listed, it searches all files matching *.C *.b *.c *.h *.m *.cc *.java *.cgi *.pl *.py *.tex *.ms SOURCE
/src/cmd/grep /bin/g SEE ALSO
ed(1), awk(1), sed(1), sam(1), regexp(7) DIAGNOSTICS
Exit status is null if any lines are selected, or non-null when no lines are selected or an error occurs. GREP(1)
All times are GMT -4. The time now is 01:47 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy