Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Simplifying awk/sed short pipeline Post 303007684 by Xterra on Monday 20th of November 2017 08:49:53 AM
Old 11-20-2017
Quote:
First of all, you must put in 'ticks' not "quotes" around the embedded g/awk code, because in the latter $1 $2 ... are substituted with positional shell parameters. The shell does this before it passes the code string to g/awk.
I get the following message if I use single quotes:
Code:
gawk: '!/EL/{
gawk: ^ invalid char ''' in expression
sed: couldn't write 72 items to stdout: Invalid argument

It runs ok using double quotes

Quote:
Unfortunately your min works only if the first line in output.txt has non-null value, so with your sample data the rest of the algorithm (which I don't really understand) doesn't work out, always printing the input line followed by a zero.
You right! Let me work a bit more on that. Any suggestions to improve the first script so I dont keep using | to stitch it all together?
Code:
sed "1d" input.txt | gawk -F "," "!/EL/{ if ( $4 > 410 && $4 < 570 && $5 > 0.5 ) print $2, $5; else print $2, 0 }" | gawk "{a[$1]+=$2}END{for(i in a){print i, a[i]}}" | sed "s/\([A-z]\)\([1-9]\) /\10\2 /" | sort > output.txt

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help in simplifying a sed command

I need to do a substitution: CPF to ,C,P,F, CPM to ,C,P,M, SPF to ,S,P,F etc. I can do each of them with separate substitutions e.g. s/CPF/,C,P,F/ but I'd like to know if there is a more elegant solution. In general, how can I use the results of the search in the substitution, ... (3 Replies)
Discussion started by: wvdeijk
3 Replies

2. Shell Programming and Scripting

Simplifying the For loop

Is there a way to simplify stating 1 through to 10, for example in a for loop construct? for x in 1 2 3 4 5 6 7 8 9 10 do .... done I have tried (1-10) with no luck.. thanks (2 Replies)
Discussion started by: sirtrancealot
2 Replies

3. Shell Programming and Scripting

simplifying awk

tcpdump -nr testdump|awk '!/:/;gsub(/^+|+$/,""){print $3};a!~$0;{a=$0};{print $3};!/length/;/./;!/11\:/;!/8 7 6 5 4 3 2 1/;!/UDP/{b=$0} END {for (j=i-1; j>=0; ) print b };{for (i=NF; i>0; i--) printf("%s ",i);printf ("\n")}' *NOTE IN j>=0 the ; ) was given a space since a smiley is showing up...... (1 Reply)
Discussion started by: sil
1 Replies

4. Shell Programming and Scripting

Double Spacing complex sed pipeline

my script: FILE="$1" echo "You Entered $FILE" if ; then tmp=$(cat $FILE | sed '/./!d' | sed -n '/regex/,/regex/{/regex/d;p}'| sed -n '/---/,+2!p' | sed -n '/#/!p' | sed 's/^*//' | sed -e\ s/*:// | sed -n '/==> /!p' | sed -n '/--> /!p' | sed -n '/regex/,+1!p' | sed -n '/======/!p' | sed -n... (1 Reply)
Discussion started by: omgsomuchppl
1 Replies

5. UNIX for Advanced & Expert Users

Simplifying my script

Hi, Is there a way to simplify the below script? Because I am having problems executing this if I added this to CRON. Also, you may notice that its objective is to put all information in one file (rm1.txt). And in addition file "sRMR_6.txt" to sRMR_23.txt" changes its information everyday.... (4 Replies)
Discussion started by: vibora
4 Replies

6. Shell Programming and Scripting

Looking for a short way to summarise many sed commands

Hello, I have a large number of sed commands that I execute one after the other, simply because I don't know if there's a shorter way to do it. I hope someone can help me save some time :-) These are my commands: 1.) remove all " in the file: sed -e 's/\"//g' file 2.) insert ( and... (3 Replies)
Discussion started by: Bloomy
3 Replies

7. Shell Programming and Scripting

need help simplifying an if statement

the code below is a small fragment of the actual line, in fact i have about 20 values i'm comparing and want to know if it can be simplified. other than the x.xx.xx format of the value they have nothing in common if || || ; then do this else do this fiany suggestions? (6 Replies)
Discussion started by: crimso
6 Replies

8. Shell Programming and Scripting

Simplifying sed/tr

Hi all, I don't have much experience with shell scripting and I was wondering if there's a shorter way to write this. Basically, given a list of strings separated by new lines, I want to prepend each string with a prefix and separate the strings with commas i.e. stra strb strc becomes... (3 Replies)
Discussion started by: vshan
3 Replies

9. UNIX for Dummies Questions & Answers

Submitting awk script into cluster short.q

Hello, I want to submit my awk script into cluster queue as my job takes about forty minutes to finish so I can not run it on the main node. My awk script is like the following and I have three files. so, I write : qsub -q short.q Myscript.awk file1 file2 file3 It submits the work into... (1 Reply)
Discussion started by: Homa
1 Replies

10. UNIX for Beginners Questions & Answers

Simplifying awk script using multiple "|"

I have the following script: awk -F "," '{ if ( $4 > 450 && $4 < 550 && $5 > 0.5 ) print $2, $5; else print $2, "0" }' test.txt | awk '{a+=$2}END{for(i in a){print i, a}}' | sort -nk 1.2 | sed 1,2d and a bunch of files that look like the test file attached here. I am outputting all... (2 Replies)
Discussion started by: Xterra
2 Replies
regex(1F)							   FMLI Commands							 regex(1F)

NAME
regex - match patterns against a string SYNOPSIS
regex [-e] [-v "string"] [pattern template] ... pattern [template] DESCRIPTION
The regex command takes a string from the standard input, and a list of pattern / template pairs, and runs regex() to compare the string against each pattern until there is a match. When a match occurs, regex writes the corresponding template to the standard output and returns TRUE. The last (or only) pattern does not need a template. If that is the pattern that matches the string, the function simply returns TRUE. If no match is found, regex returns FALSE. The argument pattern is a regular expression of the form described in regex(). In most cases, pattern should be enclosed in single quotes to turn off special meanings of characters. Note that only the final pattern in the list may lack a template. The argument template may contain the strings $m0 through $m9, which will be expanded to the part of pattern enclosed in ( ... )$0 through ( ... )$9 constructs (see examples below). Note that if you use this feature, you must be sure to enclose template in single quotes so that FMLI does not expand $m0 through $m9 at parse time. This feature gives regex much of the power of cut(1), paste(1), and grep(1), and some of the capabilities of sed(1). If there is no template, the default is $m0$m1$m2$m3$m4$m5$m6$m7$m8$m9. OPTIONS
The following options are supported: -e Evaluates the corresponding template and writes the result to the standard output. -v "string" Uses string instead of the standard input to match against patterns. EXAMPLES
Example 1 Cutting letters out of a string To cut the 4th through 8th letters out of a string (this example will output strin and return TRUE): `regex -v "my string is nice" '^.{3}(.{5})$0' '$m0'` Example 2 Validating input in a form In a form, to validate input to field 5 as an integer: valid=`regex -v "$F5" '^[0-9]+$'` Example 3 Translating an environment variable in a form In a form, to translate an environment variable which contains one of the numbers 1, 2, 3, 4, 5 to the letters a, b, c, d, e: value=`regex -v "$VAR1" 1 a 2 b 3 c 4 d 5 e '.*' 'Error'` Note the use of the pattern '.*' to mean "anything else". Example 4 Using backquoted expressions In the example below, all three lines constitute a single backquoted expression. This expression, by itself, could be put in a menu defini- tion file. Since backquoted expressions are expanded as they are parsed, and output from a backquoted expression (the cat command, in this example) becomes part of the definition file being parsed, this expression would read /etc/passwd and make a dynamic menu of all the login ids on the system. `cat /etc/passwd | regex '^([^:]*)$0.*$' ' name=$m0 action=`message "$m0 is a user"`'` DIAGNOSTICS
If none of the patterns match, regex returns FALSE, otherwise TRUE. NOTES
Patterns and templates must often be enclosed in single quotes to turn off the special meanings of characters. Especially if you use the $m0 through $m9 variables in the template, since FMLI will expand the variables (usually to "") before regex even sees them. Single characters in character classes (inside []) must be listed before character ranges, otherwise they will not be recognized. For exam- ple, [a-zA-Z_/] will not find underscores (_) or slashes (/), but [_/a-zA-Z] will. The regular expressions accepted by regcmp differ slightly from other utilities (that is, sed, grep, awk, ed, and so forth). regex with the -e option forces subsequent commands to be ignored. In other words, if a backquoted statement appears as follows: `regex -e ...; command1; command2` command1 and command2 would never be executed. However, dividing the expression into two: `regex -e ...``command1; command2` would yield the desired result. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ SEE ALSO
awk(1), cut(1), grep(1), paste(1), sed(1), regcmp(3C), attributes(5) SunOS 5.11 12 Jul 1999 regex(1F)
All times are GMT -4. The time now is 03:01 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy