Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk help

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk help
# 1  
Old 07-14-2011
awk help
Hi, I've been trying to do this without awk but it's getting too complicated but I don't know how to use arrays within awk, maybe this is how it could be done best.

I have files something like this that I want to display only lines where the value of the third column is within 2 of each other, in this case I would only want to display lines 3 and 4. Could someone suggest something?

2011-06-25 12:27:59 40 nodea down
2011-06-25 12:28:02 45 nodea up
2011-06-25 12:29:23 70 nodea down
2011-06-25 14:31:14 71 nodea up
2011-06-25 14:31:15 80 nodea down
# 2  
Old 07-14-2011
Try:
Code:
awk 'NR==1{x=$3;y=$0;next}x==$3-1{y=y"\n"$0;x=$3;p=1;next}p{print y;p=0;}{x=$3;y=$0}END{if (p){print y}}' file

# 3  
Old 07-14-2011
The code below works even if there are 3 or more lines with difference less than 3 - see example:
Code:
2011-06-25 12:27:59 40 nodea down
2011-06-25 12:28:02 45 nodea up
2011-06-25 12:29:23 70 nodea down
2011-06-25 14:31:14 71 nodea up
2011-06-25 14:31:14 73 nodea up
2011-06-25 14:31:14 75 nodea up
2011-06-25 14:31:15 80 nodea down

Code:
#!/usr/bin/ksh
typeset -i mDiff mVal3 mPVal3
mPPrint="First_Time"
while read mVal1 mVal2 mVal3 mVal4 mVal5; do
  if [[ "${mPPrint}" = "First_Time" ]]; then
    mPPrint="N"
  else
    mDiff=${mVal3}-${mPVal3}
    if [[ ${mDiff} -le 2 ]]; then
      echo ${mPVal1}' '${mPVal2}' '${mPVal3}' '${mPVal4}' '${mPVal5}
      mPPrint="Y"
    else
      if [[ "${mPPrint}" = "Y" ]]; then
        echo ${mPVal1}' '${mPVal2}' '${mPVal3}' '${mPVal4}' '${mPVal5}
        mPPrint="N"
      fi
    fi
  fi
  mPVal1=${mVal1}
  mPVal2=${mVal2}
  mPVal3=${mVal3}
  mPVal4=${mVal4}
  mPVal5=${mVal5}
done < Input_File
if [[ "${mPPrint}" = "Y" ]]; then
  echo ${mPVal1}' '${mPVal2}' '${mPVal3}' '${mPVal4}' '${mPVal5}
  mPPrint="N"
fi

This User Gave Thanks to Shell_Life For This Post:
mavictoro
# 4  
Old 07-14-2011
Quote:
Originally Posted by Shell_Life
The code below works even if there are 3 or more lines with difference less than 3 - see example:
Code:
2011-06-25 12:27:59 40 nodea down
2011-06-25 12:28:02 45 nodea up
2011-06-25 12:29:23 70 nodea down
2011-06-25 14:31:14 71 nodea up
2011-06-25 14:31:14 73 nodea up
2011-06-25 14:31:14 75 nodea up
2011-06-25 14:31:15 80 nodea down

Code:
#!/usr/bin/ksh
typeset -i mDiff mVal3 mPVal3
mPPrint="First_Time"
while read mVal1 mVal2 mVal3 mVal4 mVal5; do
  if [[ "${mPPrint}" = "First_Time" ]]; then
    mPPrint="N"
  else
    mDiff=${mVal3}-${mPVal3}
    if [[ ${mDiff} -le 2 ]]; then
      echo ${mPVal1}' '${mPVal2}' '${mPVal3}' '${mPVal4}' '${mPVal5}
      mPPrint="Y"
    else
      if [[ "${mPPrint}" = "Y" ]]; then
        echo ${mPVal1}' '${mPVal2}' '${mPVal3}' '${mPVal4}' '${mPVal5}
        mPPrint="N"
      fi
    fi
  fi
  mPVal1=${mVal1}
  mPVal2=${mVal2}
  mPVal3=${mVal3}
  mPVal4=${mVal4}
  mPVal5=${mVal5}
done < Input_File
if [[ "${mPPrint}" = "Y" ]]; then
  echo ${mPVal1}' '${mPVal2}' '${mPVal3}' '${mPVal4}' '${mPVal5}
  mPPrint="N"
fi

I think the same behavior can be accomplished by using this:
Code:
awk 'NR==1{x=$3;y=$0;next}x==$3-1||x==$3-2{y=y"\n"$0;x=$3;p=1;next}p{print y;p=0;}{x=$3;y=$0}END{if (p){print y}}' file

This User Gave Thanks to bartus11 For This Post:
mavictoro
# 5  
Old 07-14-2011
Code:
awk 'END{if(p)f(y)}function f(y){print y}($3-x)<3{y=y RS$0;x=$3;p=1;next}p{f(y);p=0}{x=$3;y=$0}' file

Last edited by danmero; 07-14-2011 at 08:13 PM.. Reason: Correct logical error
This User Gave Thanks to danmero For This Post:
mavictoro
# 6  
Old 07-14-2011
Code:
awk 'NR==1{x=$3;y=$0;next}x==$3-1||x==$3-2{y=y"\n"$0;x=$3;p=1;next}p{print y;p=0;}{x=$3;y=$0}END{if (p){print y}}' file

THis is what I need, thank you bartus11 and everyone who posted replies.
But can you break it down for me a bit? I see the $3-1 OR $3-2 part.
Last edited by Franklin52; 07-15-2011 at 03:31 AM.. Reason: Please use code tags for code and data samples, thank you
# 7  
Old 07-15-2011
An explanation? I'll dissect it for you.

Code:
NR == 1 {
	x=$3;
	y=$0;
	next
}

Init's X (column 3) and Y (entire line) using the first line. Let's go ahead and pretend the first X is 40.

Code:
x==$3-1||x==$3-2 {
	y=y"\n"$0;
	x=$3;
	p=1;
	next
}

If our next line is 41, it will satisfy this condition. 40=41-1. So this will only match if the next record is +1 or +2... hmm, Did you need it to match -1 and -2 as well?
It appends the entire line to Y. It sets P, which you can think of as a print tag, but the "next" will end processing of this record now and not yet print it right away (because maybe a 3rd line or more follows within the range...)

Code:
p {
	print y;
	p=0;
}

Lets say line 3 in our example (41,40,...) has a value of 70. So the previous block with the X=$3-1 ... is not satisfied, but this one is since we did set the P flag already. We'll print whats in our buffer Y (which are the lines containing 41 and 40), and clear that flag and start anew.

Code:
{
	x=$3;
	y=$0
}

Still on our 3rd line of text in my example, X=70, will continue through to here, and be set as the next X to be looked for (in case line 4 is 71 or 72)....

Code:
END {
	if (p) {
		print y
	}
}

If we reach the end of the file and have stuff to print, do it now. Otherwise, we only were actually printing our successful matches at the first non-match and it'd be lost.


Hope this helps.
Last edited by neutronscott; 07-15-2011 at 02:06 PM.. Reason: fixed a code tag
This User Gave Thanks to neutronscott For This Post:
mavictoro
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk output yields error: awk:can't open job_name (Autosys)

Good evening, Im newbie at unix specially with awk From an scheduler program called Autosys i want to extract some data reading an inputfile that comprises jobs names, then formating the output to columns for example 1. This is the inputfile: $ more MapaRep.txt ds_extra_nikira_usuarios... (18 Replies)
Discussion started by: alexcol
18 Replies

2. Shell Programming and Scripting

Pass awk field to a command line executed within awk

Hi, I am trying to pass awk field to a command line executed within awk (need to convert a timestamp into formatted date). All my attempts failed this far. Here's an example. It works fine with timestamp hard-codded into the command echo "1381653229 something" |awk 'BEGIN{cmd="date -d... (4 Replies)
Discussion started by: tuxer
4 Replies

3. Shell Programming and Scripting

Passing awk variable argument to a script which is being called inside awk

consider the script below sh /opt/hqe/hqapi1-client-5.0.0/bin/hqapi.sh alert list --host=localhost --port=7443 --user=hqadmin --password=hqadmin --secure=true >/tmp/alerts.xml awk -F'' '{for(i=1;i<=NF;i++){ if($i=="Alert id") { if(id!="") if(dt!=""){ cmd="sh someScript.sh... (2 Replies)
Discussion started by: vivek d r
2 Replies

4. Shell Programming and Scripting

HELP with AWK one-liner. Need to employ an If condition inside AWK to check for array variable ?

Hello experts, I'm stuck with this script for three days now. Here's what i need. I need to split a large delimited (,) file into 2 files based on the value present in the last field. Samp: Something.csv bca,adc,asdf,123,12C bca,adc,asdf,123,13C def,adc,asdf,123,12A I need this split... (6 Replies)
Discussion started by: shell_boy23
6 Replies

5. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

6. Shell Programming and Scripting

Comparison and editing of files using awk.(And also a possible bug in awk for loop?)

I have two files which I would like to compare and then manipulate in a way. File1: pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2: pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2... (1 Reply)
Discussion started by: linuxkid
1 Replies

7. Shell Programming and Scripting

Problem with awk awk: program limit exceeded: sprintf buffer size=1020

Hi I have many problems with a script. I have a script that formats a text file but always prints the same error when i try to execute it The code is that: { if (NF==17){ print $0 }else{ fields=NF; all=$0; while... (2 Replies)
Discussion started by: fate
2 Replies

8. Shell Programming and Scripting

awk: assign variable with -v didn't work in awk filter

I want to filter 2nd column = 2 using awk $ cat t 1 2 2 4 $ VAR=2 #variable worked in print $ cat t | awk -v ID=$VAR ' { print ID}' 2 2 # but variable didn't work in awk filter $ cat t | awk -v ID=$VAR '$2~/ID/ { print $0}' (2 Replies)
Discussion started by: honglus
2 Replies

9. Shell Programming and Scripting

scripting/awk help : awk sum output is not comming in regular format. Pls advise.

Hi Experts, I am adding a column of numbers with awk , however not getting correct output: # awk '{sum+=$1} END {print sum}' datafile 2.15291e+06 How can I getthe output like : 2152910 Thank you.. # awk '{sum+=$1} END {print sum}' datafile 2.15079e+06 (3 Replies)
Discussion started by: rveri
3 Replies

10. Shell Programming and Scripting

Awk problem: How to express the single quote(') by using awk print function

Actually I got a list of file end with *.txt I want to use the same command apply to all the *.txt Thus I try to find out the fastest way to write those same command in a script and then want to let them run automatics. For example: I got the file below: file1.txt file2.txt file3.txt... (4 Replies)
Discussion started by: patrick87
4 Replies
Login or Register to Ask a Question