Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Finding when a file switches direction using awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Finding when a file switches direction using awk
# 1  
Old 08-02-2011
Finding when a file switches direction using awk
Hi guys I have a file that is filled with x,y values:

0.000000 0.00129578
0.000191 0.00272187
0.000381 0.0125676
0.000572 0.0120014
0.000763 0.00203461
0.000954 0.00682248
0.001144 0.00202773
0.001335 0.000840523
0.001526 0.00451419

....5MB of that

I wanted to know if there is a way to modify this awk statement to find out when column two switches directions. What i mean is the number in column two will get bigger then start to get smaller. When this happens I want to print out the value of column one.

I used this to find the maximum value:
Code:
#awk ' { if ( $2 >= .9998 && $2 <= 1.0001 ) print $1 }' fileName.xy

I want to do something like this:
Code:
#awk ' { if ( [1 Row above $2]  >= $2 && [1 Row below $2] <= $2 ) print $1 }' fileName.xy

any ideas how to actually code/specify [1 Row above $2] and [1 Row below $2]?
# 2  
Old 08-02-2011
This will print out the change point rows:

Code:
awk '
    NR == 1 { last = $2; last_row = $0; next; }
    {
        if( $2 < last )
        {
            printf( "%s\n", last_row );
            printf( "%s\n", $0 );
            exit( 0 );
        }

        last = $2;
        last_row = $0;
    }
' inputfile

If the input file is:
Code:
1 2
2 4
3 8
4 10
5 8
4 6

The output from the programme will be:
Code:
4 10
5 8

I think this is what you had in mind. It will handle floating point numbers as well, my simple test was just that: simple.
# 3  
Old 08-02-2011
Code:
awk 'NR==1{last=$2;last_row=$1;getline;cur=$2;cur_row=$1;next} 
     {if (last>=cur&&$2<=cur) print cur_row; 
      last=cur;last_row=cur_row;cur=$2;cur_row=$1}' infile

Last edited by rdcwayx; 08-02-2011 at 09:56 PM..
# 4  
Old 08-02-2011
Niether of them worked; the problem was it seemed to pick random points not the peaks (I plotted them using GMT).

Instead of accepting defeat can I trouble you for a little explanation, if I understood the logic a little better I think I could tweak it. I am still new at awk but my professor loves it, and the more I use it the more I realize how powerful it actually is.


Code:
awk 'NR==1{last=$2;lost_row=$1;getline;cur=$2;cur_row=$1;next}

This is the part I don't quite understand. What exactly does NR == 1 accomplish? Your setting the number of rows equal to one? does a ';' seperate commands? So you set last equal to column 2, lost_row equal to column one.
Why do your run a getline? I looked it up and it is defined as:
getline - returns 1 if it finds a record, and 0 if the end of the file is encountered

Anyways I get the feeling that $1 and $2 are now rows above and below not columns?


Code:
  {if (last>=cur&&$2<=cur) print cur_row; 
      last=cur;last_row=cur_row;cur=$2;cur_row=$1}' infile

This part logically makes sense, and this is what I am trying to accomplish.
# 5  
Old 08-02-2011
Some special variables that AWK sets for each record (usually a line) that it reads:

NR = record number (by default the record separator, RS, is a newline, so NR is often the current line number). NR==1 {...} means to execute the commands in braces if this is the first record that AWK is processing.

NF = after field splitting, the number of fields (columns) in the current record.

$1 = the value of the first field

$2 = the value of the second field

$NF = the value of the last field.

Regards,
Alister
These 2 Users Gave Thanks to alister For This Post:
agama rdcwayx
# 6  
Old 08-02-2011
So that makes sense. I played around with it and have a data set that looks like:
Code:
 1	2	3	4	5	6	7
12	4	5	0	8	5	4
12	98	7	675	98	89	98

if I:
Code:
awk 'NR==3 {print $2}' file1.txt

I get 98. if I make NR == 2 i get 4.

So is there a way to check run an if statement with NR-1?

I want to compare the value directly above and below it?

Code:
awk ' { if ( [NR-1, $2]  <= $2 && [NR+1, $2] <= $2 ) print $1 }' fileName.xy

I need to specify that location make sense? there is a numeric value one row above (NR-1) and in field 2 (which I abbreviated as [NR-1,$2] ) that I want to compare to the current row NR and field 2 (abbreviated $2).

is there a way to specify that in awk?

Once again so grateful for the responses!
# 7  
Old 08-02-2011
In your request, you need three lines compare with, so need record the previous two lines each time.

NR==1{last=$2;last_row=$1;getline;cur=$2;cur_row=$1;next} will read the first two lines. getline will jump to next line easily within one line code.

If I write by this way, you will easily understand
Code:
NR==1{last=$2;last_row=$1} 
NR==2{cur=$2;cur_row=$1}

The rest code will start from line 3 which you already understood.

Test from your sample data, my code is fine. If there are any problems, you need paste some real data for test.
Last edited by rdcwayx; 08-02-2011 at 11:08 PM..
This User Gave Thanks to rdcwayx For This Post:
bflinchum
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Can I please get a push in the right direction with awk/sed?

Hi Guys, I have this task to monitor a linux box. I found a program that displays the parameters that I want and I wrote a little .sh to run that program and record output into a file. The findings look promising but I would like to graph them. My output (for every iteration) looks like... (3 Replies)
Discussion started by: DraxDomax
3 Replies

2. Shell Programming and Scripting

awk print re-direction to a file with timestamp appended

Hello I need to split big xml file into multiple files based on xml declaration. for that i have written one awk 1 liner as below awk '/<?xml\ version/{i++}{print > "outfile."i}' test123.xml this is producing the desired out put. but i want the the currenttimestamp with milliseconds in the... (3 Replies)
Discussion started by: dsdev_123
3 Replies

3. Fedora

In need of some direction

Okay, so I'm not a complete newb when it comes to using Unix/Linux. I've been using Ubuntu for a few years now and I've dipped my toes into a few other distros but now I want to get a bit serious. I'm looking at becoming a sysadmin but the trouble is...I have no idea where to start. What I'm... (1 Reply)
Discussion started by: Tamachan87
1 Replies

4. Shell Programming and Scripting

echo switches

Hello All, I am writing an shell script but abruptly its not able to recognize switches in echo statement. #!/bin/bash top -n 1 -b>ankit host=`hostname` time=`cat ankit|grep load|tr -s " "|cut -d " " -f3` load=`cat ankit|grep load|tr -s " "|cut -d "," -f4|cut -d ":" -f2` ... (3 Replies)
Discussion started by: ajaincv
3 Replies

5. Fedora

Need Direction for extra work ?

Hey , I have become pretty normal, using unix and what not and working around FEDORA 9 I was wondering does anyone have any IDEAS or have anything I should try to build or scripts to write , or possibly know any sites where I could practice some things just so I know I am writing them... (2 Replies)
Discussion started by: Producer
2 Replies

6. Shell Programming and Scripting

re-direction

Say I have a single bin directory with Linux and SunOS executables, like this: bin/myprog_lnx bin/myprog_sun Assume these programs read from stdin and write to stdout and, thus, are meant to be run like this: myprog_lnx < filein > fileout My users may log in from a Linux or Solaris... (3 Replies)
Discussion started by: gsal
3 Replies

7. Shell Programming and Scripting

awk command for finding field in a file

hi guys i have this file with column number 7 as below: 0 1416 49 37 5 3 2 0 0 0 21 0 26 ... (5 Replies)
Discussion started by: npatwardhan
5 Replies

8. Shell Programming and Scripting

awk script required for finding records in 1 file with corresponding another file.

Hi, I have a .txt file (uniqfields.txt) with 3 fields separated by " | " (pipe symbol). This file contains unique values with respect to all these 3 fields taken together. There are about 40,000 SORTED records (rows) in this file. Sample records are given below. 1TVAO|OVEPT|VO... (2 Replies)
Discussion started by: RRVARMA
2 Replies

9. UNIX for Dummies Questions & Answers

New & could use some direction!

First, I just rebuilt/installed my custom kernel & I don't know how to check if it ran properly (I'm fairly sure it did, but I'm looking for reassurance that it loaded the new kernel file). Second, I'd love to get into programming, scripting, whatever, I want my imagination to be the builder &... (2 Replies)
Discussion started by: LazySpoon
2 Replies

10. UNIX for Advanced & Expert Users

Tar switches!!!

Hi, If i want to write my data on several tapes, (more than one tape), what switch(s) i need to use with tar. In other word if my data needs the sapce more than one tape & i don't wanna to compress or ... my data. so is it possible to write up to the end of the tape & it asks to put another... (1 Reply)
Discussion started by: nikk
1 Replies
Login or Register to Ask a Question