Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Combining two awk scripts

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Combining two awk scripts
# 1  
Old 12-07-2009
Combining two awk scripts
I have a file like this consisting of blocks separated by > of two number X and T

Code:
>
10 0 
13 5.92346 
16 10.3106 
19 13.9672 
22 16.9838 
25 19.4407 
28 21.4705 
31 23.1547 
34 24.6813 
37 26.0695 
40 27.3611 
43 28.631 
46 29.8366 
49 30.9858 
52 32.0934 
55 33.1458 
58 34.1637 
61 35.1297 
64 36.0253 
67 36.9248 
70 37.8001 
73 38.6296 
76 39.4503 
79 40.2424 
82 40.997 
85 41.7681 
88 42.5001 
91 43.2316 
94 43.9289 
97 44.6221 
100 45.3015 
103 45.9617 
106 46.6138 
109 47.2457 
112 47.8904 
115 48.5016 
118 49.1305 
121 49.7498 
124 50.3272 
127 50.8841 
130 51.472 
133 52.0619 
136 52.6079 
139 53.1586 
142 53.7149 
145 54.2602 
148 54.7771 
151 55.3154 
154 55.8316 
157 56.366 
160 56.8704 
163 57.358 
166 57.8577 
169 58.338 
172 58.8308 
175 59.308 
178 59.7918 
181 60.2547 
184 60.7199 
187 61.1781 
190 61.643 
193 62.1091 
196 62.5579 
199 62.9957 
>
10 5.92346 
13 0 
16 5.92346 
19 10.3106 
22 13.9672 
25 16.9838 
28 19.4407 
31 21.4705 
34 23.1547 
37 24.6813 
40 26.0695 
43 27.3611 
46 28.631 
49 29.8366 
52 30.9858 
55 32.0934 
58 33.1458 
61 34.1637 
64 35.1297 
67 36.0253 
70 36.9248 
73 37.8001 
76 38.6295 
79 39.4503 
82 40.2424 
85 40.9969 
88 41.7681 
91 42.5001 
94 43.2316 
97 43.9289 
100 44.6221 
103 45.3016 
106 45.9617 
109 46.6137 
112 47.2457 
115 47.8904 
118 48.5016 
121 49.1304 
124 49.7498 
127 50.3272 
130 50.8841 
133 51.472 
136 52.0619 
139 52.608 
142 53.1586 
145 53.7149 
148 54.2603 
151 54.7771 
154 55.3154 
157 55.8315 
160 56.3661 
163 56.8704 
166 57.358 
169 57.8597 
172 58.3415 
175 58.8302 
178 59.3081 
181 59.7919 
184 60.2545 
187 60.7185 
190 61.1985 
193 61.6302 
196 62.1146 
199 62.5397 
>
10 10.3106 
13 5.92346 
16 0 
19 5.92346 
22 10.3106 
25 13.9672 
28 16.9838 
31 19.4407 
34 21.4705 
37 23.1547 
40 24.6814 
43 26.0695 
46 27.3611 
49 28.631 
52 29.8366 
55 30.9858 
58 32.0934 
61 33.1458 
64 34.1637 
67 35.1297 
70 36.0253 
73 36.9248 
76 37.8001 
79 38.6295 
82 39.4503 
85 40.2424 
88 40.9969 
91 41.7681 
94 42.5001 
97 43.2316 
100 43.9289 
103 44.6221 
106 45.3016 
109 45.9617 
112 46.6137 
115 47.2457 
118 47.8903 
121 48.5016 
124 49.1304 
127 49.7498 
130 50.3272 
133 50.8841 
136 51.472 
139 52.0618 
142 52.608 
145 53.1586 
148 53.715 
151 54.2604 
154 54.777 
157 55.3153 
160 55.8315 
163 56.3661 
166 56.8704 
169 57.358 
172 57.8598 
175 58.3418 
178 58.8302 
181 59.3081 
184 59.7919 
187 60.2544 
190 60.7184 
193 61.1985 
196 61.6303 
199 62.1147 
>

Consider each block as below, where there should be an Xi such that its corresponding Ti is zero

Code:
>
X1 T1
X2 T2

Xi 0
....

XN TN
>


Need to check whether abs(Xi -Xj) > 40 for j=1,...,N. If this is true, we remove the row containing the X T from the blocks

We continue doing this for all the blocks.

After we need to introduce a third column which is just a copy of each T and some headers. One of the headers is

Code:
% ( PHASES
P
%)

%( SOURCES
(10,0.0)
(13,0.0)
(16,0.0)
(19,0.0)
(22,0.0)
(25,0.0)
(28,0.0)
(31,0.0)
(34,0.0)
(37,0.0)
(40,0.0)
(43,0.0)
(46,0.0)
(49,0.0)
(52,0.0)
(55,0.0)
(58,0.0)
(61,0.0)
(64,0.0)
(67,0.0)
(70,0.0)
(73,0.0)
(76,0.0)
(79,0.0)
(82,0.0)
(85,0.0)
(88,0.0)
(91,0.0)
(94,0.0)
(97,0.0)
(100,0.0)
(103,0.0)
(106,0.0)
(109,0.0)
(112,0.0)
(115,0.0)
(118,0.0)
(121,0.0)
(124,0.0)
(127,0.0)
(130,0.0)
(133,0.0)
(136,0.0)
(139,0.0)
(142,0.0)
(145,0.0)
(148,0.0)
(151,0.0)
(154,0.0)
(157,0.0)
(160,0.0)
(163,0.0)
(166,0.0)
(169,0.0)
(172,0.0)
(175,0.0)
(178,0.0)
(181,0.0)
(184,0.0)
(187,0.0)
(190,0.0)
(193,0.0)
(196,0.0)
(199,0.0)
%)

The values for the sources part is achieved by taking each block and extracting the row for which T is Zero.

Next we introduce

Code:
%< SOURCE 1
%( PHASE 1

at the beginning of each block.

And end each block with

Code:
%)
%>

I use two awk scripts to do this

Code:
function abs(val) {
   return val>0?val:-val
}

BEGIN {
  ARGV[ARGC++] = ARGV[ARGC-1]
  max = 40
}

FNR == NR {
  />/ && idx[FNR] = ++i
  $2 || val[i] = $1
  next
}

FNR in idx {
   v = val[idx[FNR]]
}

{ !/>/ && dist = abs( $1 - v ) }

/>/ || dist < max && NF == 2 { print $0 $2 }

then use the below for the headers

Code:
BEGIN {
    ARGV[ARGC++] = ARGV[ARGC-1]
}

FNR == NR {
    ++j
    $2 || val[j] = $1
    next
}

FNR == 1 {
    print ""
    print "%( PHASES"
    print "P"
    print "%)"
    print ""
    print "%( SOURCES"
    OFS = ""
    for ( k = 1; k < j; ++k ) {
       if ( val[k] != 0 && val[k] !~ />/ ) { print "(",val[k],",0.0)" }
    }
    OFS = " "
    print "%)"
    print ""
}

/>/ { if ( FNR > 1) { print "%)"; print "%>"; print "" }
      P = 1
      next
    }

P {
    print "%< SOURCE", ++i
    print "%( PHASE 1"
    P = 0
}

!/>/ {
    print $0
}

End result would be like this

Code:
% ( PHASES
P
%)

%( SOURCES
(10,0.0)
(13,0.0)
(16,0.0)
(19,0.0)
(22,0.0)
(25,0.0)
(28,0.0)
(31,0.0)
(34,0.0)
(37,0.0)
(40,0.0)
(43,0.0)
(46,0.0)
(49,0.0)
(52,0.0)
(55,0.0)
(58,0.0)
(61,0.0)
(64,0.0)
(67,0.0)
(70,0.0)
(73,0.0)
(76,0.0)
(79,0.0)
(82,0.0)
(85,0.0)
(88,0.0)
(91,0.0)
(94,0.0)
(97,0.0)
(100,0.0)
(103,0.0)
(106,0.0)
(109,0.0)
(112,0.0)
(115,0.0)
(118,0.0)
(121,0.0)
(124,0.0)
(127,0.0)
(130,0.0)
(133,0.0)
(136,0.0)
(139,0.0)
(142,0.0)
(145,0.0)
(148,0.0)
(151,0.0)
(154,0.0)
(157,0.0)
(160,0.0)
(163,0.0)
(166,0.0)
(169,0.0)
(172,0.0)
(175,0.0)
(178,0.0)
(181,0.0)
(184,0.0)
(187,0.0)
(190,0.0)
(193,0.0)
(196,0.0)
(199,0.0)
%)

%< SOURCE 1
%( PHASE 1
10 0 0
13 5.92346 5.92346
16 10.3106 10.3106
19 13.9672 13.9672
22 16.9838 16.9838
25 19.4407 19.4407
28 21.4705 21.4705
31 23.1547 23.1547
34 24.6813 24.6813
37 26.0695 26.0695
40 27.3611 27.3611
43 28.631 28.631
46 29.8366 29.8366
49 30.9858 30.9858
%)
%>

%< SOURCE 2
%( PHASE 1
10 5.92346 5.92346
13 0 0
16 5.92346 5.92346
19 10.3106 10.3106
22 13.9672 13.9672
25 16.9838 16.9838
28 19.4407 19.4407
31 21.4705 21.4705
34 23.1547 23.1547
37 24.6813 24.6813
40 26.0695 26.0695
43 27.3611 27.3611
46 28.631 28.631
49 29.8366 29.8366
52 30.9858 30.9858
%)
%>

%< SOURCE 3
%( PHASE 1
10 10.3106 10.3106
13 5.92346 5.92346
16 0 0
19 5.92346 5.92346
22 10.3106 10.3106
25 13.9672 13.9672
28 16.9838 16.9838
31 19.4407 19.4407
34 21.4705 21.4705
37 23.1547 23.1547
40 24.6814 24.6814
43 26.0695 26.0695
46 27.3611 27.3611
49 28.631 28.631
52 29.8366 29.8366
55 30.9858 30.9858
%)
%>

... etc


I would like to combine the two scripts together and need some help to do this.

Regards
Christopher
Last edited by kristinu; 12-09-2009 at 06:28 AM..
# 2  
Old 12-07-2009
It would be easier if you post the original input and a description/example of the expected output (the one you get after using both scripts).
Last edited by radoulov; 12-07-2009 at 05:41 PM..
# 3  
Old 12-08-2009
Hi there, need some help on this one.
# 4  
Old 12-08-2009
Good topic, your codes are good enough.

Waiting for others to combine them.
# 5  
Old 12-09-2009
I'm a bit confused as I have a next in each one of them. Trying to find the best way to do this in a single awk script. I have been trying to do small thing in separate awk scripts and putting them together instead of having lots of small ones.
# 6  
Old 12-09-2009
You could try something like this:

Code:
function abs(val) {
   return val > 0 ? val : -val
}

BEGIN {
  ARGV[ARGC++] = ARGV[ARGC-1]
  max = 40
}

FNR == NR {
  />/ && idx[FNR] = ++i
  $2 || val[i] = $1 
  lr = FNR; next
}

FNR == 1 {
    print "\n% ( PHASES\nP\n%)\n\n%( SOURCES\n", x
    for ( k = 1; k < i; ++k )
      printf "(%s,0.0)\n", val[k] 
    printf "%%)\n\n", x
    }
    
FNR in idx { v = val[idx[FNR]] }

{ !/>/ && dist = abs( $1 - v ) }

/>/ { if (FNR == lr) 
        printf "%)\n\n", x
      else    
        print (++sc == 1 ? x : "%)" RS RS) "%< SOURCE " sc RS "%( PHASE 1"  
      next 
      }

dist < max && NF == 2 { print $0 $2 }

Last edited by radoulov; 12-09-2009 at 07:21 AM..
# 7  
Old 12-09-2009
Thanks very much.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Combining two scripts into a single script

Hi Folks, I have two scripts that are used to start and stop services these scripts are at the location /opt/app/tre , so that start.sh internally starts the components and stop.sh internally stop all the components, now rite now if I have to stop the services then i need to go first the... (9 Replies)
Discussion started by: punpun66
9 Replies

2. Shell Programming and Scripting

Combining echo and awk

i have a script that has many lines similar to: echo $var | awk -F"--" '{print $2}' as you can see, two commands are being run here. echo and awk. id like to combine this into one awk statement. i tried: awk -F"--" "BEGIN{print $var; print $2}" but i get error messages. (10 Replies)
Discussion started by: SkySmart
10 Replies

3. Shell Programming and Scripting

awk problem - combining awk statements

i have a datafile that has several lines that look like this: 2,dataflow,Sun Mar 17 16:50:01 2013,1363539001,2990,excelsheet,660,mortar,660,4 using the following command: awk -F, '{$3=strftime("%a %b %d %T %Y,%s",$3)}1' OFS=, $DATAFILE | egrep -v "\-OLDISSUES," | ${AWK} "/${MONTH} ${DAY}... (7 Replies)
Discussion started by: SkySmart
7 Replies

4. Shell Programming and Scripting

Combining awk statements

I have a pretty simple script below: #!/bin/sh for i in *.cfg do temp=`awk '/^InputDirectory=/' ${i}` input_dir=`echo ${temp} | awk '{ print substr( $0, 16) }'` echo ${input_dir} done As you can see its opening each cfg file and searching for the line that has "InputDirectory="... (3 Replies)
Discussion started by: ssbsts
3 Replies

5. Shell Programming and Scripting

Combining two shell scripts

Hi readers, I have two shell scripts running on linux machine. Both the scripts do the following task - Connect to DB - Create temp files - run select statement on table - write the rows returned on temp file Sample code is below #!/bin/sh export ORACLE_HOME=/content/oracle/10.2.0... (1 Reply)
Discussion started by: manucorbi
1 Replies

6. UNIX for Dummies Questions & Answers

Combining awk tests

It would be convenient to be able to combine awk tests. For example, suppose that I do this query: awk '$1 != "Bob" || $1 != "Linda" {print $2}' datafileIs there a reasonable way to combine the conditions into a single statement? For example, in egrep, I can do: egrep -v "Bob|Linda"... (4 Replies)
Discussion started by: treesloth
4 Replies

7. Shell Programming and Scripting

combining two scripts

Hi , Following are the two scripts :- Script 1) #!/bin/sh cp file.log file.log.1 Script 2) #!/bin/sh diff file.log file.log.1 > DIFFERENCE.log cp file.log file.log.1 grep "ERROR" DIFFERENCE.log if ; then echo "1" else echo "0" fi (1 Reply)
Discussion started by: himvat
1 Replies

8. Shell Programming and Scripting

combining fields in awk

I am using: ps -A -o command,%cpu to get process and cpu usage figures. I want to use awk to split up the columns it returns. If I use: awk '{print "Process: "$1"\nCPU Usage: "$NF"\n"}' the $NF will get me the value in the last column, but if there is more than one word in the... (2 Replies)
Discussion started by: json4639
2 Replies

9. Shell Programming and Scripting

Combining 2 scripts

Hello I am new to shell scripting. Below are 2 scripts which I need to combine in to single script.Can some experts guide me how to proceed? #!/bin/bash grep "2443" /f/log/s/heduler.log | grep 2443> /tmp/memorydump exec 6<"/tmp/memorydump" read -u 6 data if then echo "IN THEN" <... (4 Replies)
Discussion started by: achararun
4 Replies

10. Shell Programming and Scripting

help combining lines in awk

I seem to have gotten myself in over my head on this one. I need help combining lines together. I have a text file containing 24,000 lines (exactly why I need awk) due to bad formatting it has separated the lines (ideally it should be 12,000 lines total). Example of file: ... (2 Replies)
Discussion started by: blueheed
2 Replies
Login or Register to Ask a Question