Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Merging together two awk scripts

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Merging together two awk scripts
# 1  
Old 11-09-2011
Merging together two awk scripts
I have two awk scripts shown below. checkTrvt.awk works on file format .xt, whereas checkData.awk workds on file format .dat

I want to merge the two scripts together, if I find that the user passed .xt file I do the code for .xt file, whereas if user passes .dat file, I go through the code for .dat files.

Code:
################################################################################
#
#  --Module---------------------------------------------------------------------
#
#                             checkTrvt.awk
#
#    Prints warnings when the source to receiver distances exceed dsrmx.
#
#  --Usage----------------------------------------------------------------------
#
#    awk [optionalArgs] -f ./checkTrvt.awk file > file_error_log
#
#  --Options--------------------------------------------------------------------
#
#    -v dsrmx=value
#      Maximum source to receiver distance to check.
#
#    file
#      Input travel time data (.xt, .dat) file format.
#
#    file_error_log
#      Output file containing warnings.
#
#    -h, --help, -u, --usage, -e, --examples
#      Prints a usage message to STDOUT and exits.
#
#  --Examples-------------------------------------------------------------------
#
#    awk -v dsrmx=65 -f ./checkTrvt.awk file.xt > file_error_log
#    awk -f ./checkTrvt.awk -h
#
################################################################################

#  abs: Returns the absolute value of a number

function abs(val) {

return val > 0 ? val  \
               : -val

}

################################################################################

BEGIN {

  ARGV[ARGC++] = ARGV[ARGC-1]                                  # Read file twice

}

# --Read file: First time round-------------------------------------------------

FNR == NR {
  />/ && idx[FNR] = ++i
  $2 || val[i] = $1                                 # Set source location ($2=0)
  ++j                                                   # Store last line number
  next
}

# --Read file: Second time round------------------------------------------------

FNR in idx { v = val[idx[FNR]] }                           # Get source location

{ srdist = abs($1 - v) }                       # Set source to receiver distance

/^[0-9]+/ && srdist > dsrmx {                                      # Print error
  print "  FNR="FNR", dist("v","$1")="srdist
}

FNR == j {
  print ""
  print "--END checkTrvt.awk---------------------------------------------------"
  print ""
}

################################################################################

Code:
################################################################################
#
#  --Module---------------------------------------------------------------------
#
#                                  checkDat.awk
#
#    Prints warnings if source-receiver distances exceed dsrmx.
#
#  --Usage----------------------------------------------------------------------
#
#    awk [options] -f ./checkDat.awk fin.dat > fout.chk
#
#  --Optional Arguments---------------------------------------------------------
#
#    -v dsrmx=value
#      Source-receiver distance to check
#
#    fin.dat
#      Travel time data file
#
#    fout.chk
#      File containing warnings
#
#    -h, --help, -u, --usage, -e, --examples
#      Prints a usage message to STDOUT and exits.
#
################################################################################

#  abs: Calculates the absolute value of val

function abs(val) {

  return val > 0 ? val  \
                 : -val

}

################################################################################

# Change record Separator and Field Separator.

BEGIN {

  RS = "%<"                                            # Change record separator
  FS = "\n"                                             # Change field separator

}

# 4. Skip the first two fields (each field is a line between RS="%<")

/SOURCE [0-9]/ {
  for (i = 3; i <= NF; i++) {
    if (length($i) > 2) {                            # Skip %) and %> at the end
      split($i, a, " ")                         # Get the three separate numbers
      if ((a[2] == 0) && (a[3] == 0)) {              # Last two numbers are zero
#        print ""
        for (j = 3; j <= NF; j++) {
          if (length($j) > 2) {                      # Skip %) and %> at the end
            split($j, b, " ")
            srdist = abs(a[1] - b[1])
            if (srdist > dsrmx){
              print "WARNING",$1": dist("a[1]","b[1]")="srdist
            }
          }
        }
      }
    }
  }
}

# End of file

FNR == i {
  print ""
  print "--End of checkDat.awk-------------------------------------------------"
  print ""
}

################################################################################

Last edited by kristinu; 11-09-2011 at 12:25 PM..
# 2  
Old 11-09-2011
It'd help if you showed the input and output data you want.
# 3  
Old 11-09-2011
Format of .xt file. Consists of a list of two numbers. They come in blocks separated by >. For each block I find the number for which the second entry is 0, I then subtract all the numbers in the block from this number. If the results exceeds dsrmx, I want to log the block in a file.

Small example of .xt file

Code:
>
10 0
20 5.62454
30 10.1842
40 13.783
50 16.9736
>
10 5.54733
20 0
30 5.79345
40 10.117
50 13.8732
>
10 10.1026
20 5.61005
30 0
40 5.7841
50 10.0874
>

The command
Code:
awk -v dsrmx=25 -f ./checkTrvt.awk test.xt

gives

Code:
--Error Log-----------------------------------------------------------

  Source to receiver distance exceeding 25 Mm.

  FNR=5, dist(10,40)=30
  FNR=6, dist(10,50)=40
  FNR=12, dist(20,50)=30

--END checkTrvt.awk---------------------------------------------------

--------- Post updated at 11:17 AM ---------- Previous update was at 11:13 AM ----------

This is an example of a .dat file. In this case the blocks are separated by

Code:
%< SOURCE 1
%( PHASE 1


%)
%>

There are now three columns. I want to do the same, get the entry have the last two values 0, and subtract every first number in the block from it. If value exceeds dsrmx, I print a warning.

Code:
%( PHASES
P
%)

%( SOURCES
(10,0.0)
(20,0.0)
(30,0.0)
(40,0.0)
(50,0.0)
%)

%< SOURCE 1
%( PHASE 1
10 0 0 
20 5.62454 5.62454 
30 10.1842 10.1842 
40 13.783 13.783 
50 16.9736 16.9736 
%)
%>

%< SOURCE 2
%( PHASE 1
10 5.54733 5.54733 
20 0 0 
30 5.79345 5.79345 
40 10.117 10.117 
50 13.8732 13.8732 
%)
%>

%< SOURCE 3
%( PHASE 1
10 10.1026 10.1026 
20 5.61005 5.61005 
30 0 0 
40 5.7841 5.7841 
50 10.0874 10.0874 
%)
%>

Last edited by kristinu; 11-09-2011 at 12:36 PM..
# 4  
Old 11-09-2011
Because your 'checkTrvt.awk' script parses the .txt file twice AND because files have different structure/separators (and other peculiarities you'll have to account for)... by merging 2 scripts you may end up with the not so elegant code.

Why exactly do you want to merge the two?
Why having two separate script present an issue?
# 5  
Old 11-09-2011
Why not just have a shell script wrapper that calls the appropriate awk script based on the suffix?
# 6  
Old 11-09-2011
I was thinking of merging them as they consist of the same data bur stored in different file formats. But you do have a good point and creating a wrapper is a good idea.

---------- Post updated at 01:55 PM ---------- Previous update was at 11:59 AM ----------

I have the awk script below and I want to include a string at the end.

How can I do it?

Code:
################################################################################
#
#  --Module---------------------------------------------------------------------
#
#                               checkTrvtdat.awk
#
#    Prints warnings if source-receiver distances exceed dsrmx.
#
#  --Usage----------------------------------------------------------------------
#
#    awk [options] -f ./checkTrvtdat.awk file.dat > file_error_log
#
#  --Optional Arguments---------------------------------------------------------
#
#    -v dsrmx=value
#      Source-receiver distance to check
#
#    file.dat
#      Travel time data file
#
#    file_error_log
#      File containing warnings
#
#    -h, --help, -u, --usage, -e, --examples
#      Prints a usage message to STDOUT and exits.
#
#  --Operations-----------------------------------------------------------------
#
#    abs(val)
#    usage()
#    help(Version)
#
#  --Note-----------------------------------------------------------------------
#
#    Record separator RS set to "%<"
#    Field separator FS set to new line "\n". Each field consists of a
#    row containing three numbers separated by spaces
#
#    Take all the fields (lines) in the record and split the three numbers.
#    Take the difference between the each first number and the one with
#    zeros.
#
#  --History--------------------------------------------------------------------
#
#    V01 - DEC 2009 - Christopher Dimech
#          Initial release.
#    V02 - JAN 2010 - Christopher Dimech
#          Arguments can be passed to this awk script.
#    V03 - MAY 2010 - Christopher Dimech
#          New argument passing scheme.
#    V04 - JUN 2010 - Christopher Dimech
#          Using functions usage() and help().
#
################################################################################

#  usage: Description of this awk script

function usage() {

  print "--Usage---------------------------------------------------------------"
  print ""
  print "  awk [optionalArgs] -f ./checkDat.awk file.dat > file_error_log"
  print ""
  print "--Optional Arguments--------------------------------------------------"
  print ""
  print "  -v dsrmx=value"
  print "    Maximum source-receiver distance to check."
  print ""
  print "  file.dat"
  print "    Travel time data file."
  print ""
  print "  file_error_log"
  print "    File showing warnings."
  print ""
  print "  -h, --help, -u, --usage, -e, --examples"
  print "    Prints a usage message."
  print ""
  print "----------------------------------------------------------------------"
  print ""

}

################################################################################

#  examples:

function examples() {

  print ""
  print "--Examples------------------------------------------------------------"
  print ""
  print "  awk -v dsrmx=65 -f ./checkTrvtdat.awk file.dat > file_error_log"
  print "  awk -f ./checkTrvtdat.awk -h"
  print ""
  print "----------------------------------------------------------------------"
  print ""
}

################################################################################

#  help: Description of this awk script

function help() {

  arg1 = "awk -v dsrmax="
  arg2 = "-f ./checkTrvtdat.awk file.dat > file_error_log"
  usg1 = arg1" "arg2
  eg1 = "awk -v dsrmx=65 "arg2
  print ""
  print "--Module--------------------------------------------------------------"
  print ""
  print "                           checkDat.awk"
  print ""
  print "  Prints warnings when the source-receiver distances exceeds dsrmx"
  print ""
  print "--Usage---------------------------------------------------------------"
  print ""
  print "  awk [optionalArgs] -f ./checkTrvtdat.awk file.dat > file_error_log"
  print ""
  print "--Optional Arguments--------------------------------------------------"
  print ""
  print "  -v dsrmx=value"
  print "    Maximum source-receiver distance to check."
  print ""
  print "  fin.dat"
  print "    Input travel time data file."
  print ""
  print "  fout.chk"
  print "    Output file showing warnings."
  print ""
  print "  -h, --help, -u, --usage, -e, --examples"
  print "    Prints a usage message."
  print ""
  print "--Examples------------------------------------------------------------"
  print ""
  print "  awk -v dsrmx=65 -f ./checkDat.awk file.dat > file_error_log"
  print "  awk -f ./checkTrvtdat.awk -h"
  print ""
  print "----------------------------------------------------------------------"
  print ""

}

################################################################################

#  abs: Calculates the absolute value of val

function abs(val) {

  return val > 0 ? val  \
                 : -val

}

################################################################################

# Change record Separator and Field Separator.

BEGIN {

  c = ARGV[ARGC-1]

  if ((c == "-u") || (c == "--usage")) {
    usage()
    exit 1
  }

  if ((c == "-e") || (c == "--examples")) {
    examples()
    exit 1
  }

  if ((c == "-h") || (c == "--help")) {
    help()
    exit 1
  }

  if ( !dsrmx ) {
    print ""
    print "--Error-------------------------------------------------------------"
    print ""
    print "  dsrmx not set"
    print ""
    print "--------------------------------------------------------------------"
    print ""
    exit 1
  }

  RS = "%<"                                            # Change record separator
  FS = "\n"                                             # Change field separator

}

# Error header

NR == 1 {
  print ""
  print "--Error Log-----------------------------------------------------------"
  print ""
  print "  Source to receiver distances exceeding",dsrmx,"Mm."
  print ""
}

# 4. Skip the first two fields (each field is a line between RS="%<")

/SOURCE [0-9]/ {
  for (i = 3; i <= NF; i++) {
    if (length($i) > 2) {                            # Skip %) and %> at the end
      split($i, a, " ")                         # Get the three separate numbers
      if ((a[2] == 0) && (a[3] == 0)) {              # Last two numbers are zero
#        print ""
        for (j = 3; j <= NF; j++) {
          if (length($j) > 2) {                      # Skip %) and %> at the end
            split($j, b, " ")
            srdist = abs(a[1] - b[1])
            if (srdist > dsrmx){
              print " "$1": dist("a[1]","b[1]")="srdist
            }
          }
        }
      }
    }
  }
}

# End of file

################################################################################

When I run it on a file, I get

Code:
--Error Log-----------------------------------------------------------

  Source to receiver distances exceeding 25 Mm.

  SOURCE 1: dist(10,40)=30
  SOURCE 1: dist(10,50)=40
  SOURCE 2: dist(20,50)=30

But I want something like below

Code:
--Error Log-----------------------------------------------------------

  Source to receiver distances exceeding 25 Mm.

  SOURCE 1: dist(10,40)=30
  SOURCE 1: dist(10,50)=40
  SOURCE 2: dist(20,50)=30

--End of checkTrvtdat.awk---------------------------------------------

Input file will be:

Code:
%( PHASES
P
%)

%( SOURCES
(10,0.0)
(20,0.0)
(30,0.0)
(40,0.0)
(50,0.0)
%)

%< SOURCE 1
%( PHASE 1
10 0 0 
20 5.62454 5.62454 
30 10.1842 10.1842 
40 13.783 13.783 
50 16.9736 16.9736 
%)
%>

%< SOURCE 2
%( PHASE 1
10 5.54733 5.54733 
20 0 0 
30 5.79345 5.79345 
40 10.117 10.117 
50 13.8732 13.8732 
%)
%>

%< SOURCE 3
%( PHASE 1
10 10.1026 10.1026 
20 5.61005 5.61005 
30 0 0 
40 5.7841 5.7841 
50 10.0874 10.0874 
%)
%>

# 7  
Old 11-09-2011
Code:
END {
  print "--End of checkTrvtdat.awk---------------------------------------------"
}
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Merging two lines into one (awk)

Hi, I am attempting to merge the following lines which run over two lines using awk. INITIAL OUTPUT 2019 Sep 28 10:47:24.695 hkaet9612 last message repeated 1 time 2019 Sep 28 10:47:24.695 hkaet9612 %ETHPORT-5-IF_DOWN_INTERFACE_REMOVED: Interfa ce Ethernet1/45 is down (Interface removed)... (10 Replies)
Discussion started by: sand1234
10 Replies

2. Shell Programming and Scripting

Merging 2 slightly differnet shell scripts together to acheive same outcome

hi all. i have a kimsufi server, and i am wanting to it to syncronise back to my home linux system, but i want it to track what files have been transfered, so that if i move or delete them on my system, they wont re-sync back. i have googled around and found 2 scripts that acheive this, but... (1 Reply)
Discussion started by: jbates58
1 Replies

3. Shell Programming and Scripting

Merging rows in awk

Hello, I have a data format as follows: Ind1 0 1 2 Ind1 0 2 1 Ind2 1 1 0 Ind2 2 2 0 I want to use AWK to have this output: Ind1 00 12 21 Ind2 12 12 00 That is to merge each two rows with the same row names. Thank you very much in advance for your help. (8 Replies)
Discussion started by: Homa
8 Replies

4. Shell Programming and Scripting

Merging two files in awk

Hi, How I can merge two file columns such as the followings using awk: file 1 2 3 2 2 1 1 file 2 4 3 4 5 7 6 Result: 2 3 4 3 2 2 4 5 1 1 7 6 This is an example, at the end, I will have about 25 files that I want to merge them, it is important for me that the order in the... (7 Replies)
Discussion started by: Homa
7 Replies

5. Shell Programming and Scripting

merging files using awk

Hi, I have 2 files. File 1 chr1 1234 2468 ABC chr1 3456 4567 DEF chr2 5643 6154 XYZ : : : : so on.... File 2 chr1 1500 2500 positive chr1 2500 3500 negative chr1 3000 4500 neutral (10 Replies)
Discussion started by: Diya123
10 Replies

6. Shell Programming and Scripting

awk command - column merging

I have two files having 3 coulms and 1 column respectively file1.txt 0 22.89 35.60 10 22.80 35.61 20 22.70 35.63 30 22.32 35.68 50 19.23 35.79 75 16.10 35.59 100 15.00 35.52 125 14.45 35.46 150 13.91 35.41 200 12.94 35.28 ... (7 Replies)
Discussion started by: shashi792
7 Replies

7. Shell Programming and Scripting

merging of 2 files AWK, SHELL or something else

I have 2 files pipe delimted and want to merge them based on a key e.g file 1 123|xxx|yyy|zzz 345|xab|yzy|zyz 456|sss|ttt|foo file 2 123|hhh|ggg|xxx 345|ddd|www|ddd|fff 456|ddd|sss|sss|eee so if the key is the first field, and the result should be file 1 with field 2 from file 2... (24 Replies)
Discussion started by: klut
24 Replies

8. UNIX for Dummies Questions & Answers

merging 2 lines with awk and stripping first two words

Hey all i am pretty new to awk... here my problem. My input is something like this: type: NSR client; name: pegasus; save set: /, /var, /part, /part/part2, /testpartition, /foo/bar,... (9 Replies)
Discussion started by: bazzed
9 Replies

9. Shell Programming and Scripting

Merging lines using AWK

Hi, Anybody help on this. :( I want to merge the line with previous line, if the line starts with 7. Otherwise No change in the line. Example file aa.txt is like below 122122 222222 333333 734834 702923 389898 790909 712345 999999 My output should be written in another file... (6 Replies)
Discussion started by: senthil_is
6 Replies

10. Shell Programming and Scripting

merging of 2 files AWK - part 2

i have try , but i think i will never learn awk :( now i have 2 files : a 1:aaa:2:aaa1 2:bbb:2:bbb1 3:ccc:3:ccc1 b aaa:2 bbb:0 ccc:3 output: for all lines where a.$2 == b.$1 i want to compare a.$3 != b.$2 if true then set err=1 if false set err=0 and print all lines from file a +... (2 Replies)
Discussion started by: pp56825
2 Replies
Login or Register to Ask a Question