Sponsored Content
Full Discussion: Variable input to awk script
Top Forums Shell Programming and Scripting Variable input to awk script Post 303014783 by Subbeh on Wednesday 21st of March 2018 12:09:53 AM
Old 03-21-2018
Variable input to awk script

Hi guys,

I wrote the following function to compare two csv files column by column.
However, sometimes the input needs to be sorted before parsing it to awk.

I can do this by changing the awk arguments, but I would like to make this variable if possible. The below doesn't work since the s_args variable isn't evaluated. Is there any other way to achieve this?

Code:
function csvdiff() {
  if [ "$1" == "-s" ] ; then
    local s_args="<(sort $2) <(sort $3)"
    shift
  fi
  
  awk '
    NR==FNR { file1[FNR]=$0; next }
    FNR==(h?h:1) {
      if (!D) {
        char["\t"]=gsub(/\t/, "\t")
        char[","]=gsub(/,/, ",")
        char["|"]=gsub(/\|/, "|")
        char[";"]=gsub(/;/, ";")
        for (c in char) {
          if (char[c] >= max) {
            max=char[c]
            delim=c
          }
        }
        max=0
      }
      else delim=D

      if (h != 0 || h == "") {
        split($0, a, delim)
    
        for (i=1;i<=length(a);i++) {
          header[i]=a[i]
        }
        next
      }
    }
    FNR>(h?h:0) && file1[FNR] {
      split(file1[FNR], a, delim)
      split($0, b, delim)
      for (i=1;i<=length(a);i++) {
        if (a[i] != b[i])  {
          diff[i]++
          if (col == i) printf "%-1s%s%s\n", a[i], OFS, b[i]
        }
      }
    }
    END {
      if (FNR != length(file1)) {
        print "error: number of lines do not match: " length(file1), FNR
        exit
      }
      if (!col) {
        if (length(diff)) {
          print "column differences:"
          for (i in diff) {
            print i " " header[i] ": " diff[i]
          }
        }
        else {
          print "no differences found"
        }
      }
    }' OFS="\t" ${s_args-$@}
}

Thanks
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Reading input to create a variable in a script?

I would like to prompt for input and then use it as a variable in a script. Something like this. #!/bin/ksh echo "What is your name?: \c" read response echo "Your name is $reply" >file.txt done exit 0 What am I missing? Thanks, (7 Replies)
Discussion started by: darthur
7 Replies

2. Shell Programming and Scripting

Set/Export Env Vars from with Shell Script With Input Variable

I have a shell script I want to run that will set environment variables based on the value of an input variable submitted when the shell script is called. For example: $ mgenv.sh prod This would set environment variables for prod $ mgenv.sh test This would set environment variables... (1 Reply)
Discussion started by: brtaylor73
1 Replies

3. UNIX and Linux Applications

Input a variable and write to a file using awk

Hi I am trying to edit a csv file. Bacically I need to input a search variable and the value that must be changed in one of the fields corresponding to that searched variable. My csv file looks like so: 1,1A,5 1,1B,2 1,1C,3 2,2A,7 2,2B,4 2,2C,0 3,3A,1 3,3B,6 3,3C,4 I want to... (4 Replies)
Discussion started by: ladyAnne
4 Replies

4. Shell Programming and Scripting

Echo date variable from data input to a script

Hi, I'm trying to make a script which you type the year, select the month and day and then create the date in the format 2010-12-7. #!/bin/bash dia () { echo " Seleccione el dia:" select file in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Salir do... (6 Replies)
Discussion started by: iga3725
6 Replies

5. Shell Programming and Scripting

awk built-in variable for input file

Hi guys, Does awk have a built-in variable which I can use to display the input file it's currently reading? I'm currently concatenating multiple files using awk and later on do some parsing. But for now, I want to add an extra column in the main output data file - basically putting in the... (3 Replies)
Discussion started by: Det7
3 Replies

6. UNIX for Dummies Questions & Answers

SQL Script to use variable value from input file

Here is the requirement, When I run the "run file KSH (sql)", it should substitute '${pCW_Bgn_DT}' with 201120 and '${pCW_End_DT}' with 201124 Input File ---------- $ cat prevwk.dat 201124 20110711 run file KSH (sql) ------------------ In this file, I want to use the variables... (1 Reply)
Discussion started by: shanrice
1 Replies

7. Shell Programming and Scripting

Variable as input to awk command

Hi Gurus, I need a suggestion, please help. I have a input file as below : abc.txt : * xxxx: 00000 xxxxx: 00000 xxxx: RANDOM xxx: RANDOM **************************xxxxxxx*** * abc ****************************** abc: abc: ... (3 Replies)
Discussion started by: arunshankar.c
3 Replies

8. Programming

take input from a variable as pattern to awk

Hi everyone, Can anyone tell me how to take contents of a variable as a pattern for awk command. Am doing as below, but doesnt get any output: $c = "Tue Dec"; $log = ` awk '/ \$c /' in.txt`; print $log; (7 Replies)
Discussion started by: anandrec
7 Replies

9. Shell Programming and Scripting

Passing variable as an input file to AWK comand

Hi, I would like to compare 2 files using awk, which I can do by using: awk 'NR==FNR{a;next} (NR > 32 && $2 in a) {print $0}' File1 and File2. If the name of the File1 is in another file (for example, column 4 in File 3) then how can I pass this column 4 to the awk command. Thanks in... (1 Reply)
Discussion started by: ezhil01
1 Replies

10. Shell Programming and Scripting

Call a awk script with variable and input filename

HI, MY question is a very simple one: if i want to call an awk script with the input file name and also pass a variable value , then how to do it. #>awk -f my_script.awk -v variable=value my_inputfile.txt I can't do it like this. throws error: awk: my_script.awk:18:... (0 Replies)
Discussion started by: Onkar Banerjee
0 Replies
COMM(1) 						    BSD General Commands Manual 						   COMM(1)

NAME
comm -- select or reject lines common to two files SYNOPSIS
comm [-123f] file1 file2 DESCRIPTION
The comm utility reads file1 and file2, which should be sorted lexically, and produces three text columns as output: lines only in file1; lines only in file2; and lines in both files. The filename ``-'' means the standard input. The following options are available: -1 Suppress printing of column 1. -2 Suppress printing of column 2. -3 Suppress printing of column 3. -f Fold case in line comparisons. Each column will have a number of tab characters prepended to it equal to the number of lower numbered columns that are being printed. For example, if column number two is being suppressed, lines printed in column number one will not have any tabs preceding them, and lines printed in column number three will have one. comm assumes that the files are lexically sorted; all characters participate in line comparisons. EXIT STATUS
comm exits 0 on success, >0 if an error occurred. SEE ALSO
cmp(1), diff(1), sort(1), uniq(1) STANDARDS
The comm utility conforms to IEEE Std 1003.2-1992 (``POSIX.2''). BSD
June 6, 1993 BSD
All times are GMT -4. The time now is 10:07 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy