Variable input to awk script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Variable input to awk script
# 1  
Old 03-21-2018
Variable input to awk script

Hi guys,

I wrote the following function to compare two csv files column by column.
However, sometimes the input needs to be sorted before parsing it to awk.

I can do this by changing the awk arguments, but I would like to make this variable if possible. The below doesn't work since the s_args variable isn't evaluated. Is there any other way to achieve this?

Code:
function csvdiff() {
  if [ "$1" == "-s" ] ; then
    local s_args="<(sort $2) <(sort $3)"
    shift
  fi
  
  awk '
    NR==FNR { file1[FNR]=$0; next }
    FNR==(h?h:1) {
      if (!D) {
        char["\t"]=gsub(/\t/, "\t")
        char[","]=gsub(/,/, ",")
        char["|"]=gsub(/\|/, "|")
        char[";"]=gsub(/;/, ";")
        for (c in char) {
          if (char[c] >= max) {
            max=char[c]
            delim=c
          }
        }
        max=0
      }
      else delim=D

      if (h != 0 || h == "") {
        split($0, a, delim)
    
        for (i=1;i<=length(a);i++) {
          header[i]=a[i]
        }
        next
      }
    }
    FNR>(h?h:0) && file1[FNR] {
      split(file1[FNR], a, delim)
      split($0, b, delim)
      for (i=1;i<=length(a);i++) {
        if (a[i] != b[i])  {
          diff[i]++
          if (col == i) printf "%-1s%s%s\n", a[i], OFS, b[i]
        }
      }
    }
    END {
      if (FNR != length(file1)) {
        print "error: number of lines do not match: " length(file1), FNR
        exit
      }
      if (!col) {
        if (length(diff)) {
          print "column differences:"
          for (i in diff) {
            print i " " header[i] ": " diff[i]
          }
        }
        else {
          print "no differences found"
        }
      }
    }' OFS="\t" ${s_args-$@}
}

Thanks
# 2  
Old 03-21-2018
You could try with eval. This method should be deployed with care; you should know EXACTLY what you are evaling as you may end up running unwanted or even malicious code without control of it.
This User Gave Thanks to RudiC For This Post:
# 3  
Old 03-25-2018
Are $2 and $3 supposed to be filenames or space separated parameter lists?

Here is one bash solution using the -v option of printf:

Code:
#!/bin/bash
function csvdiff {
   if [ "$1" == "-s" ] ; then
      local s_args
      shift

      printf -v s_args "%s%s" \
          "$(printf "%s\n" $1 | sort | tr '\n' ' ' )" \
          "$(printf "%s\n" $2 | sort | tr '\n' ' ' )"
   fi
   echo "${s_args:-$@}"
}

echo unsorted
csvdiff "7 2 1 8 9" "5 1 3 7 4"
echo sorted
csvdiff -s "7 2 1 8 9" "5 1 3 7 4"

output:

Code:
unsorted
7 2 1 8 9 5 1 3 7 4
sorted
1 2 7 8 9 1 3 4 5 7

This User Gave Thanks to Chubler_XL For This Post:
# 4  
Old 03-27-2018
Thanks guys,

Not sure if it's the best solution, but in the end I chose to create another nested function to run the awk command:

Code:
function csvdiff() {
  function _csvdiff() {
    awk '
      ...
      }' OFS="\t" $@
  }
  
  if [ "$1" == "-s" ] ; then
    _csvdiff <(sort "$2") <(sort "$3")
  else
    _csvdiff $@
  fi
  
  unset -f _csvdiff
}

This seems to work fine. Thanks for the input though! Both solutions would have worked, although I'm reading in files and not space separated lists.

Last edited by Subbeh; 03-28-2018 at 12:07 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Call a awk script with variable and input filename

HI, MY question is a very simple one: if i want to call an awk script with the input file name and also pass a variable value , then how to do it. #>awk -f my_script.awk -v variable=value my_inputfile.txt I can't do it like this. throws error: awk: my_script.awk:18:... (0 Replies)
Discussion started by: Onkar Banerjee
0 Replies

2. Shell Programming and Scripting

Passing variable as an input file to AWK comand

Hi, I would like to compare 2 files using awk, which I can do by using: awk 'NR==FNR{a;next} (NR > 32 && $2 in a) {print $0}' File1 and File2. If the name of the File1 is in another file (for example, column 4 in File 3) then how can I pass this column 4 to the awk command. Thanks in... (1 Reply)
Discussion started by: ezhil01
1 Replies

3. Programming

take input from a variable as pattern to awk

Hi everyone, Can anyone tell me how to take contents of a variable as a pattern for awk command. Am doing as below, but doesnt get any output: $c = "Tue Dec"; $log = ` awk '/ \$c /' in.txt`; print $log; (7 Replies)
Discussion started by: anandrec
7 Replies

4. Shell Programming and Scripting

Variable as input to awk command

Hi Gurus, I need a suggestion, please help. I have a input file as below : abc.txt : * xxxx: 00000 xxxxx: 00000 xxxx: RANDOM xxx: RANDOM **************************xxxxxxx*** * abc ****************************** abc: abc: ... (3 Replies)
Discussion started by: arunshankar.c
3 Replies

5. UNIX for Dummies Questions & Answers

SQL Script to use variable value from input file

Here is the requirement, When I run the "run file KSH (sql)", it should substitute '${pCW_Bgn_DT}' with 201120 and '${pCW_End_DT}' with 201124 Input File ---------- $ cat prevwk.dat 201124 20110711 run file KSH (sql) ------------------ In this file, I want to use the variables... (1 Reply)
Discussion started by: shanrice
1 Replies

6. Shell Programming and Scripting

awk built-in variable for input file

Hi guys, Does awk have a built-in variable which I can use to display the input file it's currently reading? I'm currently concatenating multiple files using awk and later on do some parsing. But for now, I want to add an extra column in the main output data file - basically putting in the... (3 Replies)
Discussion started by: Det7
3 Replies

7. Shell Programming and Scripting

Echo date variable from data input to a script

Hi, I'm trying to make a script which you type the year, select the month and day and then create the date in the format 2010-12-7. #!/bin/bash dia () { echo " Seleccione el dia:" select file in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Salir do... (6 Replies)
Discussion started by: iga3725
6 Replies

8. UNIX and Linux Applications

Input a variable and write to a file using awk

Hi I am trying to edit a csv file. Bacically I need to input a search variable and the value that must be changed in one of the fields corresponding to that searched variable. My csv file looks like so: 1,1A,5 1,1B,2 1,1C,3 2,2A,7 2,2B,4 2,2C,0 3,3A,1 3,3B,6 3,3C,4 I want to... (4 Replies)
Discussion started by: ladyAnne
4 Replies

9. Shell Programming and Scripting

Set/Export Env Vars from with Shell Script With Input Variable

I have a shell script I want to run that will set environment variables based on the value of an input variable submitted when the shell script is called. For example: $ mgenv.sh prod This would set environment variables for prod $ mgenv.sh test This would set environment variables... (1 Reply)
Discussion started by: brtaylor73
1 Replies

10. UNIX for Dummies Questions & Answers

Reading input to create a variable in a script?

I would like to prompt for input and then use it as a variable in a script. Something like this. #!/bin/ksh echo "What is your name?: \c" read response echo "Your name is $reply" >file.txt done exit 0 What am I missing? Thanks, (7 Replies)
Discussion started by: darthur
7 Replies
Login or Register to Ask a Question