Sponsored Content
Full Discussion: Variable input to awk script
Top Forums Shell Programming and Scripting Variable input to awk script Post 303014783 by Subbeh on Wednesday 21st of March 2018 12:09:53 AM
Old 03-21-2018
Variable input to awk script

Hi guys,

I wrote the following function to compare two csv files column by column.
However, sometimes the input needs to be sorted before parsing it to awk.

I can do this by changing the awk arguments, but I would like to make this variable if possible. The below doesn't work since the s_args variable isn't evaluated. Is there any other way to achieve this?

Code:
function csvdiff() {
  if [ "$1" == "-s" ] ; then
    local s_args="<(sort $2) <(sort $3)"
    shift
  fi
  
  awk '
    NR==FNR { file1[FNR]=$0; next }
    FNR==(h?h:1) {
      if (!D) {
        char["\t"]=gsub(/\t/, "\t")
        char[","]=gsub(/,/, ",")
        char["|"]=gsub(/\|/, "|")
        char[";"]=gsub(/;/, ";")
        for (c in char) {
          if (char[c] >= max) {
            max=char[c]
            delim=c
          }
        }
        max=0
      }
      else delim=D

      if (h != 0 || h == "") {
        split($0, a, delim)
    
        for (i=1;i<=length(a);i++) {
          header[i]=a[i]
        }
        next
      }
    }
    FNR>(h?h:0) && file1[FNR] {
      split(file1[FNR], a, delim)
      split($0, b, delim)
      for (i=1;i<=length(a);i++) {
        if (a[i] != b[i])  {
          diff[i]++
          if (col == i) printf "%-1s%s%s\n", a[i], OFS, b[i]
        }
      }
    }
    END {
      if (FNR != length(file1)) {
        print "error: number of lines do not match: " length(file1), FNR
        exit
      }
      if (!col) {
        if (length(diff)) {
          print "column differences:"
          for (i in diff) {
            print i " " header[i] ": " diff[i]
          }
        }
        else {
          print "no differences found"
        }
      }
    }' OFS="\t" ${s_args-$@}
}

Thanks
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Reading input to create a variable in a script?

I would like to prompt for input and then use it as a variable in a script. Something like this. #!/bin/ksh echo "What is your name?: \c" read response echo "Your name is $reply" >file.txt done exit 0 What am I missing? Thanks, (7 Replies)
Discussion started by: darthur
7 Replies

2. Shell Programming and Scripting

Set/Export Env Vars from with Shell Script With Input Variable

I have a shell script I want to run that will set environment variables based on the value of an input variable submitted when the shell script is called. For example: $ mgenv.sh prod This would set environment variables for prod $ mgenv.sh test This would set environment variables... (1 Reply)
Discussion started by: brtaylor73
1 Replies

3. UNIX and Linux Applications

Input a variable and write to a file using awk

Hi I am trying to edit a csv file. Bacically I need to input a search variable and the value that must be changed in one of the fields corresponding to that searched variable. My csv file looks like so: 1,1A,5 1,1B,2 1,1C,3 2,2A,7 2,2B,4 2,2C,0 3,3A,1 3,3B,6 3,3C,4 I want to... (4 Replies)
Discussion started by: ladyAnne
4 Replies

4. Shell Programming and Scripting

Echo date variable from data input to a script

Hi, I'm trying to make a script which you type the year, select the month and day and then create the date in the format 2010-12-7. #!/bin/bash dia () { echo " Seleccione el dia:" select file in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Salir do... (6 Replies)
Discussion started by: iga3725
6 Replies

5. Shell Programming and Scripting

awk built-in variable for input file

Hi guys, Does awk have a built-in variable which I can use to display the input file it's currently reading? I'm currently concatenating multiple files using awk and later on do some parsing. But for now, I want to add an extra column in the main output data file - basically putting in the... (3 Replies)
Discussion started by: Det7
3 Replies

6. UNIX for Dummies Questions & Answers

SQL Script to use variable value from input file

Here is the requirement, When I run the "run file KSH (sql)", it should substitute '${pCW_Bgn_DT}' with 201120 and '${pCW_End_DT}' with 201124 Input File ---------- $ cat prevwk.dat 201124 20110711 run file KSH (sql) ------------------ In this file, I want to use the variables... (1 Reply)
Discussion started by: shanrice
1 Replies

7. Shell Programming and Scripting

Variable as input to awk command

Hi Gurus, I need a suggestion, please help. I have a input file as below : abc.txt : * xxxx: 00000 xxxxx: 00000 xxxx: RANDOM xxx: RANDOM **************************xxxxxxx*** * abc ****************************** abc: abc: ... (3 Replies)
Discussion started by: arunshankar.c
3 Replies

8. Programming

take input from a variable as pattern to awk

Hi everyone, Can anyone tell me how to take contents of a variable as a pattern for awk command. Am doing as below, but doesnt get any output: $c = "Tue Dec"; $log = ` awk '/ \$c /' in.txt`; print $log; (7 Replies)
Discussion started by: anandrec
7 Replies

9. Shell Programming and Scripting

Passing variable as an input file to AWK comand

Hi, I would like to compare 2 files using awk, which I can do by using: awk 'NR==FNR{a;next} (NR > 32 && $2 in a) {print $0}' File1 and File2. If the name of the File1 is in another file (for example, column 4 in File 3) then how can I pass this column 4 to the awk command. Thanks in... (1 Reply)
Discussion started by: ezhil01
1 Replies

10. Shell Programming and Scripting

Call a awk script with variable and input filename

HI, MY question is a very simple one: if i want to call an awk script with the input file name and also pass a variable value , then how to do it. #>awk -f my_script.awk -v variable=value my_inputfile.txt I can't do it like this. throws error: awk: my_script.awk:18:... (0 Replies)
Discussion started by: Onkar Banerjee
0 Replies
COLRM(1)						    BSD General Commands Manual 						  COLRM(1)

NAME
colrm -- remove columns from a file SYNOPSIS
colrm [start [stop]] DESCRIPTION
The colrm utility removes selected columns from the lines of a file. A column is defined as a single character in a line. Input is read from the standard input. Output is written to the standard output. If only the start column is specified, columns numbered less than the start column will be written. If both start and stop columns are spec- ified, columns numbered less than the start column or greater than the stop column will be written. Column numbering starts with one, not zero. Tab characters increment the column count to the next multiple of eight. Backspace characters decrement the column count by one. ENVIRONMENT
The LANG, LC_ALL and LC_CTYPE environment variables affect the execution of colrm as described in environ(7). EXIT STATUS
The colrm utility exits 0 on success, and >0 if an error occurs. SEE ALSO
awk(1), column(1), cut(1), paste(1) HISTORY
The colrm command appeared in 3.0BSD. BSD
August 4, 2004 BSD
All times are GMT -4. The time now is 06:42 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy