Shell Programming and Scripting

Hello,

I have this awk script that I want to execute by passing parameters through a shell script.

I'm a little confused. This awk script removes duplicates from an input file.

Ok, so I have a .sh file called rem_dups.sh

#!/usr/bin/sh
#---------------------------------------------------------------------
# Program ....... rem_dups.sh
# Function ...... removes duplicates from input file
# Developer ..... script_op2a
# Date .......... November 2 2010
# Parameters .... $1 = Position of Key column in input file (Required)
$2 = Unix Script directory (Required)
# $3 = Input file name of file to remove duplicates from (Required)

awk '{FS="";split($NF,a,"_"); key=$pos;site=a[3];keysite=key "_" site;
if (b[keysite]<=a[4]a[5]) {b[keysite]=a[4]a[5];c[keysite]=$0;}}
END{for (i in b) print c[i]}' $dir $filename

It needs to have 3 variable parameters.

$pos (this value needs to be the awk field like $1 $2 $3 $4 or whatever)
$dir (this needs to specify the directory of the input file
$filename (this is the name of the input file)

I want to execute it from command line like:

rem_dups.sh 1 /home/scriptop/input_file_dups.txt > output_file_no_dups.txt

Use the -v option ... short example :

Code:

awk -vkey="$pos" '{print key}' file

ok so I modified my shell script like this:
-------------------------------------------
#!/usr/bin/sh
#---------------------------------------------------------------------
# Program ....... dups_filter.sh

pos="$1"
filename="$2"

awk -v key="$pos" -v infile="$filename" '{FS="";split($NF,a,"_"); site=a[3];keysite=key "_" site;
if (b[keysite]<=a[4]a[5]) {b[keysite]=a[4]a[5];c[keysite]=$0;}}
END{for (i in b) print c[i]}' infile
#end of .sh program
---------------------------------------------------------------

I have the .sh script in the same directory as the infile dups.txt.

I execute from the command line like:
./rem_dups.sh 1 dups.txt > no_dups.txt

and I get the error
awk: Cannot find or open file infile.
The source line number is 3.

Could somebody help me with this confusion?

Please post a representative sample of the input you have and output you expect.
Note that if you want to remove duplicate lines , you might want to use the uniq command

Hello thank you,

I made a clearer post here:

https://www.unix.com/shell-programmin...#post302468464