UNIX for Beginners Questions & Answers

I am quite new to bash scripting, and are at the moment trying debug an issue i am having.

I am running a script, which is supposed to validate if the dataset is in the correct format, I seem to be caught by one of its error checking, and cannot quite figure out why this is happening..

What is this function doing?

Code:

function check_sorted_and_uniq {
  ! awk '{print $1}' $1 | sort | uniq | cmp -s - <(awk '{print $1}' $1) && \
    echo "$0: file $1 is not in sorted order or has duplicates" && exit 1;
}

The script is executed as

Code:

utils/validate_data_dir.sh --no-text --no-feats $data || exit 1;

the same error occurs if it executed with an path, as in

Code:

 utils/validate_data_dir.sh data/train

the full scrips is available here :kaldi/validate_data_dir.sh at master * kaldi-asr/kaldi * GitHub

This function doing what its name and message states, it checks if the first column of the file passed as an argument contains sorted and unique values otherwise output an error message and exit in failure.

The function code could have been written a more pedagogical way but conciseness was preferred. Here is a more readable variant:

Code:

check_sorted_and_uniq() {
  if ! cmp  <(awk '{print $1}' $1 | sort | uniq) <(awk '{print $1}' $1); then
    echo "$0: file $1 is not in sorted order or has duplicates"
    exit 1;
  fi
}

The "<(...)" syntax creates a transient file that is passed to the program as an argument and which contains the result of the command being inside the parenthesis.

Note that "awk '{print $1}' $1" is simply outputting the first column of the file passed as first argument. "$1" has totally different meanings for awk and for the shell.