Shell Programming and Scripting

Hi,

As part of our project, we need to load historical data for a year before our system is live. We have the data feed files that we need to load. However, I need to make sure that the file structure (number of fields separated by a comma) on the field is same for all the files of the same type. I wrote a shell script that selects specific rows of a all feed files for a particular feed type and counts the number of comma's in the row. At the end - if I have the same number of commas on the selected rows for all files, I am sure that the feed file structure is same.

However, I am facing some issues with my script. In some files, the combination of the head and tail commands are not returning a single row as I expect. It seems that the LF character is missing on these rows and hence “multiple rows” are getting treated as a single row. But, when I open the file in vi editor (or Textpad on Windows or in excel) the row appears to be fine. The rows seem to be multiple rows in vi and hence I am confused why the combination of the head and tail command that I have used is returning multiple rows as a single row?

I am using the following command to count the number of commas in a row (3rd row) in the file:
r3=`head -3 $file | tail -1 | tr -dc ',' | wc -c`

What complicates this scenario is that this problem is not present for all rows on all feed files. Any help will be greatly appreciated.

Thanks,
Raj

A bit more code but it works fine assuming you have valid lines of data to feed it

Code:

 
cat x.data
1,2,3
1,2,5
 
cat x1.data
1,2,2
1,2
 
 
cat xxx.ksh
=========
#!/bin/ksh
TEMPFILE=/tmp/junk.out
for file in x.data x1.data
do
awk -F"," '{print NF-1}' $file | sort -u > $TEMPFILE
num_of_matches=`wc -l <$TEMPFILE`
if [ "$num_of_matches" -gt 1 ]
then
        echo "$file columns don't match"
else
        echo "$file columns  match"
fi
done