Shell Programming and Scripting

You must be kidding! Where in your feed.txt file are the header columns updated_at and created_at? And, you specified one column to find only! What a waste of time!

---------- Post updated at 10:44 ---------- Previous update was at 10:32 ----------

Although seriously annoyed, here's a proposal that works for the repeatedly modified specs and the corrected samples:

Code:

awk -F\; 'NR==FNR               {n=split ($0, HD)
                                 DEC=$0
                                 next  
                                }
          FNR==1 && $0==DEC     {exit}
          FNR==1                {for (i=1; i<=NF; i++) 
                                   if ($i != HD[i-j]) {DEL[++j]=i; print "Column " i ": " $i " > mismatch.txt"}
                                }
                                {for (i=1; i<=j; i++) $(DEL[i])=""
                                 sub (/;;/,";"); sub (/^;|;$/, "")
                                }                       
          1
        ' OFS=";" dec.txt feed.txt

Hi,

Must admit, when I saw the initial sample data I thought OK - was a bit shocked to see the actual data sample - no resemblence at all to the original request.

Dave

Shouldn't:

Code:

                                   if ($i != HD[i-j]) {DEL[++j]=i; print "Column " i ": " $i " > mismatch.txt"}

be:

Code:

                                   if ($i != HD[i-j]) {DEL[++j]=i; print "Column " i ": " $i > "mismatch.txt"}

Hi Rudic,

Thanks for your reply. I never had any intention to create confusion. I thought I should keep the sample file simple and explain my requirement so that I can later modify the code for a file with more columns.

I really apologize for failing to explain in a clear way.

Regards
Praveen

@Don Cragun: Yes - 'twas my testing script, and I slightly lost my sang froid...

@MadeInGermany

Hi,

I too have a similar requirement. I was reffering to your code

Code:

awk '
BEGIN { FS=OFS=";" }
NR==FNR {
# 1st file: build hd[]
for (i=1; i<=NF; i++) { hd[$i] }
next
}
FNR==1 {
# header of 2nd file: if in hd[] note that column in col[]
for (i=1; i<=NF; i++) {
if ($i in hd) { col[i] } else { print "Column " i ": " $i > "mismatch.txt" }
}
}
{
# print the columns that are in col[]
sep=""
for (i=1; i<=NF; i++) {
if (i in col) { printf "%s%s", sep, $i; sep=OFS }
}
print ""
}
' declare.txt feed.txt

Where in I need to remove the new columns from a daily input data file if the same is not added in the database and report the new columns to the user.

I tried the above code to generate an output file (by using the similar sample files created by me). by slightly modifying the last line of the above code to generate the output file like

' declare.txt feed.txt > feed2.txt

Every thing works fine, it is creating a new file. If the columns are added, it is creating a "Mismatch.txt" file and if no new columns are added then it is skipping the creation of "Mismatch.txt". And in both the scenarios it is creating feed2.txt.

The code is even removing the new columns along with it's values.

But what I noticed is file size of feed2.txt is getting changed even when there is no new column added in the feed.txt file (sample file).

I have European characters like Á, Ä, Ç, Ë, Ê, É, ß, Ô, Ü in my input file "feed.txt". And I think those characters are getting messed up with some junk values when I run the script. They are not proplery shown when I open the input file in VI editor.

My question is, in the database we have the settings to be changed like charset and collatiion for recogninsing the international characters.

Is there such thing on Linux or UNIX for identifying and displaying the international characters?

I am to happy find a similar requirement with a solution which matches with my requirement. But I am stuck because of the filesize difference.

Please help me.

Merlin Joseph

There are different "locale"s, that can have different character sets.
Print the current locale:

Code:

locale

List the installed locales:

Code:

locale -a

Set a UTF (multibyte) locale, for example:

Code:

export LC_ALL; LC_ALL=en_US.ISO8859-1