Shell Programming and Scripting

Hi,

I have a pipe delimited file. I am checking for junk characters ( non printable characters and unicode values).
I am using the following code

Code:

grep '[^ -~]' file.txt

But i want to ignore the name fields. For example field2 is firstname so i want to ignore if the junk characters occur there and not consider it as a bad record.

Also, I want to process the file at once and not line by line in a loop. Thats why i was using hte above command.

---------- Post updated at 03:02 AM ---------- Previous update was at 02:13 AM ----------

Any help??

may be..

Code:

awk -F "|" '$1 ~ /[^ -~]/ || $3 ~ /[^ -~]/ || $4 ~ /[^ -~]/ {print}' file

I have around 150 columns.. It might not look so good.. instead is there any way to do it the other way round.. check for not equal to..

Is there a way to ignore 2 fields and check for rest of the fields.I am not able to achieve that in a single go, by reading the file in one shot.

Code:

awk -F "|" '{for(i=1;i<=150;i++) {if(i != 2 && $i ~ /[^ -~]/) {print}}}' file | sort -u

If you want detailed info about the matches, you can do something like this:

Code:

awk -F "|" '{for(i=1;i<=6;i++) {if(i != 2 && $i ~ /[^ -~]/) {print "found in field "i": "$0}}}' file

There could also many other ways.

The command below will look for all control characters and unicode characters.Is it correct.

Code:

grep '[^ -~]' file.txt