UNIX for Dummies Questions & Answers

I'm using shell scripting in Applescript. When searching a file with the ANSEL character set (for GEDCOM files) using (grep '1 CHAR ANSEL' filepath) gives the expected result. When searching a UNICODE formatted file (utf-16), searching for text known to exist in the file using (grep '1 CHAR UNICODE' filepath) does not find the string in the file.

How can I grep this UNICODE file? Thanks in advance.

i think utf (8/16) format files doenst have the default end-of-line identifier in such a case, the usual tools applied to other text files cannot be used with.

Try running a wc -l on the file and post the output of number of lines, for pure utf formatted files it would return a zero; for such situations customized codes need to be written.

Thanks. It does turn out to be zero. I haven't a clue what to do here. All I want to do is search such a file for the string "1 CHAR UNICODE". Any help would be appreciated.

See if you have iconv on your box - you can convert utf-16 to utf-8 which is searchable by grep. iconv should be there.

Code:

iconv -f UTF-16 -t UTF-8 myfile > temporary_file
grep '1 CHAR UNICODE' temporary_file

thanks Jim. My Mac installation does have iconv. I'll use a pipe instead, to avoid having to delete a temp file. All I need is a true or false:

iconv -f UTF-16 -t UTF-8 myfile | grep '1 CHAR UNICODE'

works for me. I am such a novice I don't know how to find a list of commands available to me.