unidesc(1)						      General Commands Manual							unidesc(1)

NAME

       unidesc - Describe the contents of a Unicode text file

SYNOPSIS

       unidesc ([option flags]) (<file name>)

       If no input file name is supplied, unidesc reads from the standard input.

DESCRIPTION

       unidesc describes the content of a Unicode text file by reporting the character ranges to which different portions of the text belong.  The
       ranges reported include both official Unicode ranges and the constructed language ranges within the Private Use Areas registered  with  the
       Conscript  Unicode  Registry (http://www.evertype.com/standards/csur/).	For each range of characters, unidesc prints the character or byte
       offset of the beginning of the range, the character or byte offset of the end of the range, and the name of the range. Offsets  start  from
       0.

       Since  the  ASCII  digits, punctuation, and whitespace characters are frequently used by other writing systems, by default these characters
       are treated as neutral, that is, as not belonging exclusively to any particular character range.  These characters are treated as belonging
       to the range of whatever characters precede them.

       If  the	input  begins  with neutral characters, they are treated as belonging to the range of whatever characters follow them. If the file
       consists entirely of neutral characters, the range is identified as Neutral followed by Basic Latin in square brackets.

       A magic number identifying the Unicode encoding is not part of the Unicode standard, so pure Unicode files do not contain a  magic  number.
       However,  informal  conventions	have  arisen for this purpose.	If the command line flag -m is given, unidesc will attempt to identify the
       Unicode subtype by examining the first few bytes of the input. If the input is identified as one of the	two  acceptable  types,  UTF-8	or
       native  order  UTF-32,  it will then proceed to describe the contents of the input. Otherwise, it will report what it has learned and exit.
       Note that if the file does contain a magic number, you must use the -m flag. Without this flag unidesc assumes that the input  consists	of
       pure Unicode with the character data beginning immediately.  It will therefore be thrown off by the magic number.

       By  default,  input  is expected to be UTF-8. Native order UTF-32 is also acceptable.  UTF-32 may be specified via the command line flag -u
       or, if the command line flag -m is given, via the magic number.

COMMAND LINE FLAGS

       -b     Give file offsets in bytes rather than characters.

       -d     Treat the ASCII digits as belonging exclusively to the Basic Latin range.

       -h     Print usage information.

       -L     List the Unicode ranges alphabetically.

       -l     List the Unicode ranges by codepoint.

       -m     Check the file's magic number to determine the Unicode subtype.

       -p     Treat ASCII punctuation as belonging exclusively to the Basic Latin range.

       -r     Instead of listing ranges as they are encountered, just list the ranges detected after all input has been read.

       -u     Input is native order UTF-32.

       -v     Print version information.

       -w     Treat ASCII whitespace as belonging exclusively to the Basic Latin range.

SEE ALSO

       uniname

REFERENCES

       Unicode Standard, version 5.0

AUTHOR

       Bill Poser
       billposer@alum.mit.edu

LICENSE

       GNU General Public License

								    June, 2007								unidesc(1)