Login or Register to Ask a Question and Join Our Community


Software Releases - RSS News

Unicode Utilities 2.26 (Default branch)

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Special Forums News, Links, Events and Announcements Software Releases - RSS News Unicode Utilities 2.26 (Default branch)
# 1  
Old 04-04-2008
Unicode Utilities 2.26 (Default branch)
ImageThe Unicode Utilities are a set of programsfor manipulating and analyzing Unicode text. uniname defaults to printing the character offset of each character, its byte offset, its hex code value, its encoding, the glyph itself, and its name. Command line options allow undesired information to be suppressed and the Unicode range to be added. unidesc reports the character ranges to which different portions of the text belong. unihist generates a histogram of the characters in its input. ExplicateUTF8 is intended for debugging or for learning about Unicode. It determines and explains the validity of a sequence of bytes as a UTF-8 encoding. unirev reverses UTF-8 strings.License: GNU General Public License (GPL)Changes:
This release adds a new utility, unifuzz, whichgenerates test input for programs expectingUnicode. In addition to generating randomsequences of characters, unifuzz can generate acharacter from each range, tokens of variouspotentially problematic characters and sequences,very long lines, strings with embedded nulls, andill-formed UTF-8.Image

More...
Login or Register to Ask a Question

Previous Thread | Next Thread
Login or Register to Ask a Question

LEARN ABOUT DEBIAN

unihist

unihist(1)						      General Commands Manual							unihist(1)

NAME

       unihist - Generate a histogram of the characters in a Unicode file

SYNOPSIS

       unihist ([option flags])

DESCRIPTION

       unihist	generates  a  histogram  of the characters in its input, which must be encoded in UTF-8 Unicode. By default, for each character it
       prints the frequency of the character as a percentage of the total, the absolute number of tokens in the input, the UTF-32 code	in   hexa-
       decimal,  and, if the character is displayable, the glyph itself as UTF-8 Unicode. Command line flags allow unwanted information to be sup-
       pressed.  In particular, note that by suppressing the percentages and counts it is possible to generate a list of the unique characters	in
       the input.

       Output is produced ordered by character code. To sort it in descending order of frequency, pipe the output into the command:

	      sort -k1 -n -r

       By  default,  unihist  handles all of Unicode. To reduce memory usage and increase speed, it may be compiled so as to handle only the Basic
       Multilingual Plane (plane 0) by defining BMPONLY.

COMMAND LINE FLAGS

       -c     Suppress printing of counts and percentages.

       -g     Suppress printing of glyphs.

       -h     Print usage information.

       -u     Suppress printing of the Unicode code as text.

       -v     Print version information.

SEE ALSO

       uniname (1)

REFERENCES

       Unicode Standard, version 5.0

AUTHOR

       Bill Poser
       billposer@alum.mit.edu

LICENSE

       GNU General Public License

								     May, 2008								unihist(1)