Login or Register to Ask a Question and Join Our Community

Linux and UNIX Man Pages

Linux & Unix Commands - Search Man Pages

utf(6) [plan9 man page]

UTF(6)								   Games Manual 							    UTF(6)

NAME

       UTF, Unicode, ASCII, rune - character set and format

DESCRIPTION

       The  Plan 9 character set and representation are based on the Unicode Standard and on the ISO multibyte UTF-8 encoding (Universal Character
       Set Transformation Format, 8 bits wide).  The Unicode Standard represents its characters in 16 bits; UTF-8 represents  such  values  in	an
       8-bit byte stream.  Throughout this manual, UTF-8 is shortened to UTF.

       In Plan 9, a rune is a 16-bit quantity representing a Unicode character.  Internally, programs may store characters as runes.  However, any
       external manifestation of textual information, in files or at the interface  between  programs,	uses  a  machine-independent,  byte-stream
       encoding called UTF.

       UTF  is	designed so the 7-bit ASCII set (values hexadecimal 00 to 7F), appear only as themselves in the encoding.  Runes with values above
       7F appear as sequences of two or more bytes with values only from 80 to FF.

       The UTF encoding of the Unicode Standard is backward compatible with ASCII: programs presented only with ASCII work on Plan 9 even  if  not
       written	to deal with UTF, as do programs that deal with uninterpreted byte streams.  However, programs that perform semantic processing on
       ASCII graphic characters must convert from UTF to runes in order to work properly with non-ASCII input.	See rune(2).

       Letting numbers be binary, a rune x is converted to a multibyte UTF sequence as follows:

       01.   x in [00000000.0bbbbbbb] -> 0bbbbbbb
       10.   x in [00000bbb.bbbbbbbb] -> 110bbbbb, 10bbbbbb
       11.   x in [bbbbbbbb.bbbbbbbb] -> 1110bbbb, 10bbbbbb, 10bbbbbb

       Conversion 01 provides a one-byte sequence that spans the ASCII character set in a compatible way.  Conversions 10 and 11 represent higher-
       valued  characters  as  sequences of two or three bytes with the high bit set.  Plan 9 does not support the 4, 5, and 6 byte sequences pro-
       posed by X-Open.  When there are multiple ways to encode a value, for example rune 0, the shortest encoding is used.

       In the inverse mapping, any sequence except those described above is incorrect and is converted to rune hexadecimal 0080.

FILES

       /lib/unicode
	      table of characters and descriptions, suitable for look(1).

SEE ALSO

       ascii(1), tcs(1), rune(2), keyboard(6), The Unicode Standard.

																	    UTF(6)

Check Out this Related Man Page

RUNE(2) 							System Calls Manual							   RUNE(2)

NAME

       runetochar, chartorune, runelen, fullrune, utflen, utfrune, utfrrune, utfutf - rune/UTF conversion

SYNOPSIS

       #include <u.h>
       #include <libc.h>

       int    runetochar(char *s, Rune *r)

       int    chartorune(Rune *r, char *s)

       int    runelen(long r)

       int    fullrune(char *s, int n)

       int    utflen(char *s)

       char*  utfrune(char *s, long c)

       char*  utfrrune(char *s, long c)

       char*  utfutf(char *s1, char *s2)

DESCRIPTION

       These routines convert to and from a UTF byte stream and runes.

       Runetochar  copies  one	rune  at  r to at most UTFmax bytes starting at s and returns the number of bytes copied.  UTFmax, defined as 3 in
       <libc.h>, is the maximum number of bytes required to represent a rune.

       Chartorune copies at most UTFmax bytes starting at s to one rune at r and returns the number of bytes copied.  If the input is not  exactly
       in UTF format, chartorune will convert to 0x80 and return 1.

       Runelen returns the number of bytes required to convert r into UTF.

       Fullrune  returns  1  if the string s of length n is long enough to be decoded by chartorune and 0 otherwise.  This does not guarantee that
       the string contains a legal UTF encoding.  This routine is used by programs that obtain input a byte at a time and need to know when a full
       rune has arrived.

       The following routines are analogous to the corresponding string routines with utf substituted for str and rune substituted for chr.

       Utflen returns the number of runes that are represented by the UTF string s.

       Utfrune	(utfrrune) returns a pointer to the first (last) occurrence of rune c in the UTF string s, or 0 if c does not occur in the string.
       The NUL byte terminating a string is considered to be part of the string s.

       Utfutf returns a pointer to the first occurrence of the UTF string s2 as a UTF substring of s1, or 0 if there is none.  If s2 is  the  null
       string, utfutf returns s1.

SOURCE

       /sys/src/libc/port/rune.c
       /sys/src/libc/port/utfrune.c

SEE ALSO

       utf(6), tcs(1)

																	   RUNE(2)
Man Page