Sponsored Content
Full Discussion: strlen for UTF-8
Top Forums Programming strlen for UTF-8 Post 302413935 by jim mcnamara on Sunday 18th of April 2010 12:20:39 PM
Old 04-18-2010
AAGH. Don't modify limits.h! You will break the compiler.

The size of a UTF-8 (8 bits) char is one byte.

You code has issues - sizeof(something) tells you how much space the compiler allocates for something, not the length of a character of a string. The answer is 1 for a single char.

mblen() gets the length of wide characters in bytes, use on wide characters.
 

10 More Discussions You Might Find Interesting

1. Programming

Problems with Strlen

hello, i have a problem with strlen. I have written this: for(y=13,z=0; cInBuf!=' ';y++) { cBuf=cInBuf; z++; } len = strlen(cBuf); out=len/2; fprintf(outfile,"F%i",out); If strlen is e.g. 22, it write F22. I want to write F2F2. How can i do this?... (5 Replies)
Discussion started by: ACeD
5 Replies

2. Shell Programming and Scripting

Problem with the strlen function in ksh

Hello, Just a little problem with the ksh function : strlen I want to use this function in this little ksh program : while read line ; do TOTO=$line TOTONB=strlen($TOTO) echo $TOTONB (3 Replies)
Discussion started by: steiner
3 Replies

3. Shell Programming and Scripting

UTF 8 and SED

Collegues I tried to manipulate a UTF 8 data using the following script. cat $1 | sed 's/ലായി$/ലായി LAYI/g' | sed 's/ുടെ/ുടെ UTE/g' | sed 's/യില്*/യില്* YIL/g' But it says that cnot exicute binary file. Any solution. Jaganadh. Linguist (1 Reply)
Discussion started by: jaganadh
1 Replies

4. Programming

'strlen' of a constant string

In a declaration, I have: const char comment_begin = "<!--"; const char comment_end = "-->"; const int comment_begin_len = strlen(comment_begin); const int comment_end_len = strlen(comment_end); When I compile, I get the warnings: emhttpc.c:64: warning: initializer element is not... (10 Replies)
Discussion started by: cleopard
10 Replies

5. Programming

pointer arithmetic vs. strlen() & strnlen()?

I have been getting some flack recently for my use of strlen() and strnlen(). Honestly I have always just taken their functionality for granted as being the easiest way of getting the length of a string. Is it really so much better to do pointer arithmetic? What am I gaining besides more... (3 Replies)
Discussion started by: jjinno
3 Replies

6. UNIX for Advanced & Expert Users

vi and UTF-8 errors

We just installed icu for UTF-8 compliance on our AIX 5.3 system. While usuing vi on some files we get the following error: ex: 0602-169 Incomplete or invalid multibyte character encountere yte character encountered, conversion failed.ex: 0602-169 Incomplete or invalidb ractersultibyte... (0 Replies)
Discussion started by: jlacasci
0 Replies

7. UNIX for Dummies Questions & Answers

UTF-8 in xterm

I need to use sort, uniq, grep, wc,... and the like to work with lists of words in UTF-8 (the "words" being phonetic transcriptions using the IPA). I have been using Google a lot and I even found at least one previous post on this topic, but it didn't help. I tried following the instructions... (2 Replies)
Discussion started by: mregine
2 Replies

8. Linux

Help to Convert file from UNIX UTF-8 to Windows UTF-16

Hi, I have tried to convert a UTF-8 file to windows UTF-16 format file as below from unix machine unix2dos < testing.txt | iconv -f UTF-8 -t UTF-16 > out.txt and i am getting some chinese characters as below which l opened the converted file on windows machine. LANG=en_US.UTF-8... (3 Replies)
Discussion started by: phanidhar6039
3 Replies

9. Programming

Segment fault related to strlen.S

Hello, This function was copied into my code, which was compiled without error/warning, but when executed there is always Segmentation fault at the end after the output (which seems correct!): void get_hashes(unsigned int hash, unsigned char *in) { unsigned char *str = in; int pos =... (7 Replies)
Discussion started by: yifangt
7 Replies

10. Shell Programming and Scripting

PHP: declared variables, strlen vs isset

greetings, pretty new to php and i think i might be missing some fundamental limitation of isset. i have two php scripts below that are executed by crond, one using --host X and one that does not. and below that are three different attempts at generating a command line that will be executed. the... (8 Replies)
Discussion started by: crimso
8 Replies
UTF8_ENCODE(3)								 1							    UTF8_ENCODE(3)

utf8_encode - Encodes an ISO-8859-1 string to UTF-8

SYNOPSIS
string utf8_encode (string $data) DESCRIPTION
This function encodes the string $data to UTF-8, and returns the encoded version. UTF-8 is a standard mechanism used by Unicode for encoding wide character values into a byte stream. UTF-8 is transparent to plain ASCII characters, is self-synchronized (meaning it is possible for a program to figure out where in the bytestream characters start) and can be used with normal string comparison functions for sorting and such. PHP encodes UTF-8 characters in up to four bytes, like this: UTF-8 encoding +------+-------------------------------------+---+ |bytes | | | | | | | | | bits | | | | | | | | representation | | | | | | +------+-------------------------------------+---+ | 1 | | | | | | | | | 7 | | | | | | | | 0bbbbbbb | | | | | | | 2 | | | | | | | | | 11 | | | | | | | | 110bbbbb 10bbbbbb | | | | | | | 3 | | | | | | | | | 16 | | | | | | | | 1110bbbb 10bbbbbb 10bbbbbb | | | | | | | 4 | | | | | | | | | 21 | | | | | | | | 11110bbb 10bbbbbb 10bbbbbb 10bbbbbb | | | | | | +------+-------------------------------------+---+ Each b represents a bit that can be used to store character data. PARAMETERS
o $data - An ISO-8859-1 string. RETURN VALUES
Returns the UTF-8 translation of $data. SEE ALSO
utf8_decode(3). PHP Documentation Group UTF8_ENCODE(3)
All times are GMT -4. The time now is 11:27 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy