Programming

Hi,

My program uses gl_get_line from libtecla to get user input from terminal. It works fine as long as I enter English at the terminal prompt. However, if I enter other languages, such as Chinese characters, either by typing in or cut-and-paste, the input characters get cleared from terminal right away. I set the locale LC_CTYPE to "en_US.UTF-8". Below is the code:

Code:

    printf("LC_CTYPE=%s\n", setlocale(LC_CTYPE, "en_US.UTF-8"));
    printf("LANG=%s\n", getenv("LANG"));
    gl = new_GetLine(1024, 2048);
    while((line=gl_get_line(gl, "input> ", NULL, -1)) != NULL &&
            strcmp(line, "exit\n") != 0)
    {
        printf("You typed: %s!\n", line);
    }
    gl = del_GetLine(gl);

The output showed that the locale (both LC_CTYPE and LANG) was set to en_US.UTF-8:

Code:

LC_CTYPE=en_US.UTF-8
LANG=en_US.UTF-8

I need to read user input in any languages, i.e., any utf-8 characters. Can someone please help me? Big thanks.

I don't think it's readline that's the problem, as much as your terminal's setting. adjusting the LC_CTYPE variable won't adjust what charset your terminal is configured for, whatever your terminal is.

Before I start my program, I can enter Chinese characters in my shell (bash). Both input and display work fine. Once I start my program from the shell, my terminal can still display characters in other languages, such as Chinese or German. If I enter command in English, my program call read it (by calling gl_get_line()), interpret it, generate content in other languages and display it to terminal just fine. The problem only had to do with my program taking input from terminal, such as:

1. If I copy some Chinese characters from a browser and paste them to the terminal where my program is waiting for input (in gl_get_line()), those characters do not appear in the terminal at all.

2. If I type in Chinese characters at my program's terminal, using a standard keyboard but with special input method from a Microsoft program that can convert sequence of English letters to Chinese characters, the Chinese characters do appear in the terminal. But as soon as I press space key which indicates the end of Chinese character input sequence and subsequently (I think) pass the control back to gl_get_line(), the characters I have just entered disappear from the terminal.

It feels like for some reason gl_get_line() erases any character that goes beyond standard ASCII range (0-127). I have read the doc for gl_get_line() and it has some info about international character support but not enough details that will help me resolve my problem. Can anyone provide some help?

Does your program include

Code:

setlocale(LC_CTYPE, "");

and is your locale correctly set?

See tecla(5) – User interface provided by the tecla library for how to enter what Sun calls international characters.

I explicitly set LC_CTYPE to en_US.UTF-8 to avoid any confusion. But at one point I did try with setlocale(LC_CTYPE, "") and result was same because the default locale in my env was set to en_US.UTF-8. I even tried to run the original demo.c that came with the libtecla download package and same problem - any foreign characters I entered or pasted at the terminal were erased.

I have confirmed with Martin Shepherd the author of libtecla that it does not support Unicode input. It only supports extended ASCII to 8-bits, such as ISO-8859-1.