Perl script backspace not working for Unicode characters


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Perl script backspace not working for Unicode characters
# 1  
Old 02-28-2011
Perl script backspace not working for Unicode characters

Hello,

My Perl script reads input from stdin and prints it out to stdout. After I read input I use BACKSPACE to erase characters. However BACKSPACE does not work with Unicode characters that are multi-bytes. On screen the character is erased but underneath only one byte is deleted instead of all bytes of the character, which makes remaining bytes incomplete UTF-8 characters. Here's my script:

Code:
#!/usr/bin/perl
binmode STDIN, ":encoding(utf8)";
binmode STDOUT, ":encoding(utf8)";
my $str;
while(1) {
  printf("please enter text > ");
  chomp($str = <STDIN>);
  printf("You entered: %s\n", $str);
}

I am using bash in a Putty terminal with UTF-8 encoding. Everything on Unicode works fine in shell including BACKSPACE. It is only a problem in my Perl script. Can anyone help? Thanks.

-Tom
# 2  
Old 03-01-2011
How old is the PERL? It seems to be ignoring the UTF-8 encoding mode. Some googled up articles say backspace should be mapped like delete ^? not ^H! Another suggests issuing the "stty iutf8" command. I thought the tty cooked handling was in the system, not PERL -- PERL just gets a line! You might run it in truss/strace/tusc to see what is passed.

http://linux.die.net/man/7/urxvt
http://sourceforge.net/tracker/index...89&atid=518973
http://www.mail-archive.com/linux-ut.../msg01645.html

Last edited by DGPickett; 03-01-2011 at 05:16 PM..
This User Gave Thanks to DGPickett For This Post:
# 3  
Old 03-01-2011
Thank you very much for the pointers. I followed tips from 2nd link and added the line in my script:

system('stty iutf8');

It worked but not completely. Normal width characters worked fine, like אזחט. But wide characters like Chinese, one BACKSPACE erased all bytes correctly (3 bytes per char), but cursor moved back only one term cell instead of two cells for wide chars. The result was that some Chinese characters were still left visible on terminal even after all of the underlying bytes were deleted.
# 4  
Old 03-08-2011
Well, it is in the realm of stty cooked output and terminal type accuracy. The maintenance of screen display is outside the application, even PERL, too. It is the flip side. I wonder if there is an 'outf8' for stty?

dtterm, xterm and Terminals Capable of Input and Output of UTF-8 Characters (International Language Environments Guide)

The Unicode HOWTO: Locale setup

UTF-8 and Unicode FAQ
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Display unicode characters in zos shell

Hi all, I have a shell script that has several strings with \uxxxx characters distributed within. I would like to display these characters when I execute the script and echo the strings. I am running on zos in an sh environment. Some strings look like this: "Chcete-li pou\u017e\u00edt" <---... (1 Reply)
Discussion started by: adam.wis
1 Replies

2. Shell Programming and Scripting

Remove ^? characters in shell variables on using backspace

Friends, I observed a peculiar problem in shell. if I set a variable using standard input and backspace was used by the user, then the variable get ^? characters embedded in the variable. ### echo "Enter value for X=" read X echo $X expr $X + 1 ### If the variable is echoed, then there... (3 Replies)
Discussion started by: sachinverma
3 Replies

3. Solaris

[SOLVED] Backspace not working!!!!!

Hi friends, Hope u r doing well. It is a very strange problem that I've never faced when I used linux or freebsd. When a type a command in Solaris 10, and if I make a mistake, the backspace doesn't work, when I press the backspace key three times forexample, this is what I get, ^H ^H ^H. The same... (2 Replies)
Discussion started by: gabam
2 Replies

4. Shell Programming and Scripting

stty erase ^h not working for backspace

Hi , I have to press shift + Backspace to do backspace on my unix termminal everytime. How can i configure it to a normal backspace only. Please help me here. PFB the contents of the stty -a : dbtgr@hpxi017:/pocuser/C5/aimsys/dbtgr> stty -a speed 38400 baud; line = 0; rows = 35; columns =... (4 Replies)
Discussion started by: kunwar
4 Replies

5. Programming

How to make gl_get_line read unicode characters

Hi, My program uses gl_get_line from libtecla to get user input from terminal. It works fine as long as I enter English at the terminal prompt. However, if I enter other languages, such as Chinese characters, either by typing in or cut-and-paste, the input characters get cleared from terminal... (5 Replies)
Discussion started by: tdw
5 Replies

6. UNIX for Dummies Questions & Answers

remove special and unicode characters

Hi, How do I remove the lines where special characters or Unicode characters appear? The following query does work but I wonder if there is a better way. cat test.txt | egrep -v '\)|#|,|&|-|\(|\\|\/|\.' The following lines show that my query is incomplete. Warning: The word "*Khan" is... (1 Reply)
Discussion started by: shantanuo
1 Replies

7. Shell Programming and Scripting

Help replacing or scrubbing unicode characters

I have a csv (tab delimited) file that is created by an application (that I didn't write). Every so often it throw out a <U+FEFF> (Zero Width no break space) character at the begining of a tabbed field. The charcater is invisible to some editors, but it shows up bolded in less. The issue is... (3 Replies)
Discussion started by: roninuta
3 Replies

8. AIX

problem with Unicode characters insertion

hi, I have a problem with unicode chars ( chinese, japanese etc ) insertion using sqlplus prompt. When i wrote a proc program for it i am able to create records. But when i fore the same query on sql prompt it stores reverse ????? ..some junk. widechar columns are mapped with NVARCHAR datatype.... (0 Replies)
Discussion started by: suman_jakkula
0 Replies

9. Programming

How to display unicode characters / unicode string

I have a stream of characters like "\u8BBE\u5907\u7BA1" and i want to display it. I tried following things already without any luck. 1) printf("%s",L("\u8BBE\u5907\u7BA1")); 2) printf("%lc",0x8BBE); 3) setlocale followed by fwide followed by wprintf 4) also changed the local manually... (3 Replies)
Discussion started by: jackdorso
3 Replies

10. Shell Programming and Scripting

Backspace Not Working in Script

Hello, I've written a Perl script that prompts for asnwers to questons. At those prompts, the backspace key shows up as ^H^H. I would like the users to have the ablility to use the backspace key. I'm running bash shell and don't otherwise have this problem. Any help would be greatly... (4 Replies)
Discussion started by: Atama
4 Replies
Login or Register to Ask a Question