Sponsored Content
Top Forums Shell Programming and Scripting Help replacing or scrubbing unicode characters Post 302159855 by RTM on Friday 18th of January 2008 10:31:16 PM
Old 01-18-2008
Not sure it will work, but dos2unix may fix it. Check the man page - dos2unix and unix2dos should be on your system.
 

10 More Discussions You Might Find Interesting

1. Programming

How to display unicode characters / unicode string

I have a stream of characters like "\u8BBE\u5907\u7BA1" and i want to display it. I tried following things already without any luck. 1) printf("%s",L("\u8BBE\u5907\u7BA1")); 2) printf("%lc",0x8BBE); 3) setlocale followed by fwide followed by wprintf 4) also changed the local manually... (3 Replies)
Discussion started by: jackdorso
3 Replies

2. AIX

problem with Unicode characters insertion

hi, I have a problem with unicode chars ( chinese, japanese etc ) insertion using sqlplus prompt. When i wrote a proc program for it i am able to create records. But when i fore the same query on sql prompt it stores reverse ????? ..some junk. widechar columns are mapped with NVARCHAR datatype.... (0 Replies)
Discussion started by: suman_jakkula
0 Replies

3. UNIX for Dummies Questions & Answers

replacing characters

Hi, I have a script for replacing bad characters in filenames for f in *; do mv $f `echo $f | tr '+' '_'` done; this replaces + for _ But I need to replace all bad characters ? / % + to _ Pls how can i do this in one script ? (3 Replies)
Discussion started by: palmer18
3 Replies

4. Shell Programming and Scripting

replacing characters

hi all I have a file that has sone spaces in start then / at last. i want to get rid of this. how to do? eg. 11414/ 49878/ 27627/ I WANT THE FILE AS 11414 49878 27627 PLEASE HELP (3 Replies)
Discussion started by: infyanurag
3 Replies

5. UNIX for Dummies Questions & Answers

remove special and unicode characters

Hi, How do I remove the lines where special characters or Unicode characters appear? The following query does work but I wonder if there is a better way. cat test.txt | egrep -v '\)|#|,|&|-|\(|\\|\/|\.' The following lines show that my query is incomplete. Warning: The word "*Khan" is... (1 Reply)
Discussion started by: shantanuo
1 Replies

6. Shell Programming and Scripting

Replacing characters

Hi fellow experts, I have a question for you. Data looks like: 00877,05/13/2010,PBO,P,0000708331,518 00877,05/13/2010,PBO,P,0000708331,519 ... ... 00877,05/13/2010,PBO,P,0000708331,2103 00877,05/13/2010,PBO,P,0000708331,2104,etc,etc Basically I have to replace 518,519,2103,2104,... (4 Replies)
Discussion started by: Devski123
4 Replies

7. Programming

How to make gl_get_line read unicode characters

Hi, My program uses gl_get_line from libtecla to get user input from terminal. It works fine as long as I enter English at the terminal prompt. However, if I enter other languages, such as Chinese characters, either by typing in or cut-and-paste, the input characters get cleared from terminal... (5 Replies)
Discussion started by: tdw
5 Replies

8. Shell Programming and Scripting

Perl script backspace not working for Unicode characters

Hello, My Perl script reads input from stdin and prints it out to stdout. After I read input I use BACKSPACE to erase characters. However BACKSPACE does not work with Unicode characters that are multi-bytes. On screen the character is erased but underneath only one byte is deleted instead of all... (3 Replies)
Discussion started by: tdw
3 Replies

9. Shell Programming and Scripting

sed replacing specific characters and control characters by escaping

sed -e "s// /g" old.txt > new.txt While I do know some control characters need to be escaped, can normal characters also be escaped and still work the same way? Basically I do not know all control characters that have a special meaning, for example, ?, ., % have a meaning and have to be escaped... (11 Replies)
Discussion started by: ijustneeda
11 Replies

10. Shell Programming and Scripting

Display unicode characters in zos shell

Hi all, I have a shell script that has several strings with \uxxxx characters distributed within. I would like to display these characters when I execute the script and echo the strings. I am running on zos in an sh environment. Some strings look like this: "Chcete-li pou\u017e\u00edt" <---... (1 Reply)
Discussion started by: adam.wis
1 Replies
UNICODE(1)						      General Commands Manual							UNICODE(1)

NAME
unicode - command line unicode database query tool SYNOPSIS
unicode [options] string DESCRIPTION
This manual page documents the unicode command. unicode is a command line unicode database query tool. OPTIONS
-h --help Show help and exit. -x --hexadecimal Assume string to be a hexadecimal number -d --decimal Assume string to be a decimal number -r --regexp Assume string to be a regular expression -s --string Assume string to be a sequence of characters -a --auto Try to guess type of string from one of the above (default) -mMAXCOUNT --max=MAXCOUNT Maximal number of codepoints to display, default: 20; use 0 for unlimited -iCHARSET --io=IOCHARSET I/O character set. For maximal pleasure, run unicode on UTF-8 capable terminal and specify IOCHARSET to be UTF-8. unicode tries to guess this value from your locale, so with properly set up locale, you should not need to specify it. -cADDCHARSET --charset-add=ADDCHARSET Show hexadecimal reprezentation of displayed characters in this additional charset. -CUSE_COLOUR --colour=USE_COLOUR USE_COLOUR is one of on off auto --colour=on will use ANSI colour codes to colourise the output --colour=off won't use colours. --colour=auto will test if standard output is a tty, and use colours only when it is. --color is a synonym of --colour -v --verbose Be more verbose about displayed characters, e.g. display Unihan information, if available. -w --wikipedia Spawn browser pointing to Wikipedia entry about the character. USAGE
unicode tries to guess the type of an argument. For example, you can use any of the following to display information about U+00E1 LATIN SMALL LETTER A WITH ACUTE (a): unicode 00E1 unicode U+00E1 unicode a unicode 'latin small letter a with acute' You can specify a range of characters as argumets, unicode will show these characters in nice tabular format, aligned to 256-byte bound- aries. Use two dots ".." to indicate the range, e.g. unicode 0450..0520 will display the whole cyrillic and hebrew blocks (characters from U+0400 to U+05FF) unicode 0400.. will display just characters from U+0400 up to U+04FF BUGS
Tabular format does not deal well with full-width, combining, control and RTL characters. SEE ALSO
ascii(1) AUTHOR
Radovan Garabik <garabik @ kassiopeia.juls.savba.sk> 2003-01-31 UNICODE(1)
All times are GMT -4. The time now is 10:16 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy