01-21-2008
Do I understand it correctly that you want to frequently get rid of this character from some *.csv files? In my opinion best would be a simple C/C++ application that reads the file byte-by-byte and writes it byte-by-byte with skipping the unwanted ones. To improve performance you could optimize reads/writes.
10 More Discussions You Might Find Interesting
1. Programming
I have a stream of characters like "\u8BBE\u5907\u7BA1"
and i want to display it.
I tried following things already without any luck.
1) printf("%s",L("\u8BBE\u5907\u7BA1"));
2) printf("%lc",0x8BBE);
3) setlocale followed by fwide followed by wprintf
4) also changed the local manually... (3 Replies)
Discussion started by: jackdorso
3 Replies
2. AIX
hi,
I have a problem with unicode chars ( chinese, japanese etc ) insertion using sqlplus prompt.
When i wrote a proc program for it i am able to create records.
But when i fore the same query on sql prompt it stores reverse ????? ..some junk.
widechar columns are mapped with NVARCHAR datatype.... (0 Replies)
Discussion started by: suman_jakkula
0 Replies
3. UNIX for Dummies Questions & Answers
Hi, I have a script for replacing bad characters in filenames
for f in *; do mv $f `echo $f | tr '+' '_'`
done;
this replaces + for _
But I need to replace all bad characters ? / % + to _
Pls how can i do this in one script ? (3 Replies)
Discussion started by: palmer18
3 Replies
4. Shell Programming and Scripting
hi all
I have a file that has sone spaces in start then / at last. i want to get rid of this. how to do?
eg.
11414/
49878/
27627/
I WANT THE FILE AS
11414
49878
27627
PLEASE HELP (3 Replies)
Discussion started by: infyanurag
3 Replies
5. UNIX for Dummies Questions & Answers
Hi,
How do I remove the lines where special characters or Unicode characters appear?
The following query does work but I wonder if there is a better way.
cat test.txt | egrep -v '\)|#|,|&|-|\(|\\|\/|\.'
The following lines show that my query is incomplete.
Warning: The word "*Khan" is... (1 Reply)
Discussion started by: shantanuo
1 Replies
6. Shell Programming and Scripting
Hi fellow experts,
I have a question for you. Data looks like:
00877,05/13/2010,PBO,P,0000708331,518
00877,05/13/2010,PBO,P,0000708331,519
...
...
00877,05/13/2010,PBO,P,0000708331,2103
00877,05/13/2010,PBO,P,0000708331,2104,etc,etc
Basically I have to replace 518,519,2103,2104,... (4 Replies)
Discussion started by: Devski123
4 Replies
7. Programming
Hi,
My program uses gl_get_line from libtecla to get user input from terminal. It works fine as long as I enter English at the terminal prompt. However, if I enter other languages, such as Chinese characters, either by typing in or cut-and-paste, the input characters get cleared from terminal... (5 Replies)
Discussion started by: tdw
5 Replies
8. Shell Programming and Scripting
Hello,
My Perl script reads input from stdin and prints it out to stdout. After I read input I use BACKSPACE to erase characters. However BACKSPACE does not work with Unicode characters that are multi-bytes. On screen the character is erased but underneath only one byte is deleted instead of all... (3 Replies)
Discussion started by: tdw
3 Replies
9. Shell Programming and Scripting
sed -e "s// /g" old.txt > new.txt
While I do know some control characters need to be escaped, can normal characters also be escaped and still work the same way? Basically I do not know all control characters that have a special meaning, for example, ?, ., % have a meaning and have to be escaped... (11 Replies)
Discussion started by: ijustneeda
11 Replies
10. Shell Programming and Scripting
Hi all,
I have a shell script that has several strings with \uxxxx characters distributed within. I would like to display these characters when I execute the script and echo the strings. I am running on zos in an sh environment. Some strings look like this:
"Chcete-li pou\u017e\u00edt" <---... (1 Reply)
Discussion started by: adam.wis
1 Replies
UNICODE(1) General Commands Manual UNICODE(1)
NAME
unicode - command line unicode database query tool
SYNOPSIS
unicode [options] string
DESCRIPTION
This manual page documents the unicode command.
unicode is a command line unicode database query tool.
OPTIONS
-h --help
Show help and exit.
-x --hexadecimal
Assume string to be a hexadecimal number
-d --decimal
Assume string to be a decimal number
-r --regexp
Assume string to be a regular expression
-s --string
Assume string to be a sequence of characters
-a --auto
Try to guess type of string from one of the above (default)
-mMAXCOUNT
--max=MAXCOUNT
Maximal number of codepoints to display, default: 20; use 0 for unlimited
-iCHARSET
--io=IOCHARSET
I/O character set. For maximal pleasure, run unicode on UTF-8 capable terminal and specify IOCHARSET to be UTF-8. unicode tries to
guess this value from your locale, so with properly set up locale, you should not need to specify it.
-cADDCHARSET
--charset-add=ADDCHARSET
Show hexadecimal reprezentation of displayed characters in this additional charset.
-CUSE_COLOUR
--colour=USE_COLOUR
USE_COLOUR is one of on off auto
--colour=on will use ANSI colour codes to colourise the output
--colour=off won't use colours.
--colour=auto will test if standard output is a tty, and use colours only when it is.
--color is a synonym of --colour
-v --verbose
Be more verbose about displayed characters, e.g. display Unihan information, if available.
-w --wikipedia
Spawn browser pointing to Wikipedia entry about the character.
USAGE
unicode tries to guess the type of an argument. For example, you can use any of the following to display information about U+00E1 LATIN
SMALL LETTER A WITH ACUTE (a):
unicode 00E1
unicode U+00E1
unicode a
unicode 'latin small letter a with acute'
You can specify a range of characters as argumets, unicode will show these characters in nice tabular format, aligned to 256-byte bound-
aries. Use two dots ".." to indicate the range, e.g.
unicode 0450..0520
will display the whole cyrillic and hebrew blocks (characters from U+0400 to U+05FF)
unicode 0400..
will display just characters from U+0400 up to U+04FF
BUGS
Tabular format does not deal well with full-width, combining, control and RTL characters.
SEE ALSO
ascii(1)
AUTHOR
Radovan Garabik <garabik @ kassiopeia.juls.savba.sk>
2003-01-31 UNICODE(1)