Sponsored Content
Top Forums Shell Programming and Scripting Help replacing or scrubbing unicode characters Post 302159855 by RTM on Friday 18th of January 2008 10:31:16 PM
Old 01-18-2008
Not sure it will work, but dos2unix may fix it. Check the man page - dos2unix and unix2dos should be on your system.
 

10 More Discussions You Might Find Interesting

1. Programming

How to display unicode characters / unicode string

I have a stream of characters like "\u8BBE\u5907\u7BA1" and i want to display it. I tried following things already without any luck. 1) printf("%s",L("\u8BBE\u5907\u7BA1")); 2) printf("%lc",0x8BBE); 3) setlocale followed by fwide followed by wprintf 4) also changed the local manually... (3 Replies)
Discussion started by: jackdorso
3 Replies

2. AIX

problem with Unicode characters insertion

hi, I have a problem with unicode chars ( chinese, japanese etc ) insertion using sqlplus prompt. When i wrote a proc program for it i am able to create records. But when i fore the same query on sql prompt it stores reverse ????? ..some junk. widechar columns are mapped with NVARCHAR datatype.... (0 Replies)
Discussion started by: suman_jakkula
0 Replies

3. UNIX for Dummies Questions & Answers

replacing characters

Hi, I have a script for replacing bad characters in filenames for f in *; do mv $f `echo $f | tr '+' '_'` done; this replaces + for _ But I need to replace all bad characters ? / % + to _ Pls how can i do this in one script ? (3 Replies)
Discussion started by: palmer18
3 Replies

4. Shell Programming and Scripting

replacing characters

hi all I have a file that has sone spaces in start then / at last. i want to get rid of this. how to do? eg. 11414/ 49878/ 27627/ I WANT THE FILE AS 11414 49878 27627 PLEASE HELP (3 Replies)
Discussion started by: infyanurag
3 Replies

5. UNIX for Dummies Questions & Answers

remove special and unicode characters

Hi, How do I remove the lines where special characters or Unicode characters appear? The following query does work but I wonder if there is a better way. cat test.txt | egrep -v '\)|#|,|&|-|\(|\\|\/|\.' The following lines show that my query is incomplete. Warning: The word "*Khan" is... (1 Reply)
Discussion started by: shantanuo
1 Replies

6. Shell Programming and Scripting

Replacing characters

Hi fellow experts, I have a question for you. Data looks like: 00877,05/13/2010,PBO,P,0000708331,518 00877,05/13/2010,PBO,P,0000708331,519 ... ... 00877,05/13/2010,PBO,P,0000708331,2103 00877,05/13/2010,PBO,P,0000708331,2104,etc,etc Basically I have to replace 518,519,2103,2104,... (4 Replies)
Discussion started by: Devski123
4 Replies

7. Programming

How to make gl_get_line read unicode characters

Hi, My program uses gl_get_line from libtecla to get user input from terminal. It works fine as long as I enter English at the terminal prompt. However, if I enter other languages, such as Chinese characters, either by typing in or cut-and-paste, the input characters get cleared from terminal... (5 Replies)
Discussion started by: tdw
5 Replies

8. Shell Programming and Scripting

Perl script backspace not working for Unicode characters

Hello, My Perl script reads input from stdin and prints it out to stdout. After I read input I use BACKSPACE to erase characters. However BACKSPACE does not work with Unicode characters that are multi-bytes. On screen the character is erased but underneath only one byte is deleted instead of all... (3 Replies)
Discussion started by: tdw
3 Replies

9. Shell Programming and Scripting

sed replacing specific characters and control characters by escaping

sed -e "s// /g" old.txt > new.txt While I do know some control characters need to be escaped, can normal characters also be escaped and still work the same way? Basically I do not know all control characters that have a special meaning, for example, ?, ., % have a meaning and have to be escaped... (11 Replies)
Discussion started by: ijustneeda
11 Replies

10. Shell Programming and Scripting

Display unicode characters in zos shell

Hi all, I have a shell script that has several strings with \uxxxx characters distributed within. I would like to display these characters when I execute the script and echo the strings. I am running on zos in an sh environment. Some strings look like this: "Chcete-li pou\u017e\u00edt" <---... (1 Reply)
Discussion started by: adam.wis
1 Replies
dos2unix(1)							   User Commands						       dos2unix(1)

NAME
dos2unix - convert text file from DOS format to ISO format SYNOPSIS
dos2unix [-ascii] [-iso] [-7] [-437 | -850 | -860 | -863 | -865] originalfile convertedfile DESCRIPTION
The dos2unix utility converts characters in the DOS extended character set to the corresponding ISO standard characters. This command can be invoked from either DOS or SunOS. However, the filenames must conform to the conventions of the environment in which the command is invoked. If the original file and the converted file are the same, dos2unix will rewrite the original file after converting it. OPTIONS
The following options are supported: -ascii Removes extra carriage returns and converts end of file characters in DOS format text files to conform to SunOS requirements. -iso This is the default. It converts characters in the DOS extended character set to the corresponding ISO standard characters. -7 Converts 8 bit DOS graphics characters to 7 bit space characters so that SunOS can read the file. On non-i386 systems, dos2unix will attempt to obtain the keyboard type to determine which code page to use. Otherwise, the default is US. The user may override the code page with one of the following options: -437 Use US code page -850 Use multilingual code page -860 Use Portuguese code page -863 Use French Canadian code page -865 Use Danish code page OPERANDS
The following operands are required: originalfile The original file in DOS format that is being converted to ISO format. convertedfile The new file in ISO format that has been converted from the original DOS file format. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWesu | +-----------------------------+-----------------------------+ SEE ALSO
unix2dos(1), ls(1), attributes(5) DIAGNOSTICS
File filename not found, or no read permission The input file you specified does not exist, or you do not have read permission. Check with the SunOS command, ls -l (see ls(1)). Bad output filename filename, or no write permission The output file you specified is either invalid, or you do not have write permission for that file or the directory that contains it. Check also that the drive or diskette is not write-protected. Error while writing to temporary file An error occurred while converting your file, possibly because there is not enough space on the current drive. Check the amount of space on the current drive using the DIR command. Also be certain that the default diskette or drive is write-enabled (not write-pro- tected). Notice that when this error occurs, the original file remains intact. Translated temporary file name = filename. Could not rename temporary file to filename. The program could not perform the final step in converting your file. Your converted file is stored under the name indicated on the second line of this message. SunOS 5.11 14 Sep 2000 dos2unix(1)
All times are GMT -4. The time now is 03:29 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy