Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Find Unicode Character in File

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Find Unicode Character in File
# 1  
Old 04-11-2008
Find Unicode Character in File
I have a very large file in Unix that I would like to search for all instances of the unicode character 0x17. I need to remove these characters because the character is causing my SAX Parser to throw an exception. Does anyone know how to find a unicode character in a file?

Thank you for your assistance.
# 2  
Old 04-11-2008
"0x17" is not a Unicode (UTF-16 or UTF-32) character per se.

For those not familiar with Unicode, UTF-16 basically means that
every "character" is stored as 2 bytes whereas UTF-32 means every
"character" is stored as 4 bytes.

On a practical level, it means that most standard ASCII characters are
either preceded by or followed by either a single NUL (0x00) or 3 NULs
depending on whether data storage is Big-Endian or Little-Endian.

Which Unicode "format" is your file using?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find character and Replace character for given position

Hi, i want find the character '-' in a file from position 284-298, if it occurs i need to replace it with 'O ' for the position in the file. How to do that using SED command. thanks in advance, Sara (9 Replies)
Discussion started by: Sara183
9 Replies

2. Shell Programming and Scripting

Find string in a file and append character

Hi Experts, Is there a way to find a string in a file then append a character to that string then save the file or save to another file. Here is an example. >cat test.txt NULL NULL NULL 9,800.00 NULL 1,234,567.01 I want to find all NON NULL String and add a dollar sign to those... (9 Replies)
Discussion started by: brichigo
9 Replies

3. Shell Programming and Scripting

Find position of character in multiple strings in a file

Greetings. I have a file with information like this: AMNDHRKEOEU?AMNDHRKEOEU?AMNDHRKEOEU?AMNDHRKEOEU? AMNDHRKEEU?AMNDHREOEU? AMNDHREU?AHRKEOEU?AMNDHRKEU?AMNDKEOEU? What I need to extract is the position, in every line, of every occurrence of '?' A desired output would be something... (6 Replies)
Discussion started by: Twinklefingers
6 Replies

4. Shell Programming and Scripting

How to find character position in file?

how to find character positionin file? i.e string = "123X568" i want to find the position of character "X". Thanks (6 Replies)
Discussion started by: LiorAmitai
6 Replies

5. HP-UX

how to find the character encoding of a file in hp_ux

how to find the character encoding of a file in hp_ux (1 Reply)
Discussion started by: alokjyotibal
1 Replies

6. Shell Programming and Scripting

How do I replace a unicode character using sed

I have a unicode character {Unicode: 0x1C} in my file and I need to replace it with a blank. How would a sed command look like? cat file1 | sed "s/(//g;" > file2 Is X28 the right value for this Unicode character?? (4 Replies)
Discussion started by: Hangman2
4 Replies

7. Solaris

An invalid XML character (Unicode: 0x1a)

While uploading an exl file to my application in Solaris 10 the upload failed with error Error! Parsing Error: /SPLM/TC83/tcdata83/model/model_dbextract.xml Line:65576 Column:73 An invalid XML character (Unicode: 0x1a) was found in the value of attribute "unitOfMeasureSymbol" and element is ... (12 Replies)
Discussion started by: karghum
12 Replies

8. UNIX for Dummies Questions & Answers

How to find the ^M(control M) character in unix file?

can any one say about command to find "^M" (Control M)characters in a unix text file. ^M comes when a file ftped from windows to unix without using bin mode. I need the command to find lik this, ex.txt: ------------------------------ ...,name,time^M go^M ...file,end^M... (5 Replies)
Discussion started by: prsam
5 Replies

9. UNIX for Dummies Questions & Answers

find and remove rows from file where multi occurrences of character found

I have a '~' delimited file of 6 - 7 million rows. Each row should contain 13 columns delimited by 12 ~'s. Where there are 13 tildes, the row needs to be removed. Each row contains alphanumeric data and occasionally a ~ ends up in a descriptive field and therefore acts as a delimiter, resulting in... (1 Reply)
Discussion started by: kpd
1 Replies

10. Programming

How to display unicode characters / unicode string

I have a stream of characters like "\u8BBE\u5907\u7BA1" and i want to display it. I tried following things already without any luck. 1) printf("%s",L("\u8BBE\u5907\u7BA1")); 2) printf("%lc",0x8BBE); 3) setlocale followed by fwide followed by wprintf 4) also changed the local manually... (3 Replies)
Discussion started by: jackdorso
3 Replies
Login or Register to Ask a Question