Encoding of a text issue


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Encoding of a text issue
# 1  
Old 04-29-2013
Encoding of a text issue

I created one file on windows system and is visible as :

Code:
TestTable,INSERT,večilnin1ईगल受害者是第,2010-02-02 10:10:10.612447,137277,ईगल受害者是第večilnin!@#$%^&*()_+=-{}][:;"<>?/.'|\`~@email.com,10021/1.0.0.112,واندلعت بعد ذلك انه فξέσπασ他嚼口香糖强maĉa犠牲者6号でしたuvalหัวใจของเขาโขลก večilnin  6 a tSeinרייסט אַראָפּ איןईगल受害者是第,2010-02-02 10:10:10.612447

But when send this file to unix system, the file is visible as :

Code:
TestTable,INSERT,žvečilnin1ई-ल -害...是第,2010-02-02  10:10:10.612447,137277,ई-ल-害...是第žvečilnin!@#$%^&*()_+=-{}][:;"<& gt;?/.'|\`~@email.com,10021/1.0.0.112,واندلعت بعد  ذلك انه فξ*σπασ-嚼口香-强maĉa 牲...6号で-たŽuvalหัวใจข*งเขาโขลก žvečilnin ° 6 a  étéSeinרייסט אַראָפּ איןई-ल-害...是第,2010-02-02 10:10:10.612447

How can i see the file in it's original form in unix systems as well ?
# 2  
Old 04-29-2013
What are the encoding schemes on both machines? You'll have to either switch one of the encoding schemes or to convert the file (recode, iconv, dos2unix, ...)
# 3  
Old 04-29-2013
in windows, the file was opened with UTF-8 and in unix system , it was
en_US.UTF-8, found by
Code:
echo $LANG

# 4  
Old 04-29-2013
Using en_US.UTF-8on a Unix system, I get:

Code:
$ cat infile
TestTable,INSERT,žvečilnin1ईगल受害者是第,2010-02-02 10:10:10.612447,137277,ईगल受害者是第žvečilnin!@#$%^&*()_+=-{}][:;"<>?/.'|\`~@email.com,10021/1.0.0.112,واندلعت بعد ذلك انه فξέσπασ他嚼口香糖强maĉa犠牲者6号でしたŽuvalหัวใจของเขาโขลก žvečilnin  6 a tSeinרייסט אַראָפּ איןईगल受害者是第,2010-02-02 10:10:10.612447

What is the output of
Code:
locale

on your Unix system?

It could be that the file is in different format when stored on your windows system, then when it is pasted into a web page.. Could you post the original Windows file?
# 5  
Old 04-29-2013
Code:
!@#$%^&*()_+=-{}][:;"<>?/.'|\`~

vs
Code:
!@#$%^&*()_+=-{}][:;"<& gt;?/.'|\`~

See the & gt; part? Figure out why that is there and you will be on your way to a solution. Think!
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Solaris

View file encoding then change encoding.

Hi all!! Im using command file -i myfile.xml to validate XML file encoding, but it is just saying regular file . Im expecting / looking an output as UTF8 or ANSI / ASCII Is there command to display the files encoding? Thank you! (2 Replies)
Discussion started by: mrreds
2 Replies

2. UNIX for Dummies Questions & Answers

Issue with selecting the text in gvim

Hi, I wanted to select the text in gvim using mouse for coping. some times it works fine and full text will be selected and copied to clipboard if it is like shown in first fig. Paste of the copied text can be done by clicking the middle mouse button. But some times the text will be selected as... (2 Replies)
Discussion started by: twistedpair
2 Replies

3. Shell Programming and Scripting

How to find the file encoding and updating the file encoding?

Hi, I am beginner to Unix. My requirement is to validate the encoding used in the incoming file(csv,txt).If it is encoded with UTF-8 format,then the file should remain as such otherwise i need to chnage the encoding to UTF-8. Please advice me how to proceed on this. (7 Replies)
Discussion started by: cnraja
7 Replies

4. UNIX for Dummies Questions & Answers

"vi" text editor character encoding?

Hi! I've got a shell account on a FreeBSD machine. It doesn't have 'vim' installed, but only the original 'vi' text editor ("Version 1.79 (10/23/96) The CSRG, University of California, Berkeley.") So, in PuTTY I've chosen "UTF-8 translation" to have my non-english characters appear correctly.... (2 Replies)
Discussion started by: Gew
2 Replies

5. UNIX for Dummies Questions & Answers

Encoding Type

Hi, Where can I find the encoding type in a unix server ? Thanks in advance !!! (1 Reply)
Discussion started by: risshanth
1 Replies

6. Shell Programming and Scripting

Text formating issue

Hi, I am trying to format the output of my script in a tabular format using the html code in my shell script. I am using html code for the first time in a shell script.I am trying to execute this script in Ksh but it is not getting executed. Can you please help me in figuring this out.... (6 Replies)
Discussion started by: isaacsam
6 Replies

7. AIX

get the file encoding

Hello! The system is AIX 5.3 Give please command or script to get the file encoding (1 Reply)
Discussion started by: vinment
1 Replies

8. UNIX for Dummies Questions & Answers

Sed text replacement issue.

Hi, Im trying to find and replace text within a unix file using sed. The command that i have been using is sed '/,null,/ s//, ,/g' result.txt>result.tmp for replacing ",null," with ", ,". But this only replaces the first occurrance of ,null, in every line. I want to do it globally. It... (7 Replies)
Discussion started by: sohaibs
7 Replies

9. UNIX for Dummies Questions & Answers

encoding

Hi, I'm using putty and when I try to write it writes | (or when I try to write , it writes i) I tried to change settings/translation of putty but with no success I have KSH # locale LANG= LC_CTYPE="C" LC_NUMERIC="C" LC_TIME="C" LC_COLLATE="C" LC_MONETARY="C" LC_MESSAGES="C"... (3 Replies)
Discussion started by: palmer18
3 Replies
Login or Register to Ask a Question