How to find the file encoding and updating the file encoding?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to find the file encoding and updating the file encoding?
# 1  
Old 01-12-2011
How to find the file encoding and updating the file encoding?

Hi,
I am beginner to Unix.
My requirement is to validate the encoding used in the incoming file(csv,txt).If it is encoded with UTF-8 format,then the file should remain as such otherwise i need to chnage the encoding to UTF-8.
Please advice me how to proceed on this.
# 2  
Old 01-19-2011
Hi,
Any suggestion for the above question.
# 3  
Old 01-19-2011
You have not said what platform you are on. There are different utilities on different platforms which can help you. For example, Solaris has the auto_ef (Auto Encoding Finder) utility which does what you are want. On GNU/Linux the file utility provides this information.
# 4  
Old 01-19-2011
I need it for AIX platform.
# 5  
Old 01-19-2011
On AIX you probably want to be using iconv

Code:
iconv -f IBM-930 -t UTF-8 infile > newfile

I doubt you can automate the detection of the encoding used on files it it's not specified somehow in the files themselves. Try playing around with AIX file -i and see what it thinks of your input files.

Last edited by Chubler_XL; 05-25-2011 at 11:02 PM.. Reason: Fixed typo in code (iconv not iconf)
# 6  
Old 05-25-2011
can some one tell how to do same think in hp-ux
# 7  
Old 05-25-2011
I think iconv is available on hp-ux too.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to know file encoding?

how can i know what format a file is * example: UTF-8 ANSI UCS2 i am in a... (8 Replies)
Discussion started by: tricampeon81
8 Replies

2. Solaris

View file encoding then change encoding.

Hi all!! I´m using command file -i myfile.xml to validate XML file encoding, but it is just saying regular file . I´m expecting / looking an output as UTF8 or ANSI / ASCII Is there command to display the files encoding? Thank you! (2 Replies)
Discussion started by: mrreds
2 Replies

3. UNIX for Advanced & Expert Users

ISO 88591 file encoding charset in Linux

Hello Experts, please help to provide any insight as I am facing issue migrating java application from hpux to redhat. The java program is using InputStreamReader to read a file without specifying any charset parameter. However, in new Linux Redhat 5.6 environent, when reading a file that... (1 Reply)
Discussion started by: sonic_air
1 Replies

4. UNIX for Dummies Questions & Answers

Determing the encoding of a file

Hi, I am trying to determine the encoding for the file, because to convert to UTF-8, it seems as though I have to know the encoding of the source. Tried this file <filename> give me this: <filename>:data or International Language text Tried to see the locale and this is the output:... (6 Replies)
Discussion started by: MIA651
6 Replies

5. HP-UX

how to find the character encoding of a file in hp_ux

how to find the character encoding of a file in hp_ux (1 Reply)
Discussion started by: alokjyotibal
1 Replies

6. Shell Programming and Scripting

Cygwin vi XML file encoding problem

Hi, I have got a zip (binary) file transferred from MacOS (thus it has additional __MACOSX directory packed inside). On extracting this zip, there are few *.xml files available. When I opened this *.xml file in vim editor using Cygwin (on windows) the editor displayed in the bottom. I tried... (4 Replies)
Discussion started by: royalibrahim
4 Replies

7. AIX

get the file encoding

Hello! The system is AIX 5.3 Give please command or script to get the file encoding (1 Reply)
Discussion started by: vinment
1 Replies

8. Shell Programming and Scripting

get the file encoding

Hello! The system is AIX 5.3 Give please command or script to get the file encoding Thanks (2 Replies)
Discussion started by: vinment
2 Replies

9. Shell Programming and Scripting

Delete original wav file if lame was successful encoding.

In a bash script: src=”cooltrack.wav” dst=”cooltrack.mp3” lame $src $dst I would like to add some line that would delete the source wav file like: rm $src but I would like this only if the encoding was successful. What should I include before deleting the original to check that the... (2 Replies)
Discussion started by: Aia
2 Replies

10. UNIX for Dummies Questions & Answers

File encoding in Unix

1. I have a shell script which creates a file using cat command. How can i find what encoding the file follows (e.g. UTF8, ANSI)? 2. I want to convert that file to PC-ANSI format. How can i achieve that? I am using HP-Unix. (6 Replies)
Discussion started by: ssmallya
6 Replies
Login or Register to Ask a Question