Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Convert to UTF8 File - Unix

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Convert to UTF8 File - Unix
# 1  
Old 11-18-2011
Convert to UTF8 File - Unix
All,

I have several *.dat files which is created in windows (ANSI Endoing) Or PC File format, once I copy those files to unix.

How can I convert those file to utf8 encoding ?
I tired iconv, it says not supported

Please help

Thanks

- S
# 2  
Old 11-18-2011
ANSI needs no translation. UTF-8 is just ANSI plus extended characters.

But iconv works on pure, raw text. The extension ".dat" suggests it's not pure, raw text but some sort of data file with a complex structure. Unfortunately, ".dat" is such a generic extension it could mean almost anything.

What are these files? What made them?
# 3  
Old 11-18-2011
Thanks for the Tip,

actually the .dat files is from a different source system, with a specific delimeter for each field ..etc
you are correct, those are not ANSI files.

could you let me know, in what way we can convert the .dat files to utf8 ?

- S
# 4  
Old 11-18-2011
Not without knowing what it actually is, no. "PC file format" tells me nothing. PC's use lots of different files. ".dat" is especially generic and meaningless.

Try
Code:
hexdump -C filename.dat | head

, then copy-paste what it prints into your next post, inside [code]code tags[/code]

Also try running file filename.dat
Last edited by vgersh99; 11-18-2011 at 03:55 PM.. Reason: added [noparse] tags
# 5  
Old 11-18-2011
I am not sure whether this Information would help.
Source is a AS400 File, sample looks like this

0¿ ¿ ¿abc¿ ¿1¿123¿ ¿test¿test1¿SANDSLI¿5254¿ ¿0¿N¿ ¿0¿0¿0¿NORMAL¿ ¿ ¿000¿000¿TEST¿ ¿0¿0¿ ¿ ¿

with the delimiter '¿'

Thx
- S
# 6  
Old 11-21-2011
Fine. What program in your AS/400 created that file? Smilie

Please do what I asked last week:

Code:
hexdump -C filename.dat | head

so I can see what the actual unmangled contents of the file are.
Last edited by Corona688; 11-21-2011 at 12:21 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Read csv file, convert the data and make one text file in UNIX shell scripting

I have input data looks like this which is a part of a csv file 7,1265,76548,"0102:04" 8,1266,76545,"0112:04" I need to make the output data should look like this and the output data will be part of text file: 7|1265000 |7654899 |A| 8|12660000 |76545999 |B| The logic behind the... (6 Replies)
Discussion started by: RJG
6 Replies

2. UNIX for Advanced & Expert Users

Convert CSV file to nested XML file using UNIX/PERL?

we have a CSV which i need to convert to XML using Perl or Unix shell scripting. I was able to build this XML in oracle database. However, SQL/XML query is running for long time. Hence, I'm considering to write a Perl or shell script to generate this XML file. Basically need to build this XML... (3 Replies)
Discussion started by: laknar
3 Replies

3. Linux

How to create new empty utf8 file for appending?

Hey i try to create empty text file with utf-8 encoding without success what is the right way to do this ? tried with touch test.txt iconv -f UTF-8 -t UTF-8 test.txt or iconv -f latin1 -t UTF-8 test.txt (8 Replies)
Discussion started by: umen
8 Replies

4. UNIX Desktop Questions & Answers

How to convert file in Unix

we have a file that shows encoding as small-endian (in NotePad+). I can not read this file in my program. If I change it to ANSI (again in Notepad+) and save it, I am able to read it in my program. Can I use the iconv function to change this file to ANSI ? Thanks in advance for your help. (4 Replies)
Discussion started by: ricka360
4 Replies

5. Shell Programming and Scripting

Convert all files in current folder from UTF8 to ANSI, name unchanged.

Asking for a Linux command line to convert all files in current folder from UTF8 to ANSI, name unchanged. Best Regards Pei (3 Replies)
Discussion started by: jiapei100
3 Replies

6. Shell Programming and Scripting

Convert MS Excel file to Tab limiter file in UNIX

Hi, We have a couple of ms excel files in unix server.We need convert the excel files to files TAB limiter format file with using unix script. Could you please advise on this (2 Replies)
Discussion started by: koti_rama
2 Replies

7. Shell Programming and Scripting

How to replace comma by slash using sed in an UTF8 file

Hello all, I'd like to replace "," by "/" in a utf8 file from postion X to Y. Comma "," is also defined as delimiter. 12345678901234567890,123456789012345,12345678901234567890, aaaa,aaaa,aaaaa ,bbb,bbbb,bbbbb ,cccccc,cc , Result should be... (1 Reply)
Discussion started by: fmofmo
1 Replies

8. Shell Programming and Scripting

Converting Unicode file to UTF8 format

Hi, I have a file in my desktop which is a unicode format. After this file is transferred to Unix using FTP, we are seeing some special character (like rectangle box type) at the first line. The same file is saved as UTF8 (using textpad tool, selecting encode to UTF-8 option) on my desktopand... (7 Replies)
Discussion started by: vfrg
7 Replies

9. UNIX for Dummies Questions & Answers

Convert UTF8 Format file to ANSI format

:confused: Hi i am trying to convert a file which is in UTF8 format to ANSI format i tried to use the function ICONV but it is throwing error Function i used it as $ iconv -f UTF8 -t ANSI filename Error iam getting is NOT Supported UTF8 to ANSI please some help me out on... (9 Replies)
Discussion started by: rajreddy
9 Replies

10. UNIX for Advanced & Expert Users

Convert UTF8 Format file to ANSI format

:) Hi i am trying to convert a file which is in UTF8 format to ANSI format i tried to use the function ICONV but it is throwing error Function i used it as $ iconv -f UTF8 -t ANSI filename Error iam getting is NOT Supported UTF8 to ANSI please some help me out on this.........Let me... (1 Reply)
Discussion started by: rajreddy
1 Replies
Login or Register to Ask a Question