Sponsored Content
Top Forums UNIX for Advanced & Expert Users ISO 88591 file encoding charset in Linux Post 302768763 by sonic_air on Saturday 9th of February 2013 01:02:41 PM
Old 02-09-2013
ISO 88591 file encoding charset in Linux

Hello Experts, please help to provide any insight as I am facing issue migrating java application from hpux to redhat. The java program is using InputStreamReader to read a file without specifying any charset parameter.

However, in new Linux Redhat 5.6 environent, when reading a file that contains Latin char, I have to either
1) set my locale to ISO 88591 or
2) specifying InputStreamReader to read as ISO 88591 or
3) convert the file using iconv from ISO 88591 to UTF-8
to read the file contents correctly.

The problem here is, I need to read files from different encoding, thus option 1 & 2 is out. While for option 3, the file is some kind of binary file, I could not use file command to determine the file encoding before issue iconv. Smilie

To my knowledge Java InputStreamReader will use system's locale setting if no charset is specified.

Code:
New server: Red Hat Enterprise Linux Server release 5.6 (Tikanga)
(gcc version 4.1.2 20080704 (Red Hat 4.1.2-50))
locale:
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Code:
Old server: HP-UX czhs0850 B.11.11
locale:
LANG=
LC_CTYPE="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_MESSAGES="C"
LC_ALL=

I am not sure why previous hpux does not have this problem, it seems like the same file is interpreted as UTF8 while it reach hpux server but treated as ISO 88591 in Redhat server. I have tried to change Redhat locale to "C" as well but it's not working either.

Could it be I need to specify the encoding to UTF8 while mounting the file system? so that all incoming files write to the server will be treated as UTF8?
 

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

.iso in linux???

dear experts How can I extract the files and install an application that has a postfix of .iso ??? could you please advise what is iso in linux Red hat and what is a file.iso ?? Very Best Regards (7 Replies)
Discussion started by: Reza Nazarian
7 Replies

2. Red Hat

mounting ISO in linux

Hi Guys, I'm having a bit of trouble and im not sure what is the deal, I'm trying to mount an ISO on my RHEL box and it is not letting me... mount -o loop -t iso9660 /home/bgalante/rhel-5-server-i386-disc3.iso /mnt mount: Not a directory any idea what i am doing wrong? (2 Replies)
Discussion started by: BG_JrAdmin
2 Replies

3. Shell Programming and Scripting

Problem identifying charset of a file

Hi all, My objective is to find out the charset using which a file is encoded. (The OS is SunOs) I have set NLS_LANG to AR8MSWIN1256 and spooled the file. When viewed the file using vi, I saw the following \307\341\321\355\307\326 I then inserted the line containing these codes in a... (3 Replies)
Discussion started by: sridhar_423
3 Replies

4. Shell Programming and Scripting

How to find the file encoding and updating the file encoding?

Hi, I am beginner to Unix. My requirement is to validate the encoding used in the incoming file(csv,txt).If it is encoded with UTF-8 format,then the file should remain as such otherwise i need to chnage the encoding to UTF-8. Please advice me how to proceed on this. (7 Replies)
Discussion started by: cnraja
7 Replies

5. Red Hat

How to make boot.iso image from rhel6 installation dvd iso ?

Hello Everyone, Can anyone let me know how to make minimal boot.iso from rhl6 installation dvd iso image. I have a dvd image with me but i want to make just a minimal boot media. Somehow it is not shipped with dvd iso. I know we can download boot.iso from redhat site but is there any anyway we... (5 Replies)
Discussion started by: Rohit Bhanot
5 Replies

6. UNIX for Dummies Questions & Answers

RHEL6 Terminal Charset Encoding

Hi All, I'm facing an issue when i ssh to a router and exporting the output to a txt file. ssh johndoe@10.0.0.1 -a | tee file.txt Closing the connection and opening the .txt file. There are strange 'domino's' appearing here and there. See the screenshot below. ... (2 Replies)
Discussion started by: Antonio Fargas
2 Replies

7. Red Hat

RHEL Linux 6.2 to 7.3 - Bootable iso

Hi All On one of my sandbox machine, I want to replace RHEL 6.2 to RHEL 7.3. I am using both developer editions. rhel-server-7.3-x86_64-dvd.iso ... This is what I have downloaded from Red Hat Enterprise Linux Download | Red Hat Developers My understanding is this file would work as a... (1 Reply)
Discussion started by: videsh77
1 Replies

8. Solaris

View file encoding then change encoding.

Hi all!! I´m using command file -i myfile.xml to validate XML file encoding, but it is just saying regular file . I´m expecting / looking an output as UTF8 or ANSI / ASCII Is there command to display the files encoding? Thank you! (2 Replies)
Discussion started by: mrreds
2 Replies
GIT-CHECK-ATTR(1)						    Git Manual							 GIT-CHECK-ATTR(1)

NAME
git-check-attr - Display gitattributes information SYNOPSIS
git check-attr [-a | --all | attr...] [--] pathname... git check-attr --stdin [-z] [-a | --all | attr...] < <list-of-paths> DESCRIPTION
For every pathname, this command will list if each attribute is unspecified, set, or unset as a gitattribute on that pathname. OPTIONS
-a, --all List all attributes that are associated with the specified paths. If this option is used, then unspecified attributes will not be included in the output. --cached Consider .gitattributes in the index only, ignoring the working tree. --stdin Read file names from stdin instead of from the command-line. -z The output format is modified to be machine-parseable. If --stdin is also given, input paths are separated with a NUL character instead of a linefeed character. -- Interpret all preceding arguments as attributes and all following arguments as path names. If none of --stdin, --all, or -- is used, the first argument will be treated as an attribute and the rest of the arguments as pathnames. OUTPUT
The output is of the form: <path> COLON SP <attribute> COLON SP <info> LF unless -z is in effect, in which case NUL is used as delimiter: <path> NUL <attribute> NUL <info> NUL <path> is the path of a file being queried, <attribute> is an attribute being queried and <info> can be either: unspecified when the attribute is not defined for the path. unset when the attribute is defined as false. set when the attribute is defined as true. <value> when a value has been assigned to the attribute. Buffering happens as documented under the GIT_FLUSH option in git(1). The caller is responsible for avoiding deadlocks caused by overfilling an input buffer or reading from an empty output buffer. EXAMPLES
In the examples, the following .gitattributes file is used: *.java diff=java -crlf myAttr NoMyAttr.java !myAttr README caveat=unspecified o Listing a single attribute: $ git check-attr diff org/example/MyClass.java org/example/MyClass.java: diff: java o Listing multiple attributes for a file: $ git check-attr crlf diff myAttr -- org/example/MyClass.java org/example/MyClass.java: crlf: unset org/example/MyClass.java: diff: java org/example/MyClass.java: myAttr: set o Listing all attributes for a file: $ git check-attr --all -- org/example/MyClass.java org/example/MyClass.java: diff: java org/example/MyClass.java: myAttr: set o Listing an attribute for multiple files: $ git check-attr myAttr -- org/example/MyClass.java org/example/NoMyAttr.java org/example/MyClass.java: myAttr: set org/example/NoMyAttr.java: myAttr: unspecified o Not all values are equally unambiguous: $ git check-attr caveat README README: caveat: unspecified SEE ALSO
gitattributes(5). GIT
Part of the git(1) suite Git 1.8.5.3 01/14/2014 GIT-CHECK-ATTR(1)
All times are GMT -4. The time now is 01:53 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy