Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Convert UTF-8 file to ASCII/ISO8859-1 OR replace characters
Top Forums Shell Programming and Scripting Convert UTF-8 file to ASCII/ISO8859-1 OR replace characters Post 302980631 by RudiC on Wednesday 31st of August 2016 03:22:30 AM
Old 08-31-2016
That is a complex specification to read (and understand). If I did - which I don't dare to assert - a solution to replace "extended" characters with "nomal" ones would be
Code:
echo "\"#$%&0@ABab£¥§©ほぼぽま~अ" | LC_ALL=C sed 'p; s/[\xe0-\xef]../X/g; s/[\xc0-\xcf]./Y/g'
"#$%&0@ABab£¥§©ほぼぽま~अ
"#$%&0@ABabYYYYXXXX~X

Here the characters in question are replaced by X or Y, depending on their representation in UTF-8 - replace them by what you want, e.g. spaces or nothing. If you want to translate € to EUR, do so BEFORE those substitutions above, but be aware that then your "fixed length" will be lost.
Last edited by RudiC; 08-31-2016 at 04:55 AM..
 

10 More Discussions You Might Find Interesting

1. Programming

Howto convert Ascii -> UTF-8 & back C++

While working with russian text under FreeBSD&MySQL I need to convert a string from MySQL to the Unicode format. I've just started my way in C++ under FreeBSD , so please explain me how can I get ascii code of Char variable and also how can i get a character into variable with the specified ascii... (3 Replies)
Discussion started by: macron
3 Replies

2. Shell Programming and Scripting

Replace characters in a string using their ascii value

Hi All, In the HP Unix that i'm using when i initialise a string as Stalled="'30¬G'" Stalled=$Stalled" '30¬C'", it is taking the character ¬ as a comma. I need to grep for 30¬G 30¬C in a file and take its count. But since this character ¬ is not being understood, the count returns a zero. The... (2 Replies)
Discussion started by: roops
2 Replies

3. Shell Programming and Scripting

replace UTF-8 characters with tr

Hi, I try to get tr to replace multibytes characters by ascii equivalent. For example "Je vais à l'école" ---> 'Je vais a l'ecole" But my version of tr (5.97) doesn't seem to support multibyte sets. $ locale charmap; echo "Je vais à l'école" | tr éà ea UTF-8 Je vais aa l'aacole I try to... (2 Replies)
Discussion started by: ripat
2 Replies

4. Shell Programming and Scripting

read in a file character by character - replace any unknown ASCII characters with spa

Can someone help me to write a script / command to read in a file, character by character, replace any unknown ASCII characters with space. then write out the file to a new filename/ Thanks! (1 Reply)
Discussion started by: raghav525
1 Replies

5. Shell Programming and Scripting

convert ascii values into ascii characters

Hi gurus, I have a file in unix with ascii values. I need to convert all the ascii values in the file to ascii characters. File contains nearly 20000 records with ascii values. (10 Replies)
Discussion started by: sandeeppvk
10 Replies

6. Shell Programming and Scripting

Remove characters other than ISO8859-1

Hi please help in writing a script for replacing all the non-iso8859-1 characters to question marks. I need a pattern of this kind "sed s/<non-iso char range>/?/g < ipfile > opfile" Please help me in this. (2 Replies)
Discussion started by: rprajendran
2 Replies

7. Red Hat

Can't convert 7bit ASCII to UTF-8

Hello, I am trying to convert a 7bit ASCII file to UTF-8. I have used iconv before though it can't recognize it for some reason and says unknown file encoding. When I used ascii2uni package with different package, ./ascii2uni -a K -a I -a J -a X test_file > new_test_file It still... (2 Replies)
Discussion started by: rockf1bull
2 Replies

8. Linux

Help to Convert file from UNIX UTF-8 to Windows UTF-16

Hi, I have tried to convert a UTF-8 file to windows UTF-16 format file as below from unix machine unix2dos < testing.txt | iconv -f UTF-8 -t UTF-16 > out.txt and i am getting some chinese characters as below which l opened the converted file on windows machine. LANG=en_US.UTF-8... (3 Replies)
Discussion started by: phanidhar6039
3 Replies

9. Shell Programming and Scripting

Search and Replace Extended Ascii Characters

We are getting extended Ascii characters in the input file and my requirement is to search and replace them with a space. I am using the following command LANG=C sed -e 's// /g' It is doing a good job, but in some cases it is replacing the extended characters with two spaces. So my input... (12 Replies)
Discussion started by: ysvsr1
12 Replies

10. Shell Programming and Scripting

Convert Hex to Ascii in a Ascii file

Hi All, I have an ascii file in which few columns are having hex values which i need to convert into ascii. Kindly suggest me what command can be used in unix shell scripting? Thanks in Advance (2 Replies)
Discussion started by: HemaV
2 Replies

LEARN ABOUT DEBIAN

locale::currency

Locale::Currency(3perl) 				 Perl Programmers Reference Guide				   Locale::Currency(3perl)

NAME

       Locale::Currency - standard codes for currency identification

SYNOPSIS

	   use Locale::Currency;

	   $curr = code2currency('usd');     # $curr gets 'US Dollar'
	   $code = currency2code('Euro');    # $code gets 'eur'

	   @codes   = all_currency_codes();
	   @names   = all_currency_names();

DESCRIPTION

       The "Locale::Currency" module provides access to standard codes used for identifying currencies and funds, such as those defined in ISO
       4217.

       Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 4217
       three-letter codes will be used.

SUPPORTED CODE SETS

       There are several different code sets you can use for identifying currencies. The ones currently supported are:

       alpha
	   This is a set of three-letter (uppercase) codes from ISO 4217 such as EUR for Euro.

	   Two of the codes specified by the standard (XTS which is reserved for testing purposes and XXX which is for transactions where no
	   currency is involved) are omitted.

	   This code set is identified with the symbol "LOCALE_CURR_ALPHA".

	   This is the default code set.

       num This is the set of three-digit numeric codes from ISO 4217.

	   This code set is identified with the symbol "LOCALE_CURR_NUMERIC".

ROUTINES

       code2currency ( CODE [,CODESET] )
       currency2code ( NAME [,CODESET] )
       currency_code2code ( CODE ,CODESET ,CODESET2 )
       all_currency_codes ( [CODESET] )
       all_currency_names ( [CODESET] )
       Locale::Currency::rename_currency  ( CODE ,NEW_NAME [,CODESET] )
       Locale::Currency::add_currency  ( CODE ,NAME [,CODESET] )
       Locale::Currency::delete_currency  ( CODE [,CODESET] )
       Locale::Currency::add_currency_alias  ( NAME ,NEW_NAME )
       Locale::Currency::delete_currency_alias	( NAME )
       Locale::Currency::rename_currency_code  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Currency::add_currency_code_alias  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Currency::delete_currency_code_alias  ( CODE [,CODESET] )
	   These routines are all documented in the Locale::Codes man page.

SEE ALSO

       Locale::Codes
       Locale::Constants
       http://www.iso.org/iso/support/currency_codes_list-1.htm
	   The ISO 4217 data.

AUTHOR

       See Locale::Codes for full author history.

       Currently maintained by Sullivan Beck (sbeck@cpan.org).

COPYRIGHT

	  Copyright (c) 1997-2001 Canon Research Centre Europe (CRE).
	  Copyright (c) 2001	  Michael Hennecke
	  Copyright (c) 2001-2010 Neil Bowers
	  Copyright (c) 2010-2011 Sullivan Beck

       This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.14.2							    2011-09-26						   Locale::Currency(3perl)
All times are GMT -4. The time now is 08:27 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy