I am not too sure about MIME::Base64 as I have not used it before. However, base64 itself is encoding-agnostic, that is, it encodes/decodes without regard to whatever encoding the original message is, because it is not only used to encode textual data, but also images, zip files or just about any binary data you can imagine, that do not have the notion of an "encoding" at all. So what Base64 sees and acts on, is just a bytestream. it doesn't really care what is inside.
So, for a text message:
In other words, you still need to manually handle the decoding to have Perl decode it as UTF-8 properly. By default, Perl treats everything as ASCII, so that may explain why you get the output wrong.
Perl has specific quirks with respect to Unicode. That really much depends on the version of Perl you are using. I have had a rather thorough investigation of Perl Unicode support in 5.8 branch, but not sure if any changes have been implemented in 5.10. If you have Perl 5.6 or earlier, chances are the Perl Unicode support is not adequate to ensure Unicode-safety.
I am unable to explain so much with so little space here. I recommend you start with the perluniintro manpage for further information:
While working with russian text under FreeBSD&MySQL I need to convert a string from MySQL to the Unicode format.
I've just started my way in C++ under FreeBSD , so please explain me how can I get ascii code of Char variable and also how can i get a character into variable with the specified ascii... (3 Replies)
Hello,
I am trying to convert a 7bit ASCII file to UTF-8.
I have used iconv before though it can't recognize it for some reason and says unknown file encoding.
When I used ascii2uni package with different package, ./ascii2uni -a K -a I -a J -a X test_file > new_test_file
It still... (2 Replies)
I have a shell script running to load some data from a text file to database. Text file contains some non-ASCII characters like ü. How can i convert these characters to UTF-8 codes before loading to DB. (5 Replies)
can someone help me in converting hex streams to decimal values using perl script
Hex value:
$my_hex_stream="0c07ac14001676";
Every hex value in the above stream should be converted in to decimal and separated by comma.
The output should be: 12,07,172,20,00,22,118 (2 Replies)
Sometimes we recieve some excel files containing French/Japanese characters over the mail, and these files are manually transferred to the server by using SFTP (security is not a huge concern here). The data is changed to text format before transferring it using Notepad.
Problem is: When saving... (4 Replies)
Hi,
I have tried to convert a UTF-8 file to windows UTF-16 format file as below from unix machine
unix2dos < testing.txt | iconv -f UTF-8 -t UTF-16 > out.txt
and i am getting some chinese characters as below which l opened the converted file on windows machine.
LANG=en_US.UTF-8... (3 Replies)
Hello all
i have utf-8 file that i try to convert to WINDOWS-1251 on linux
without any success
the file name is utf-8 when i try to do :
file -bi test.txt
it gives me :
text/plain; charset=utf-8
when i try to convert the file i do :
/usr/bin/iconv -f UTF-8 -t WINDOWS-1251 test.txt >... (1 Reply)
Hi All,
I am trying to obtain count of characters using awk, but "length" function returns a value of 1 for 2-byte or 3-byte characters as well unlike wc -c command.
I have tried to use the below commands within awk function, but it does not seem to work
{
cmd="wc -c "stringtocheck
( cmd )... (6 Replies)
I am trying to develop a script which will work on a source UTF-8 file and perform one or more of the following
It will accept the target encoding as an argument e.g. US-ASCII or ISO-8859-1, etc
1. It should replace all occurrences of characters outside target character set by " " (space) or... (3 Replies)
Dears,
I have a shell script - working perfectly on Oracle Linux - that detects the encoding (the charset to be exact) of the files in a specified directory using the "file" command (The file command outputs the charset in Linux, but doesn't do that in AIX), then if the file isn't a UTF-8 text... (4 Replies)
Discussion started by: JeanM-1
4 Replies
LEARN ABOUT MOJAVE
base64
base64(n) Text encoding & decoding binary data base64(n)
__________________________________________________________________________________________________________________________________________________NAME
base64 - base64-encode/decode binary data
SYNOPSIS
package require Tcl 8
package require base64 ?2.4.2?
::base64::encode ?-maxlen maxlen? ?-wrapchar wrapchar? string
::base64::decode string
_________________________________________________________________DESCRIPTION
This package provides procedures to encode binary data into base64 and back.
::base64::encode ?-maxlen maxlen? ?-wrapchar wrapchar? string
Base64 encodes the given binary string and returns the encoded result. Inserts the character wrapchar every maxlen characters of
output. wrapchar defaults to newline. maxlen defaults to 76.
Note well: If your string is not simple ascii you should fix the string encoding before doing base64 encoding. See the examples.
The command will throw an error for negative values of maxlen, or if maxlen is not an integer number.
::base64::decode string
Base64 decodes the given string and returns the binary data. The decoder ignores whitespace in the string.
EXAMPLES
% base64::encode "Hello, world"
SGVsbG8sIHdvcmxk
% base64::encode [string repeat xyz 20]
eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6
eHl6eHl6eHl6
% base64::encode -wrapchar "" [string repeat xyz 20]
eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6eHl6
# NOTE: base64 encodes BINARY strings.
% set chemical [encoding convertto utf-8 "Cu2088Hu2081u2080Nu2084Ou2082"]
% set encoded [base64::encode $chemical]
Q+KCiEjigoHigoBO4oKET+KCgg==
% set caffeine [encoding convertfrom utf-8 [base64::decode $encoded]]
BUGS, IDEAS, FEEDBACK
This document, and the package it describes, will undoubtedly contain bugs and other problems. Please report such in the category base64
of the Tcllib SF Trackers [http://sourceforge.net/tracker/?group_id=12883]. Please also report any ideas for enhancements you may have for
either package and/or documentation.
KEYWORDS
base64, encoding
CATEGORY
Text processing
COPYRIGHT
Copyright (c) 2000, Eric Melski
Copyright (c) 2001, Miguel Sofer
base64 2.4.2 base64(n)