Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Problems with Dutch and converting files to UNIX Post 302893954 by RudiC on Saturday 22nd of March 2014 11:34:54 AM
Old 03-22-2014
That file1.txt has a three byte UTF-8 representation of FEFF as an intro; the rest is normal UTF-8 chars, even the é (=0xE9). The normal UTF-16 intro would be a two byte FFFE. So maybe that file has undergone another uncontrolled conversion before, e.g. a little edian - big endian one?
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

converting files from unix to windows

Need Help?? We receive Files From GM Motors and they written on a Sun Workstation using the Tar Command on a 4mm Dat Tape. We have an HP sure Store 24 Tape drive that will Execpt but when i do that it says that the media is bad. was wondering if there was any software that would read it in its... (2 Replies)
Discussion started by: jefft1976
2 Replies

2. UNIX Desktop Questions & Answers

Converting BMP to BM (or other unix format)

Hey pllz, ive got a little problem, i want to convert a bmp of gif or jpg to an unix format (bm) anybody got any suggestions ? greets\EJ (1 Reply)
Discussion started by: EJ =)
1 Replies

3. OS X (Apple)

Converting Unix executable files

I loaded OS X Panther on my Mac G4 and found that many files previously saved as Word or Word Perfect files were inadventently converted to Unix executable files. When I try to read these in Word, it cannot recognize or translate the file properly. Does anyone know how to translate these files? Is... (4 Replies)
Discussion started by: Steven Greenber
4 Replies

4. UNIX for Dummies Questions & Answers

Converting Unix text to windows

I am trying to FTP a text file from a machine running LynxOS and I am having problems with the way windows "sees" the characters. For example this is how windows presents the text:     DevProcRcpClass The boxes are what I am having problems with. When viewing the same file on a... (3 Replies)
Discussion started by: mchristisen
3 Replies

5. Windows & DOS: Issues & Discussions

Converting UNIX scripts to DOS

Is there a tool available to convert UNIX (BASH Shell) scripts to DOS scripts? I understand that DOS scripting is far inferior to unix scripting, and therfore this conversion may not be possible. Alternativley, perhaps I could convert my Unix scripts to C... then compile it for a windows... (2 Replies)
Discussion started by: Crozz
2 Replies

6. Shell Programming and Scripting

Converting DOS filetype to UNIX

Hello folks I am working on a project that requires me to write a script that operates on a bunch of text files. When I try less file.txt I see a bunch of ^M's everywhere. Some Googling tells me that this is because the files have a DOS fileformat and found the following fixes: sed 's/^M$//'... (5 Replies)
Discussion started by: ksk
5 Replies

7. Shell Programming and Scripting

awk - problems by converting date-format

Hi i try to change the date-format from DD/MM/YYYY into MM/DD/YY. Input-Data: ... 31/12/2013,23:40,198.00,6.20,2,2,2,1,11580.0,222 31/12/2013,23:50,209.00,7.30,2,2,3,0,4380.0 01/01/2014,00:00,205.90,8.30,2,2,3,1,9360.0,223 ... Output-Data should be: ...... (7 Replies)
Discussion started by: IMPe
7 Replies

8. Shell Programming and Scripting

Converting Multiline Files to Flat Files?

How to convert this: F1-R1 F1-R2 F1-R3 into a flat file for bash?? Each record F2-R1 F2-R2 F2-R3 F3-R1 F3-R2 F3-R3 F4-R1 F4-R2 F4-R3is on one line with all fields for that record, put into an output file. The output file should look like this when converted: F1-R1,F2-R1,F3-R1,F4-R1... (6 Replies)
Discussion started by: bud1738
6 Replies

9. Homework & Coursework Questions

Converting .dat to UNIX

I uploaded a .dat file from sftp to my server and after using dos2unix to convert the file and check my work it says that the file was not transferred correctly and that the content is garbled. Please help (3 Replies)
Discussion started by: Ovid158
3 Replies
PPI::Token::BOM(3)					User Contributed Perl Documentation					PPI::Token::BOM(3)

NAME
PPI::Token::BOM - Tokens representing Unicode byte order marks INHERITANCE
PPI::Token::BOM isa PPI::Token isa PPI::Element DESCRIPTION
This is a special token in that it can only occur at the beginning of documents. If a BOM byte mark occurs elsewhere in a file, it should be treated as PPI::Token::Whitespace. We recognize the byte order marks identified at this URL: <http://www.unicode.org/faq/utf_bom.html#BOM> UTF-32, big-endian 00 00 FE FF UTF-32, little-endian FF FE 00 00 UTF-16, big-endian FE FF UTF-16, little-endian FF FE UTF-8 EF BB BF Note that as of this writing, PPI only has support for UTF-8 (namely, in POD and strings) and no support for UTF-16 or UTF-32. We support the BOMs of the latter two for completeness only. The BOM is considered non-significant, like white space. METHODS
There are no additional methods beyond those provided by the parent PPI::Token and PPI::Element classes. SUPPORT
See the support section in the main module AUTHOR
Chris Dolan <cdolan@cpan.org> COPYRIGHT
Copyright 2001 - 2011 Adam Kennedy. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. The full text of the license can be found in the LICENSE file included with this module. perl v5.16.3 2011-02-26 PPI::Token::BOM(3)
All times are GMT -4. The time now is 02:48 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy