Sponsored Content
Top Forums Shell Programming and Scripting How to remove Unicode <feff> from top of file? Post 302739735 by biztank on Tuesday 4th of December 2012 10:27:08 PM
Old 12-04-2012
I tried it in many forms but its no help at all...

Code:
 cat xyz.csv | grep -v $(echo -ne '^\0376\0377')
 
or
 
 grep -v $(echo -ne '^\0376\0377') xyz.csv > fixed.txt

what was your thought process...

Thanks for the post though... Smilie
 

10 More Discussions You Might Find Interesting

1. Programming

How to display unicode characters / unicode string

I have a stream of characters like "\u8BBE\u5907\u7BA1" and i want to display it. I tried following things already without any luck. 1) printf("%s",L("\u8BBE\u5907\u7BA1")); 2) printf("%lc",0x8BBE); 3) setlocale followed by fwide followed by wprintf 4) also changed the local manually... (3 Replies)
Discussion started by: jackdorso
3 Replies

2. Shell Programming and Scripting

grep for a particular pattern and remove few lines above top and bottom of the patter

grep for a particular pattern and remove 5 lines above the pattern and 6 lines below the pattern root@server1 # cat filename Shell Programming and Scripting test1 Shell Programminsada asda dasd asd Shell Programming and Scripting Post New Thread Shell Programming and S sadsa ... (17 Replies)
Discussion started by: fed.linuxgossip
17 Replies

3. AIX

want to remove some line from top of file.

Hi All, I have AIX 5.3 server. I have one big file. in that i want to remove 5000 line from top. is there any command for this? Thanks, Vishal (6 Replies)
Discussion started by: vishalpatel03
6 Replies

4. UNIX for Dummies Questions & Answers

remove special and unicode characters

Hi, How do I remove the lines where special characters or Unicode characters appear? The following query does work but I wonder if there is a better way. cat test.txt | egrep -v '\)|#|,|&|-|\(|\\|\/|\.' The following lines show that my query is incomplete. Warning: The word "*Khan" is... (1 Reply)
Discussion started by: shantanuo
1 Replies

5. UNIX for Dummies Questions & Answers

Remove Unicode/special chars from XML

Hi, We are receiving an XML file in Unix which has some special characters between tags like '^' etc <Tag> 1e^O7f%<2304e.$d8f57e8^Bf-&e.^Zh7/327e^O7 </Tag> We need to remove all special characters like ^ ones and also any '&' or '<' or '>' being sent within the start and close tags i.e.... (6 Replies)
Discussion started by: dsrookie7
6 Replies

6. Shell Programming and Scripting

Unicode file validation

I don't want HTML_CONTENT,RICH_CONTENT,TEXT_CONTENT columns data in the file and reset of data we need to extract. Find the attached file. Need to extract date in between DI_UX_ROW_END tag. Can help me using unix command using AWK. Thanks, (2 Replies)
Discussion started by: bmk
2 Replies

7. Shell Programming and Scripting

Request for advise on how to remove control characters in a UNIX file extracted from top command

Hi, Please excuse for posting new thread on control characters, I am facing some difficulties in removing the control character from a file extracted from top command, i am able to see control characters using more command and in vi mode, through cat control characters are not visible ... (8 Replies)
Discussion started by: karthikram
8 Replies

8. Shell Programming and Scripting

Remove top 3 duplicates

hello , I have a requirement with input in below format abc 123 xyz bcd 365 kii abc 987 876 cdf 987 uii abc 456 yuu bcd 654 rrr Expecting Output abc 456 yuu bcd 654 rrr cdf 987 uii (1 Reply)
Discussion started by: Tomlight
1 Replies

9. Shell Programming and Scripting

Remove top and bottom for each column

Dear All I was wondering if someone could help me in resolving an issue. I have a file like this: column1 column2 2 4 3 5 8 9 0 12 0 0 0 0 9 0 87 0 1 0 1 0 1 0 4 0 (2 Replies)
Discussion started by: giuliangiuseppe
2 Replies

10. UNIX for Beginners Questions & Answers

Bash script - Remove the 3 top level of a full path filename

Hello. Source file are in : /a/b/c/d/e/f/g/some_file Destination is : /d/e where sub-directories "f" and "g" may missing or not. After copying I want /a/b/c/d/e/f/g/file1 in /d/e/f/g/file1 On source /a is top-level directory On destination /d is top-level directory I would like... (2 Replies)
Discussion started by: jcdole
2 Replies
PPI::Token::BOM(3)					User Contributed Perl Documentation					PPI::Token::BOM(3)

NAME
PPI::Token::BOM - Tokens representing Unicode byte order marks INHERITANCE
PPI::Token::BOM isa PPI::Token isa PPI::Element DESCRIPTION
This is a special token in that it can only occur at the beginning of documents. If a BOM byte mark occurs elsewhere in a file, it should be treated as PPI::Token::Whitespace. We recognize the byte order marks identified at this URL: <http://www.unicode.org/faq/utf_bom.html#BOM> UTF-32, big-endian 00 00 FE FF UTF-32, little-endian FF FE 00 00 UTF-16, big-endian FE FF UTF-16, little-endian FF FE UTF-8 EF BB BF Note that as of this writing, PPI only has support for UTF-8 (namely, in POD and strings) and no support for UTF-16 or UTF-32. We support the BOMs of the latter two for completeness only. The BOM is considered non-significant, like white space. METHODS
There are no additional methods beyond those provided by the parent PPI::Token and PPI::Element classes. SUPPORT
See the support section in the main module AUTHOR
Chris Dolan <cdolan@cpan.org> COPYRIGHT
Copyright 2001 - 2011 Adam Kennedy. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. The full text of the license can be found in the LICENSE file included with this module. perl v5.18.2 2011-02-25 PPI::Token::BOM(3)
All times are GMT -4. The time now is 07:23 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy