Sponsored Content
Top Forums Shell Programming and Scripting Removing these non-ASCII characters from a file Post 302828157 by Yoda on Monday 1st of July 2013 11:37:00 PM
Old 07-02-2013
Run octal dump and check what these characters are:
Code:
od -c < file

If these are control characters:
Code:
awk '{gsub(/[[:cntrl:]]/,X)}1' file > out

This User Gave Thanks to Yoda For This Post:
 

10 More Discussions You Might Find Interesting

1. HP-UX

Hex characters of ascii file

Hi, Whats the command or how do you display the hexadecimal characters of an ascii file. thanks Bud (2 Replies)
Discussion started by: budrito
2 Replies

2. Shell Programming and Scripting

Weird Ascii characters in file names

Hi. I have files in my OS that has weird file names with not-conventional ascii characters. I would like to run them but I can't refer them. I know the ascii # of the problematic characters. I can't change their name since it belongs to a 3rd party program... but I want to run it. is there... (2 Replies)
Discussion started by: yamsin789
2 Replies

3. Shell Programming and Scripting

Removing certain characters in a file

Hi I have a file that has semicolons in it (;) is there a way to just remove these in the file. Example name: Joe Smith; group: Group1; name: Mary White; group: Group2; (2 Replies)
Discussion started by: bombcan
2 Replies

4. Shell Programming and Scripting

convert ascii values into ascii characters

Hi gurus, I have a file in unix with ascii values. I need to convert all the ascii values in the file to ascii characters. File contains nearly 20000 records with ascii values. (10 Replies)
Discussion started by: sandeeppvk
10 Replies

5. Shell Programming and Scripting

Removing ^M characters from a file

Hi, I want to removing ^M characters from a file and combine the line with the next line. ex: issue i have: ABC^M^M DEF solution i need: ABCDEF I found that you by using the following command you can remove new line characters. tr -d '\r' < infile.csv > outfile.csv still... (10 Replies)
Discussion started by: mwrg
10 Replies

6. Shell Programming and Scripting

New line characters in Ascii file

I am having a file(1234.txt) downloaded from windows server (in Ascii format).However when i ftp this file to Unix server and try to work with it..i am unable to do anything.When i try to open the file using vi editor the file opens in the following format ... @ @ @ @ @ @ @ @... (4 Replies)
Discussion started by: appu2176
4 Replies

7. Shell Programming and Scripting

How to check if the file has EBCDIC or ascii characters

Hi, is there a way to check if the initial few characters are ebcdic or ascii in a file? (1 Reply)
Discussion started by: ahmedwaseem2000
1 Replies

8. Shell Programming and Scripting

Removing last and first characters in a file

bash-3.00$ cat temp.txt ./a/REA01/ces1/apps/ces_ces1_init3_aa.ear/ces.war/WEB-INF/classes/reds/common/environment.properties ./a/REA01/ces1/apps/ces_ces1_init3_aa.ear/commonproperties/hi/HostIntegration.properties... (9 Replies)
Discussion started by: bhas85
9 Replies

9. Shell Programming and Scripting

Identify extended ascii characters in a file

Hi, Is there a way to identify the lines in a file having extended ascii characters and display the same? For instance I have a file abc.txt having below data aaa|bbb|111|This is first line aaa|bbb|222|This is secõnd line aaa|bbb|333|This is third line aaa|bbb|444|This is foùrth line... (3 Replies)
Discussion started by: decci_7
3 Replies

10. Shell Programming and Scripting

Convert UTF-8 file to ASCII/ISO8859-1 OR replace characters

I am trying to develop a script which will work on a source UTF-8 file and perform one or more of the following It will accept the target encoding as an argument e.g. US-ASCII or ISO-8859-1, etc 1. It should replace all occurrences of characters outside target character set by " " (space) or... (3 Replies)
Discussion started by: hemkiran.s
3 Replies
TRIM(3) 								 1								   TRIM(3)

trim - Strip whitespace (or other characters) from the beginning and end of a string

SYNOPSIS
string trim (string $str, [string $character_mask = " 0r B"]) DESCRIPTION
This function returns a string with whitespace stripped from the beginning and end of $str. Without the second parameter, trim(3) will strip these characters: o " " (ASCII 32 ( 0x20)), an ordinary space. o " " (ASCII 9 ( 0x09)), a tab. o " " (ASCII 10 ( 0x0A)), a new line (line feed). o " " (ASCII 13 ( 0x0D)), a carriage return. o "" (ASCII 0 ( 0x00)), the NUL-byte. o "x0B" (ASCII 11 ( 0x0B)), a vertical tab. PARAMETERS
o $str - The string that will be trimmed. o $character_mask - Optionally, the stripped characters can also be specified using the $character_mask parameter. Simply list all characters that you want to be stripped. With .. you can specify a range of characters. RETURN VALUES
The trimmed string. EXAMPLES
Example #1 Usage example of trim(3) <?php $text = " These are a few words :) ... "; $binary = "x09Example stringx0A"; $hello = "Hello World"; var_dump($text, $binary, $hello); print " "; $trimmed = trim($text); var_dump($trimmed); $trimmed = trim($text, " ."); var_dump($trimmed); $trimmed = trim($hello, "Hdle"); var_dump($trimmed); $trimmed = trim($hello, 'HdWr'); var_dump($trimmed); // trim the ASCII control characters at the beginning and end of $binary // (from 0 to 31 inclusive) $clean = trim($binary, "x00..x1F"); var_dump($clean); ?> The above example will output: string(32) " These are a few words :) ... " string(16) " Example string " string(11) "Hello World" string(28) "These are a few words :) ..." string(24) "These are a few words :)" string(5) "o Wor" string(9) "ello Worl" string(14) "Example string" Example #2 Trimming array values with trim(3) <?php function trim_value(&$value) { $value = trim($value); } $fruit = array('apple','banana ', ' cranberry '); var_dump($fruit); array_walk($fruit, 'trim_value'); var_dump($fruit); ?> The above example will output: array(3) { [0]=> string(5) "apple" [1]=> string(7) "banana " [2]=> string(11) " cranberry " } array(3) { [0]=> string(5) "apple" [1]=> string(6) "banana" [2]=> string(9) "cranberry" } NOTES
Note Possible gotcha: removing middle characters Because trim(3) trims characters from the beginning and end of a string, it may be confusing when characters are (or are not) removed from the middle. trim('abc', 'bad') removes both 'a' and 'b' because it trims 'a' thus moving 'b' to the beginning to also be trimmed. So, this is why it "works" whereas trim('abc', 'b') seemingly does not. SEE ALSO
ltrim(3), rtrim(3), str_replace(3). PHP Documentation Group TRIM(3)
All times are GMT -4. The time now is 06:49 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy