Visit Our UNIX and Linux User Community

Full Discussion: Replace Junk chars (Sed)
Top Forums Shell Programming and Scripting Replace Junk chars (Sed) Post 302111295 by alfredo123 on Tuesday 20th of March 2007 03:52:28 PM
Old 03-20-2007
Replace Junk chars (Sed)

I know this has been asked previously on this forum...But I think I have a different scenario to present.

I ahve a file tht looks like this (note:there are control Z and other chars tht are not visible on line with anme bowers)

Code:
BB7118450      	6004718		BIANCALANA =HEI         
BZ5842819      	1493750		ZACHAR=/                
AB1250620      	0300080		BOWERS               
MR0408256      	0458355		DYCUS/BOWERS/PA         
AB9381233      	6811741		BOWERS *6*              
BL6039588      	0184195		LIMERES JR. |           
AD1901253      	0458720		DURAZO - CELAYA         
BE4340698      	0609541		EL-HAJJAOUI(LUN         
AH5528267      	6786292		HARSANY, JR ++          
BD0762662      	4527818		D`SOUZA                 
AC6629577      	3059914		CANOVA`                 
AC9130991      	0485282		CARROLL 16!            
AT8874679      	0490912		TAIT 20!               
AH8136459      	0136813		HUNTER 06! (16

This file has special chars and Control M/Z chars....I need to remove all these junk chars and replace with a space. I tried the following commands..

Code:
sed 's/[!@#\$%^&*()<>+|={}\/`-]//g' special.txt

The above worked except for the fact that the Control Z chars were not replaced.

I tried
Code:
tr -d '\032' < special.txt | sed 's/[!@#\$%^&*()<>+|={}\/`\"\^M\'026'-]//g' > t

but some additional chars were deleted...

Any help in getting the command right would be appreciated...
 
Test Your Knowledge in Computers #322
Difficulty: Easy
HTML stands for Hypertext Markdown Language.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

replace ascii chars without loosing it.

Hi, Can some one tell, how to replace ascii non printable TAB from the while to something, then later on replace it back to TAB. Basciallz we do bulk data processing, our processin treats TAB as new field , So I thought we can replace it with something and later on revert it. TIA (4 Replies)
Discussion started by: braindrain
4 Replies

2. Shell Programming and Scripting

Clense Junk Data File - Using Shell or awk or sed

Hello Shell Gurus i need help in solving this puzzle. We have a junk data file that needs to be fed into the database. Need to clense the data file thru shell script. I am not a expert and so need help with Here is what i need to do on the input file -Step -1 Replace all pipes |' within... (1 Reply)
Discussion started by: rimss
1 Replies

3. Shell Programming and Scripting

replace chars,

:rolleyes: Hi, I want to replace the particular text in the middle of the line. Ex. In the line 40-50 wanted to replace the char as 'x' (7 Replies)
Discussion started by: Jairaj
7 Replies

4. UNIX for Dummies Questions & Answers

Using SED to get n chars after given value from file

Hello, my name is Marc, I'm a linux starter :) and I hope you specialists can help me solving this issue. I have a file containing a lot of data. Somewhere in this file, there's a string called "Faultdump", directly followed by 64 chars of HEX data. I need to get the HEX part. I accomplished... (12 Replies)
Discussion started by: Kally
12 Replies

5. Shell Programming and Scripting

is is possible remove junk chars from the strings?

Hello friendz, dfl;g435hkd.fg ..this is what I am getting. I want to print strings without junk chars. I want to exactly like this... dflg435hkd.fg ...need some specific operators also. for example, dot or comma should allow. plz help me out ya! ~Balan:confused: (1 Reply)
Discussion started by: balan_mca
1 Replies

6. Shell Programming and Scripting

sed discard chars after last _

Hi, I'd get fields like unix_linux_form_yyyyddmmhhmi.file.txt shell_programming_and_scripting.txt so on... and want them as below unix_linux_form shell_programming_and I could remove everything after a '.' as below echo $field | sed 's/\..*//' but how to remove... (14 Replies)
Discussion started by: dips_ag
14 Replies

7. Shell Programming and Scripting

How to erase junk Chars coming in only the first line

I have a file containing few thousands of lines. when I do cat on it , i find it having two special Chars at the start of first line alone as shown down here. HDR|20111024|01 If i delete this line and do a cat on file , the current first line is shown to have the same special Chars. ... (3 Replies)
Discussion started by: subramanian2008
3 Replies

8. Shell Programming and Scripting

Replace char between chars - help needed

Hello, I have a csv file with "^" as text delimiters and "|" as field delimiters. It's converted from a xls file. One record looks like this: ^Tablete Internet^|Archos|501838|^Tableta Internet ARCHOS 80 G9 ...| ... (more lines) ... "501|838"^|330.00|USD|sl|12|0|Link|^router wireless 150... (10 Replies)
Discussion started by: go0ogl3
10 Replies

9. UNIX for Dummies Questions & Answers

Grep or sed to search, replace/insert chars!

HI All Im trying to come up with an approach to finding a string, using a portion of that string to insert it on lines starting with the value "GOTO" appending to end of line after removing PT's ( See example below! ) EXAMPLE: 1. I would like to search for the line that starts with "TLAXIS/"... (7 Replies)
Discussion started by: graymj
7 Replies

10. UNIX for Dummies Questions & Answers

How to replace and remove few junk characters from a specific field?

I would like to remove all characters starting with "%" and ending with ")" in the 4th field - please help!! 1412007819.864 /device/services/heartbeatxx 204 0.547%!i(int=0) 0.434 0.112 1412007819.866 /device/services/heartbeatxx 204 0.547%!i(int=1) 0.423 0.123... (10 Replies)
Discussion started by: snemuk14
10 Replies
uniq(1) 							   User Commands							   uniq(1)

NAME
uniq - report or filter out repeated lines in a file SYNOPSIS
uniq [-c | -d | -u] [-f fields] [-s char] [ input_file [output_file]] uniq [-c | -d | -u] [-n] [ + m] [ input_file [output_file]] DESCRIPTION
The uniq utility will read an input file comparing adjacent lines, and write one copy of each input line on the output. The second and suc- ceeding copies of repeated adjacent input lines will not be written. Repeated lines in the input will not be detected if they are not adjacent. OPTIONS
The following options are supported: -c Precedes each output line with a count of the number of times the line occurred in the input. -d Suppresses the writing of lines that are not repeated in the input. -f fields Ignores the first fields fields on each input line when doing comparisons, where fields is a positive decimal integer. A field is the maximal string matched by the basic regular expression: [[:blank:]]*[^[:blank:]]* If fields specifies more fields than appear on an input line, a null string will be used for comparison. -s chars Ignores the first chars characters when doing comparisons, where chars is a positive decimal integer. If specified in con- junction with the -f option, the first chars characters after the first fields fields will be ignored. If chars specifies more characters than remain on an input line, a null string will be used for comparison. -u Suppresses the writing of lines that are repeated in the input. -n Equivalent to -f fields with fields set to n. +m Equivalent to -s chars with chars set to m. OPERANDS
The following operands are supported: input_file A path name of the input file. If input_file is not specified, or if the input_file is -, the standard input will be used. output_file A path name of the output file. If output_file is not specified, the standard output will be used. The results are unspeci- fied if the file named by output_file is the file named by input_file. EXAMPLES
Example 1: Using the uniq command The following example lists the contents of the uniq.test file and outputs a copy of the repeated lines. example% cat uniq.test This is a test. This is a test. TEST. Computer. TEST. TEST. Software. example% uniq -d uniq.test This is a test. TEST. example% The next example outputs just those lines that are not repeated in the uniq.test file. example% uniq -u uniq.test TEST. Computer. Software. example% The last example outputs a report with each line preceded by a count of the number of times each line occurred in the file: example% uniq -c uniq.test 2 This is a test. 1 TEST. 1 Computer. 2 TEST. 1 Software. example% ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of uniq: LANG, LC_ALL, LC_CTYPE, LC_MES- SAGES, and NLSPATH. EXIT STATUS
The following exit values are returned: 0 Successful completion. >0 An error occurred. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWesu | +-----------------------------+-----------------------------+ |CSI |Enabled | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ SEE ALSO
comm(1), pack(1), pcat(1), sort(1), uncompress(1), attributes(5), environ(5), standards(5) SunOS 5.10 20 Dec 1996 uniq(1)

Featured Tech Videos

All times are GMT -4. The time now is 09:27 AM.
Unix & Linux Forums Content Copyright 1993-2020. All Rights Reserved.
Privacy Policy