Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Replacing junk characters

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Replacing junk characters
# 8  
Old 07-12-2010
Quote:
Originally Posted by methyl
The problem with the "tr" approach is that there must be a matching entry on both sides of the translate.
..., it would I think need 31 '#" characters to match the ranges ...
Both sides of this translation have exactly one character:
Code:
tr '[\000-\011\013-\037]' '#'

The brackets form a range--a kind of wildcard that matches specifically itemized characters.
# 9  
Old 07-13-2010
Quote:
Originally Posted by KenJackson
Both sides of this translation have exactly one character:
Code:
tr '[\000-\011\013-\037]' '#'

The brackets form a range--a kind of wildcard that matches specifically itemized characters.
Your understanding of tr is flawed. The ranges are expanded, so \000-\011 has the same length as explicitly including each member, 10 characters. Also, the "[" and "]" in that command are not meta characters, they match literal bracket characters. tr ranges do not use opening and closing square brackets (unlike regular expressions and shell glob patterns).

For further info, see the EXTENDED DESCRIPTION section @ tr

Regards,
Alister
# 10  
Old 07-13-2010
Quote:
Originally Posted by methyl
The problem with the "tr" approach is that there must be a matching entry on both sides of the translate.

This should work for the two characters concerned.

Code:
cat junk1.txt | tr '\032\015' '##' > test1


With my unix "tr", it would I think need 31 '#" characters to match the ranges in KenJackson's post (untested). Hope this helps.
I tried the above one to replace the characters. It is working. Is it possible to identify the records which are changed and push them to a file.

This command will send all the records to new file. Instead only the records that were changed.test1 file will have all the changed records. Can I have some other file test2 which will have the original records .
# 11  
Old 07-13-2010
@KenJackson
Just out of interest, got yours working with two syntaxes. With my unix "tr" it definitely doesn't work with just one or two hashes:
Code:
cat filename | tr '[\000-\011][\013-\037]' '###############################'
cat filename | tr '\000-\011\013-\037' '###############################'

@alister
In standard unix "tr" the square brackets are indeed optional but they only affect the translation table if you specify too many as if you were using a regular expression [[ ]] .

In these nine examples, only the third one affects the [] in the input stream. Also the standard tables such as '[:lower:]' only work if you have the [].

Code:
echo "abcdef[]ghijk1234567890" | tr -d 'a-z'
[]1234567890

echo "abcdef[]ghijk1234567890" | tr -d '[a-z]'
[]1234567890

echo "abcdef[]ghijk1234567890" | tr -d '[[a-z]]'
1234567890
 
echo "abcdef[]ghijk1234567890" | tr ':lower:' ':upper:'
abcdef[]ghijk1234567890

echo "abcdef[]ghijk1234567890" | tr '[:lower:]' '[:upper:]'
ABCDEF[]GHIJK1234567890

echo "abcdef[]ghijk1234567890" | tr '[[:lower:]]' '[[:upper:]]'
ABCDEF[]GHIJK1234567890

echo "abcdef[]ghijk1234567890" | tr -d ':lower:'
abcdf[]ghijk1234567890

echo "abcdef[]ghijk1234567890" | tr -d '[:lower:]'
[]1234567890

echo "abcdef[]ghijk1234567890" | tr -d '[[:lower:]]'
[]1234567890

Last edited by methyl; 07-13-2010 at 08:25 AM.. Reason: Various typos, layout and highlighting.
This User Gave Thanks to methyl For This Post:
alister
# 12  
Old 07-13-2010
Quote:
Originally Posted by alister
Your understanding of tr is flawed. The ranges are expanded, so \000-\011 has the same length as explicitly including each member, 10 characters. Also, the "[" and "]" in that command are not meta characters, they match literal bracket characters. tr ranges do not use opening and closing square brackets (unlike regular expressions and shell glob patterns).
Hi Alister,

I don't claim to be a guru. But according to your explanation, this command should output abc, right?
I would be interested to hear an explanation of the actual output, #b#.
Code:
echo abc | tr '[123acxyz]' '#'



---------- Post updated at 07:31 AM ---------- Previous update was at 07:24 AM ----------

OK. Maybe we are working with different versions here. Here's mine.
Code:
tr --version
tr (GNU coreutils) 8.4
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Jim Meyering.

# 13  
Old 07-13-2010
@KenJackson
GNU is "Not Unix". There is a staggering variety of syntax and command behaviour across unix and unix-like Operating Systems. There are however some really cool command extensions in GNU and I think we just found another one!
# 14  
Old 07-13-2010
Quote:
Originally Posted by methyl
The problem with the "tr" approach is that there must be a matching entry on both sides of the translate.

This should work for the two characters concerned.

Code:
cat junk1.txt | tr '\032\015' '##' > test1


With my unix "tr", it would I think need 31 '#" characters to match the ranges in KenJackson's post (untested). Hope this helps.

This seems to be working for me as i told earlier. Is it possible to send the original version of the replaced records ( modified with # for \032) in a separate file.

The requirement is to send that file seperately. I need the original records which were replaced by tr command.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Need to remove Junk characters

Hi All, I have a issue that we are getting Junk characters from source and i am not able to load that records to Database. Line breakers Junk Characters (Â and different every time) Japanese Characters Every time I am using grep command and awk -F "\007" to find them and delete that... (1 Reply)
Discussion started by: spradeep86
1 Replies

2. Solaris

Junk characters in Solaris 11

Hi, I rebooted a Solaris 11 box and after that date stamp is coming in junk in almost all directories. root@tstilp05 # ls -l total 112 drwxrwxr-x 9 root sys 19 juin 1 03:10 adm drwxr-xr-x 6 root sys 6 sept. 19 2012 ai drwxr-xr-x 3 root bin ... (3 Replies)
Discussion started by: solaris_1977
3 Replies

3. UNIX for Dummies Questions & Answers

How to remove JUNK characters (FROM�)

Hi I have to remove the junk characters from my file. Please help.. File content : CURITY_CODE_GSD) FROM� DL_CB_SOD_EOD_VALUATION WHERE� ASOF (1 Reply)
Discussion started by: arukuku
1 Replies

4. Shell Programming and Scripting

Junk characters in mailx output

I have script which send a mail with top output. The script look like $ cat health.sh #!/bin/sh maillist="email address" rm /home/rtq1/file top -n 1 | head 15 > file cat file | mailx -s "Daily Health Report from `hostname` ..." "${maillist}" But now i am getting some junk characters along... (1 Reply)
Discussion started by: Renjesh
1 Replies

5. Shell Programming and Scripting

Handling Junk Characters

Urgently ur help is needed. Actually my req is i have an input file, that input file may have junk characters (^M, ^Z) etc... eg: cat file name abc^Z addres name2 msdmskd^Z address2 I want to validate the record and display where exactly this junk character resides. I want to... (3 Replies)
Discussion started by: help_scr_seeker
3 Replies

6. UNIX for Dummies Questions & Answers

how to grep junk characters in a file

hi guys, I am generating a file from datastage (an etl tool). Now the file is having some junk characters like ( Á,L´±,ñ and so on).. I want to use the grep function to figure out all the junk characters and their location. Can somebody help me out in finding it out.. if possible i... (1 Reply)
Discussion started by: mac4rfree
1 Replies

7. UNIX for Advanced & Expert Users

junk characters in the begining of every line

Hi Experts, here is a background to my problem : I am exporting data from teradata using fastexport utility, as varchar data. This pads additional two bytes (2 places as seen in notepad) in the resultset. I have found out other means of avoiding it but can't use varchar option in that... (5 Replies)
Discussion started by: sumoka
5 Replies

8. Shell Programming and Scripting

finding junk characters

Hi, Is there anyway to find the junk characters in a file.Consider the file has data as given below: 123|abc^M|Doctor^C #record 1 234|def|Med #record 2 345|dfg^C|Wrong^V #record 3 The junk characters are highlighted and this is a pipe delimited file. Is there anyway to... (20 Replies)
Discussion started by: ashwin3086
20 Replies

9. Solaris

junk characters in ls -l output in solaris10

Dear all, I have installed Solaris10 in a x86 machine.When the ls -l output is taken,at the Month's place some junk characters appear.Rest everything is fine. Cna somebody help..? thanks :b: (7 Replies)
Discussion started by: ragtechy
7 Replies

10. UNIX for Dummies Questions & Answers

Email ends with Junk Characters

I have written the following code ...to include the Subject, Message Body and Attachment with sendmail. When I send mail from my Unix account to diffrent mail servers like Yahoo , Hotmail etc..I recv the Message Body but there is no newline character at the end of each line.... Also I recv the... (5 Replies)
Discussion started by: Amruta Pitkar
5 Replies
Login or Register to Ask a Question