finding junk characters


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting finding junk characters
# 8  
Old 06-25-2010
[/COLOR]ya i have tried that.. still it is not working..
The ascii value is \032 . It will be seen as ^Z character in VI.
# 9  
Old 06-25-2010
Quote:
Originally Posted by bartus11

Other way is:
Code:
awk '!/[[:cntrl:]]/' file

But it might not work with all AWK implementations.
Ashiwn..did you try the second method suggested by bartus11? it should be working.

Guru.
# 10  
Old 06-25-2010
Hey.. I tried that..
This is the result.It is printing the record with the junk characters as well.

$ awk '/[^a-zA-Z|#$%]/' junk1.txt
15|Susan|123
john|zint^Z|123
# 11  
Old 06-25-2010
Quote:
Originally Posted by ashwin3086
Hey.. I tried that..
This is the result.It is printing the record with the junk characters as well.

$ awk '/[^a-zA-Z|#$%]/' junk1.txt
15|Susan|123
john|zint^Z|123
^Z is not printable character, so it can't be output by AWK. Maybe there are just two normal characters: "^" and "Z"? Try
Code:
cat junk1.txt

and see if "^Z" apears there. If it does, then it is not CTRL+Z, just regular "^" and "Z". Also you can post output of
Code:
xxd junk1.txt

so we can see exactly how does your test data look like.
# 12  
Old 06-25-2010
Quote:
Originally Posted by ashwin3086
Hi,

@methyl : It is not working.
The sample data is this:
15|Jsus
15|Susan

The first record has a special character in it which is actually a junk character ( ascii value is \032) which is invisible.

The above command doesnt catch that as a bad record.

Odd. It should match [:cntrl:].
Code:
$ printf '\032' | awk '/[[:cntrl:]]/ {print "MATCHED SUCCESSFULLY"}'
MATCHED SUCCESSFULLY

Regards,
Alister

---------- Post updated at 10:05 AM ---------- Previous update was at 10:01 AM ----------

A posix-compliant sed alternative which only needs to read the file once:
Code:
sed -n '/^[^[:cntrl:]]*$/{p;d;}; w junk' data > nojunk

# 13  
Old 06-25-2010
Hi,

This is the output i get if i try xxd and "od -c ". If u see there is a special character after ziet ( 032) in the first record. Second record is good one.

$ od -c junk1.txt
0000000 2 0 6 1 | j o h n | z i e t 032 |
0000020 7 \n 1 2 3 4 | a s b c | b c f g
0000040 | 1 0 \n
0000044

$ xxd junk1.txt
0000000: 3230 3631 7c6a 6f68 6e7c 7a69 6574 1a7c 2061|john|ziet.|
0000010: 370a 3132 3334 7c61 7362 637c 6263 6667 7.1234|asbc|bcfg
0000020: 7c31 300a |10.
# 14  
Old 06-25-2010
Quote:
Originally Posted by ashwin3086
Hey.. I tried that..
This is the result.It is printing the record with the junk characters as well.

$ awk '/[^a-zA-Z|#$%]/' junk1.txt
15|Susan|123
john|zint^Z|123
You rewrote the code wrong... it is not
Code:
awk '/[^a-zA-Z|#$%]/' junk1.txt

but
Code:
awk '!/[^a-zA-Z|#$%]/' junk1.txt

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Need to remove Junk characters

Hi All, I have a issue that we are getting Junk characters from source and i am not able to load that records to Database. Line breakers Junk Characters (Â and different every time) Japanese Characters Every time I am using grep command and awk -F "\007" to find them and delete that... (1 Reply)
Discussion started by: spradeep86
1 Replies

2. Solaris

Junk characters in Solaris 11

Hi, I rebooted a Solaris 11 box and after that date stamp is coming in junk in almost all directories. root@tstilp05 # ls -l total 112 drwxrwxr-x 9 root sys 19 juin 1 03:10 adm drwxr-xr-x 6 root sys 6 sept. 19 2012 ai drwxr-xr-x 3 root bin ... (3 Replies)
Discussion started by: solaris_1977
3 Replies

3. UNIX for Dummies Questions & Answers

How to remove JUNK characters (FROM�)

Hi I have to remove the junk characters from my file. Please help.. File content : CURITY_CODE_GSD) FROM� DL_CB_SOD_EOD_VALUATION WHERE� ASOF (1 Reply)
Discussion started by: arukuku
1 Replies

4. Shell Programming and Scripting

Junk characters in mailx output

I have script which send a mail with top output. The script look like $ cat health.sh #!/bin/sh maillist="email address" rm /home/rtq1/file top -n 1 | head 15 > file cat file | mailx -s "Daily Health Report from `hostname` ..." "${maillist}" But now i am getting some junk characters along... (1 Reply)
Discussion started by: Renjesh
1 Replies

5. Shell Programming and Scripting

Handling Junk Characters

Urgently ur help is needed. Actually my req is i have an input file, that input file may have junk characters (^M, ^Z) etc... eg: cat file name abc^Z addres name2 msdmskd^Z address2 I want to validate the record and display where exactly this junk character resides. I want to... (3 Replies)
Discussion started by: help_scr_seeker
3 Replies

6. UNIX for Dummies Questions & Answers

how to grep junk characters in a file

hi guys, I am generating a file from datastage (an etl tool). Now the file is having some junk characters like ( Á,L´±,ñ and so on).. I want to use the grep function to figure out all the junk characters and their location. Can somebody help me out in finding it out.. if possible i... (1 Reply)
Discussion started by: mac4rfree
1 Replies

7. Shell Programming and Scripting

Replacing junk characters

Hi, I have a file with data as given below $cat file1 123|abc|345 345|def|567 The first record is good record. The second record has an invisible junk character like \032. I was replace all the occurences of that invisible character with #. I want to do this for a set of... (16 Replies)
Discussion started by: ashwin3086
16 Replies

8. Shell Programming and Scripting

Remove junk characters using Perl

Guys, can you help me in removing the junk character "^S" from the below line using perl Reference Data Not Recognised ^S Where a value is provided by the consuming system, which is not reco Thanks, M.Mohan (1 Reply)
Discussion started by: mohan_xunil
1 Replies

9. Solaris

junk characters in ls -l output in solaris10

Dear all, I have installed Solaris10 in a x86 machine.When the ls -l output is taken,at the Month's place some junk characters appear.Rest everything is fine. Cna somebody help..? thanks :b: (7 Replies)
Discussion started by: ragtechy
7 Replies

10. UNIX for Dummies Questions & Answers

Email ends with Junk Characters

I have written the following code ...to include the Subject, Message Body and Attachment with sendmail. When I send mail from my Unix account to diffrent mail servers like Yahoo , Hotmail etc..I recv the Message Body but there is no newline character at the end of each line.... Also I recv the... (5 Replies)
Discussion started by: Amruta Pitkar
5 Replies
Login or Register to Ask a Question