Identify records having junk characters in unix - Shell Programming and Scripting

sureshg_sampat · #1 11-14-2007

Hi Friends,

I need to have a command in Unix which output all teh records havingg junk characters in a file....

I know a command cat -tv <Filename> which opens the file and we can check for any junk character in it.

But my requirement is to fetch ONLY THOSE records having junk characters.
Please suggest

Thanks in advance,
Suresh.

matrixmadhan · #2 11-14-2007

What do you mean by junk characters ?

characters within specific ascii range ???

porter · #3 11-14-2007

Code:

#!/bin/sh

while read N
do
        if hasjunk "$N"
        then
             echo "$N"
        fi
done

sureshg_sampat · #4 11-14-2007

Hi ,

Junk characters means somethin like this when I did a cat on the unix file

|×©××××ª× ×¢×× ××¨××¦× ×××©×¨××ª ×××× ××× ××××¨×× ×- ×©× × ××©××××××× ××××¢× ××××¤× ×××§× ×× ××××¨ ××××©×××××××××× ××× × ××¡×¨××× ××©× × ××××××× ×©×× ×× × ××§××××. ×××§×©× ××¡××¨××××× ×©× × ×©×"×: 482304481-××©××× ×©× 3 ××××××ª ×©× ××¡×¨× ×¨×§ ×©× ××××× ×©× ××¡×¨ ×××¢× ×ª× ××××¨ ××××©××××§××× ×ª×× 6 ×××× ××× ×¢××¨× ××××©×××××¡××¨× ××¡××¤××ª. ××¢××¨×ª× ××ª××× ××ª ××ª ××¢×× ×××. ××××ª|

Thanks and Regards,
Suresh

matrixmadhan · #5 11-14-2007

© - ascii value - 169

this link should be useful to you,

Unicode/UTF-8-character table - starting from code position 0080

something like this should do it,

Code:

#! /opt/third-party/bin/perl

open(FILE, "<", $ARGV[0]) || die ("unable to open <$!>\n");

while( read(FILE, $data, 1) == 1 ) {
  $ordVal = ord($data);
  if( $ordVal == 169 ) {
    # similarly for other characters as well,
    # better option would be to build a range for that
    # do the processing here
  }
}

close(FILE);

exit 0

Lakris · #6 11-14-2007

Maybe the file command can help You?
Otherwise You must be more specific, You may be using character sets that come out strange in terminal but ok in any other application.

Example:
file *|grep text
in a random directory it would give me something like
ecl: ASCII text
gitt: Bourne-Again shell script text executable
HELP: ASCII English text
t2s: POSIX shell script text executable
time2Long.java: ASCII Java program text

(and lines sorted out could be lines like
Firefox_wallpaper.png: PNG image data, 1914 x 818, 8-bit/color RGB, non-interlaced
FW6AK115310.pdf: PDF document, version 1.3
itinerary-hotel-3S69Q2.RTF: Rich Text Format data, version 1, ANSI
)
Please be more specific if You can.

/Lakris

prowla · #7 11-14-2007

Use a language which has the isctrl() function; I think Perl does.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
how do I identify files with characters beyond a certain range.	kcsunsun01dev	Shell Programming and Scripting	3	02-14-2008 02:51 PM
UNIx junk emails	sireesha15	UNIX for Dummies Questions & Answers	6	12-14-2006 12:08 PM
Email ends with Junk Characters	Amruta Pitkar	UNIX for Dummies Questions & Answers	5	08-16-2006 05:43 PM
Identify a remote machine as windows or unix	u449064	UNIX for Advanced & Expert Users	5	08-05-2006 09:03 AM
how does unix identify C and other language code!	a25khan	UNIX for Dummies Questions & Answers	2	01-21-2004 07:44 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode