junk characters in the begining of every line


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users junk characters in the begining of every line
# 1  
Old 02-03-2011
junk characters in the begining of every line

Hi Experts,

here is a background to my problem :

I am exporting data from teradata using fastexport utility, as varchar data.
This pads additional two bytes (2 places as seen in notepad) in the resultset.
I have found out other means of avoiding it but can't use varchar option in that solution.
(all this without using shell script)

solution:
i used cut -c 3- <filename> to get rid of it & it's working fine.

but i feel it's not very convincing solution. here is a sample data :
¢
ž
¦

£



¡

Please suggest solution.
# 2  
Old 02-03-2011
Why do you feel this is not a very convincing solution?
# 3  
Old 02-04-2011
:-)

good question.

I feel so because , I would like to generalize this piece of code.

To handle the unwanted characters except for [0-9],[a-z],[A-Z],@,.,-

is there any way to find out only junk characters ?
# 4  
Old 02-04-2011
Hi, the first two bytes appear to be bytes that have a meaning that is not representing ascii characters. However if you display them they are wrongfully interpreted as ascii characters. So I don't think you can use ascii values to determine which characters should or should not be printed. For example if the first character is a capital D you would then not remove it. But what if the line starts with DqHello, then these characters are clearly invalid although they are in the right character class. So I would tend to think that just removing the first two characters is the best option, no?
# 5  
Old 02-04-2011
Those two characters appear to be channel control or carriage return control, like FORTRAN, MVS, or VMS can use. They probably represent the length of the line.

scrutinizer is quite correct. You could get ostensibly valid characters appearing there that are clearly garbage.
# 6  
Old 02-07-2011
Ok.

Thanks a lot!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Need to remove Junk characters

Hi All, I have a issue that we are getting Junk characters from source and i am not able to load that records to Database. Line breakers Junk Characters (Â and different every time) Japanese Characters Every time I am using grep command and awk -F "\007" to find them and delete that... (1 Reply)
Discussion started by: spradeep86
1 Replies

2. Shell Programming and Scripting

How to rename all files at a time by appending some characters at the begining?

Hi I have a list a filename in a directory starting with particular pattern for example: abc_1234.txt abc_7565.txt abc_7676.txt abc_7765.txt i need to rename all these files by appending bck. or bck_ Expected output: bck.abc_1234.txt bck.abc_7565.txt bck.abc_7676.txt... (1 Reply)
Discussion started by: Little
1 Replies

3. Solaris

Junk characters in Solaris 11

Hi, I rebooted a Solaris 11 box and after that date stamp is coming in junk in almost all directories. root@tstilp05 # ls -l total 112 drwxrwxr-x 9 root sys 19 juin 1 03:10 adm drwxr-xr-x 6 root sys 6 sept. 19 2012 ai drwxr-xr-x 3 root bin ... (3 Replies)
Discussion started by: solaris_1977
3 Replies

4. UNIX for Dummies Questions & Answers

How to remove JUNK characters (FROM�)

Hi I have to remove the junk characters from my file. Please help.. File content : CURITY_CODE_GSD) FROM� DL_CB_SOD_EOD_VALUATION WHERE� ASOF (1 Reply)
Discussion started by: arukuku
1 Replies

5. Shell Programming and Scripting

Handling Junk Characters

Urgently ur help is needed. Actually my req is i have an input file, that input file may have junk characters (^M, ^Z) etc... eg: cat file name abc^Z addres name2 msdmskd^Z address2 I want to validate the record and display where exactly this junk character resides. I want to... (3 Replies)
Discussion started by: help_scr_seeker
3 Replies

6. UNIX for Dummies Questions & Answers

how to grep junk characters in a file

hi guys, I am generating a file from datastage (an etl tool). Now the file is having some junk characters like ( Á,L´±,ñ and so on).. I want to use the grep function to figure out all the junk characters and their location. Can somebody help me out in finding it out.. if possible i... (1 Reply)
Discussion started by: mac4rfree
1 Replies

7. Shell Programming and Scripting

Replacing junk characters

Hi, I have a file with data as given below $cat file1 123|abc|345 345|def|567 The first record is good record. The second record has an invisible junk character like \032. I was replace all the occurences of that invisible character with #. I want to do this for a set of... (16 Replies)
Discussion started by: ashwin3086
16 Replies

8. Shell Programming and Scripting

finding junk characters

Hi, Is there anyway to find the junk characters in a file.Consider the file has data as given below: 123|abc^M|Doctor^C #record 1 234|def|Med #record 2 345|dfg^C|Wrong^V #record 3 The junk characters are highlighted and this is a pipe delimited file. Is there anyway to... (20 Replies)
Discussion started by: ashwin3086
20 Replies

9. Shell Programming and Scripting

Remove junk characters using Perl

Guys, can you help me in removing the junk character "^S" from the below line using perl Reference Data Not Recognised ^S Where a value is provided by the consuming system, which is not reco Thanks, M.Mohan (1 Reply)
Discussion started by: mohan_xunil
1 Replies

10. UNIX for Dummies Questions & Answers

Email ends with Junk Characters

I have written the following code ...to include the Subject, Message Body and Attachment with sendmail. When I send mail from my Unix account to diffrent mail servers like Yahoo , Hotmail etc..I recv the Message Body but there is no newline character at the end of each line.... Also I recv the... (5 Replies)
Discussion started by: Amruta Pitkar
5 Replies
Login or Register to Ask a Question