Shell Programming and Scripting

Hello,

Im downloading the file from windows server through FTP, the downloaded file is containing some junk character at very start of the file as below and causing my whole script is to fail,
how to download without junk or how to remove these before processing it?

Code:

▒▒"nmdbfnmdsfsdf"
       nmdsbnmfmdsf
        nmsbmfdsbnmfbds
nmbdsnmfbsnmdbfnmds
        sdmnbfnmbdsf

...
...

Im using the below code to download the file

Code:

get_file ()
{
ftp -vn 10.430.21.10 21 <<EOF
user "138" "abcd@9197"
pass
bi
cd /PVR
get testfile
bye
EOF
}

You need to use as instead of bi (ASCII , not BINARY transfer)

Also, if you still get those characters, they are probably BOM characters inserted by Windows.
See if this thread helps ..


--
BTW probably not, but if those are genuine passwords then you need to anonymize them before posting here..

Tried, the as option but no changes,still the these character causing these issue in my scripts.

I dont know what is this....

Code:

▒▒

actually my awk is hunting one string from the first line, but these junk characters is not allowing awk to proceed...

What is your LANG setting?? What LANG setting does the remote computer use? awk is barfing on locale (LANG encoding)
-- because it thinks that leading character means something.

You can see the hex for the problem character: (Linux example)

Code:

od -x -N 10 filename

-- The very first hex entry.

If the character were a BOM for 16bit encoding and it was wrong the whole file would be garbage. So, I think Scrutinizer is correct.

Looks like two times the UTF-8 three byte representation of 0x2592 - a faint memory alludes that these could be MAC control chars.
Why don't you delete them awk 'NR==1 {$0=substr($0,7)}1' file?

Dont know about the remote server as i do not have full access but i have my loacal AIX box lang setting as below

Code:

od -x -N 10 filename
0000000  fffe 2200 4500 7600 6500
0000012

As suggested by rudie tried to remove these bytes as below.

Code:

tail -n +4 file

It got removed successfully but seems that whole file is having this garbage value but not visible in the new truncated file, really in a puzzling situation now , that was the final touch everything was written on by creating dummy file/text , now once i acted on the original downloaded file it very much disappointed me.


This is the actual code which i was trying to search the pattern , but giving me the null value, on my dummy text it was really working fine.

Code:

awk '/'severe.job'/ {print;for(i=1; i<=4; i++) {getline; print}}' filename

Hey - this is a totally different story! That snippet is from a utf-16 coded file. Use iconv -futf-16 -tutf-8 file.