Shell Programming and Scripting

Actually, I downloaded the report from a merchant's website and it usually downloads in the same format. Earlier, we used to extract the file on Windows, then place it on our FTP server location. From this location, my ETL tool picks up the file and loads it into our company Database. Since this process involves a lot of manual intervention, I am trying to automate the process as much as I can. For this reason, I was currently figuring out a process on how to extract the txt file present in the compressed file. The file uncompress quickly and easily using a winzip program... Still need help doing it in Linux, so I would at least avoid the extraction part... What I can then do is, directly download the file from the website and place it on my FTP location and the rest is taken care by the script...

Still, I do not know how you transferred the raw .gz file from Windows onto UNIX. If you transferred the file to UNIX with FTP, it's possible you transferred it in ASCII mode rather than binary mode. The feature is designed to translate native Windows \r\n newlines into native UNIX \n newlines. This is useful for uncompressed text but you can probably imagine it'd be disastrous in binary data, where \n does not mean a new line!

If you are uploading with FTP, issue the binary command before you put anything.

I FTPd using Winscp to transfer the file that I downloaded from the merchant's web portal into Linux location. winzip works and extracts the file very easily, but I am having trouble using unzip command too... Even this is giving me an error as following.

Code:

Archive:  D_R_38957222_20111410.txt.gz
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of D_R_38957222_20111410.txt.gz or
        D_R_38957222_20111410.txt.gz.zip, and cannot find D_R_38957222_20111410.txt.gz.ZIP, period.

Code:

file D_R_38957222_20111410.txt.gz

gives me output as

Code:

D_R_38957222_20111410.txt.gz: gzip compressed data, from FAT filesystem (MS-DOS, OS/2, NT)

Do not understand why gunzip command doesnt uncompress the data in the file when the file command says it is compressed using gzip

Again, I suspect the file was transferred in ASCII mode which corrupted it. This is a common pitfall with FTP. You would never notice this problem transferring raw text, but it will scramble anything else!

The file command only checks the first few bytes of the file to make a reasonable guess at what kind it is. Most binary files begin with a type code, header, or something. It wouldn't notice any corruption that happened later.

If the file were a regular .gzfile, then any one of your solutions might have worked I guess... this file might be corrupted like you said and since the file uncompresses easily using winzip, also I do not know how to check if the file is corrupted or not, I am stuck with the issue. I did not know how the filecommand worked, thank you for helping on the issue, I still haven't figured out any solutions to it though... Will definitely find an answer and post the answer here so someone else might benefit from it

For the third time, check whether your FTP client is uploading in ASCII mode -- this will scramble any non-text file!

You could also try an alternate method of uploading it, like SCP. All SCP transfers are binary, there is no ASCII mode to turn off.

@Corona688: I have checked under the FTP client's (WinSCP) settings and the transfer mode is set to Text(Plain text, html, scripts,....) I do not think it is uploading the files in ASCII mode. Also attached is the screenshot of the settings window