Is the following script uncompressing .gz file into txt file
what does the below script doing, the first path with wild card clinical_event* has all .gz compressed files multiple and the final path has 1 single clinical_event.txt file
almost 350GB in size.
CLINICAL_EVENT* i have all .gz file 20 files and in the final path i only have 1 file .txt with almost 350GB file clinical_event.txt
is it uncompressing all .gz files almost 20 numbers from first path and consolidating into one big uncompressed .txt file with same name clinical_event.txt
Thanks a lot for the helpful info.
Moderator's Comments:
Please use CODE tags for full line and multi-line sample input, output, and code segments and ICODE tags for partial line sample input, output, and code segments.
Last edited by Don Cragun; 05-15-2016 at 04:51 PM..
Reason: Change ICODE tags to CODE tags and add ICODE tags.
here are the exact full paths:
doe the above script line picking all .gz files which are almost 20 numbers and consolidating into one uncompressed txt file to final path?
the only file i see is clinical_event.txt with size almost 350GB.
Moderator's Comments:
PLEASE do not use ICODE tags when displaying full-line and multi-line code segments; use CODE tags instead.
Last edited by Don Cragun; 05-15-2016 at 06:15 PM..
Reason: Change ICODE tags to CODE tags again.
here are the exact full paths:
doe the above script line picking all .gz files which are almost 20 numbers and consolidating into one uncompressed txt file to final path?
the only file i see is clinical_event.txt with size almost 350GB.
Moderator's Comments:
PLEASE do not use ICODE tags when displaying full-line and multi-line code segments; use CODE tags instead.
And, as requested before, what output do you get from the command:
hadoop is a command line utility with many subcommands.
the fs is one of those subcommands.
The subcommand fs accepts many options, one of those options is -text, which requires a source, one of the sources can be multiple gzip files and they get outputted into text format.
Another option of the fs subcomand is -put which requires two arguments, a local source and a destination.
Let's use that to know what's going on:
There are two instances of hadoop fs working together joined by a pipe (|), where the output of the first one, becomes the first argument of the second one.
In essence, it converts all those .gzip files (assuming that CLINICAL_EVENT*, refers to only gzip files, like you said) to text format and pushes it to standard output where the second picks it up, via standard input, (that's the - highlighted in red) and places it into /user/hive/warehouse/stage.db/Clinical_event/final/clinical_event.txt, which is a single file.
Legend: Blue: subcommand with option Red: stdin Purple: pipe
Hello Unix Shell Script Experts,
I have a script that would mask the columns in .csv file or .txt file.
First the script will untar the .zip files from Archive folder and processes into work folder and finally pushes the masked .csv files into Feed folder.
Two parameters are passed
... (5 Replies)
Hi
I have a few hundred files with extension .tar.Z. These files were archived (tar) and compressed (Z) on a UNIX system. I need to unzip them but not extract them. In other words they need to go to .tar extension. I would like to do this on my MAC or on a windows pc. I do not have a UNIX... (3 Replies)
so...
Lets assume I have a text file.
The text file contains multiple "#" symbols.
I want to replace all thos "#"s with a STRING using DOS/Batch
I want to add a certain TEXT to the end of each line.
How can I do this WITHOUT aid of sed, grep or anything linux related ? (1 Reply)
Hello,
I have Fedora 14 installed on my machine I have a .zip file ( some latex package) which I want to unzip to some location in the Latex paths /usr/share.../texmf/..
so I went to super user mode, created the directory for this package over there,
mkdir logo
and tried... (1 Reply)
Hi,
I have already read a lot of posts on sending attachments in unix...but none of them were of help for my problem...so here goes..
i wanna attach a text file and send to a mail id..used the following code :
uuencode "$File1" "$File1" ;|mail -s "$Mail_sub" abc@abc.com
it works... (2 Replies)
Hi,
I am looking for the unix script which can takes the 2 month old data from a TXT file (there is one txt file in whiche messages are appended on daily basis) and compress them into new file.Please halp me out. (2 Replies)
Hi,
how do u uncompress a .Z file?
i have a file called software.all.Z and have to change it to software.all. how to do that?
thanks!
karthik (3 Replies)
Hi Friends,
i am trying to uncompress a file which is of the following type
filename.tar.gz.
when i am using gunzip filename.tar.gz to unzip it, i am getting the error as
filename .tar.gz: invalid compressed data--format violated
what is the mistake that i am doing ?
any other way to... (2 Replies)
Hi!! All,
I have a compressed C file made on Linux. On Linux I used to uncompress it using the command:
zcat filename -xvf- |tar
I am not able to uncompress the same on Hp-UX-10.2. Could someone help me out how to do it ?? Do I have to use any other command to uncompress it?? (2 Replies)