Sponsored Content
Top Forums Shell Programming and Scripting Is the following script uncompressing .gz file into txt file Post 302973170 by cplusplus1 on Sunday 15th of May 2016 03:25:21 PM
Old 05-15-2016
Is the following script uncompressing .gz file into txt file

what does the below script doing, the first path with wild card clinical_event* has all .gz compressed files multiple and the final path has 1 single clinical_event.txt file

almost 350GB in size.

Code:
hadoop fs -text /user/hive/warehouse/stage.db/CLINICAL_EVENT/CLINICAL_EVENT* | hadoop fs -put - $BASE_FOLDER/$base_name/final/clinical_event.txt

CLINICAL_EVENT* i have all .gz file 20 files and in the final path i only have 1 file .txt with almost 350GB file clinical_event.txt

is it uncompressing all .gz files almost 20 numbers from first path and consolidating into one big uncompressed .txt file with same name clinical_event.txt


Thanks a lot for the helpful info.
Moderator's Comments:
Mod Comment Please use CODE tags for full line and multi-line sample input, output, and code segments and ICODE tags for partial line sample input, output, and code segments.

Last edited by Don Cragun; 05-15-2016 at 04:51 PM.. Reason: Change ICODE tags to CODE tags and add ICODE tags.
 

10 More Discussions You Might Find Interesting

1. Programming

Uncompressing a Linux file on HP-UX

Hi!! All, I have a compressed C file made on Linux. On Linux I used to uncompress it using the command: zcat filename -xvf- |tar I am not able to uncompress the same on Hp-UX-10.2. Could someone help me out how to do it ?? Do I have to use any other command to uncompress it?? (2 Replies)
Discussion started by: jyotipg
2 Replies

2. Solaris

uncompressing a .gz file

Hi Friends, i am trying to uncompress a file which is of the following type filename.tar.gz. when i am using gunzip filename.tar.gz to unzip it, i am getting the error as filename .tar.gz: invalid compressed data--format violated what is the mistake that i am doing ? any other way to... (2 Replies)
Discussion started by: sveera
2 Replies

3. AIX

uncompressing file

Hi, how do u uncompress a .Z file? i have a file called software.all.Z and have to change it to software.all. how to do that? thanks! karthik (3 Replies)
Discussion started by: karthikosu
3 Replies

4. Shell Programming and Scripting

extract one file form .tar.gz without uncompressing .tar.gz file

hi all, kindly help me how to extract one file form .tar.gz without uncompressing .tar.gz file. thanks in advance bali (2 Replies)
Discussion started by: balireddy_77
2 Replies

5. Shell Programming and Scripting

unix script to takes the old data from a TXT file and compress them into new file

Hi, I am looking for the unix script which can takes the 2 month old data from a TXT file (there is one txt file in whiche messages are appended on daily basis) and compress them into new file.Please halp me out. (2 Replies)
Discussion started by: vpandey
2 Replies

6. UNIX for Dummies Questions & Answers

Binary txt file received when i use uuencode to send txt file as attachment

Hi, I have already read a lot of posts on sending attachments in unix...but none of them were of help for my problem...so here goes.. i wanna attach a text file and send to a mail id..used the following code : uuencode "$File1" "$File1" ;|mail -s "$Mail_sub" abc@abc.com it works... (2 Replies)
Discussion started by: ash22
2 Replies

7. Shell Programming and Scripting

Uncompressing .zip file specific directory Fedora 14

Hello, I have Fedora 14 installed on my machine I have a .zip file ( some latex package) which I want to unzip to some location in the Latex paths /usr/share.../texmf/.. so I went to super user mode, created the directory for this package over there, mkdir logo and tried... (1 Reply)
Discussion started by: ajayram
1 Replies

8. Windows & DOS: Issues & Discussions

2 Questions: replace text in txt file, add text to end of txt file

so... Lets assume I have a text file. The text file contains multiple "#" symbols. I want to replace all thos "#"s with a STRING using DOS/Batch I want to add a certain TEXT to the end of each line. How can I do this WITHOUT aid of sed, grep or anything linux related ? (1 Reply)
Discussion started by: pasc
1 Replies

9. OS X (Apple)

Uncompressing but not extracting tar.Z file

Hi I have a few hundred files with extension .tar.Z. These files were archived (tar) and compressed (Z) on a UNIX system. I need to unzip them but not extract them. In other words they need to go to .tar extension. I would like to do this on my MAC or on a windows pc. I do not have a UNIX... (3 Replies)
Discussion started by: kalbano
3 Replies

10. Shell Programming and Scripting

Help with Shell Scrip in Masking particular columns in .csv file or .txt file using shell script

Hello Unix Shell Script Experts, I have a script that would mask the columns in .csv file or .txt file. First the script will untar the .zip files from Archive folder and processes into work folder and finally pushes the masked .csv files into Feed folder. Two parameters are passed ... (5 Replies)
Discussion started by: Mahesh G
5 Replies
DH_COMPRESS(1)							     Debhelper							    DH_COMPRESS(1)

NAME
dh_compress - compress files and fix symlinks in package build directories SYNOPSIS
dh_compress [debhelperoptions] [-Xitem] [-A] [file...] DESCRIPTION
dh_compress is a debhelper program that is responsible for compressing the files in package build directories, and makes sure that any symlinks that pointed to the files before they were compressed are updated to point to the new files. By default, dh_compress compresses files that Debian policy mandates should be compressed, namely all files in usr/share/info, usr/share/man, files in usr/share/doc that are larger than 4k in size, (except the copyright file, .html and other web files, image files, and files that appear to be already compressed based on their extensions), and all changelog files. Plus PCF fonts underneath usr/share/fonts/X11/ FILES
debian/package.compress These files are deprecated. If this file exists, the default files are not compressed. Instead, the file is ran as a shell script, and all filenames that the shell script outputs will be compressed. The shell script will be run from inside the package build directory. Note though that using -X is a much better idea in general; you should only use a debian/package.compress file if you really need to. OPTIONS
-Xitem, --exclude=item Exclude files that contain item anywhere in their filename from being compressed. For example, -X.tiff will exclude TIFF files from compression. You may use this option multiple times to build up a list of things to exclude. -A, --all Compress all files specified by command line parameters in ALL packages acted on. file ... Add these files to the list of files to compress. CONFORMS TO
Debian policy, version 3.0 SEE ALSO
debhelper(7) This program is a part of debhelper. AUTHOR
Joey Hess <joeyh@debian.org> 8.9.0ubuntu2.1 2012-06-12 DH_COMPRESS(1)
All times are GMT -4. The time now is 08:25 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy