Sponsored Content
Top Forums Shell Programming and Scripting Is the following script uncompressing .gz file into txt file Post 302973178 by Aia on Sunday 15th of May 2016 06:08:55 PM
Old 05-15-2016
hadoop is a command line utility with many subcommands.
the fs is one of those subcommands.
The subcommand fs accepts many options, one of those options is -text, which requires a source, one of the sources can be multiple gzip files and they get outputted into text format.
Another option of the fs subcomand is -put which requires two arguments, a local source and a destination.

Let's use that to know what's going on:
There are two instances of hadoop fs working together joined by a pipe (|), where the output of the first one, becomes the first argument of the second one.
In essence, it converts all those .gzip files (assuming that CLINICAL_EVENT*, refers to only gzip files, like you said) to text format and pushes it to standard output where the second picks it up, via standard input, (that's the - highlighted in red) and places it into /user/hive/warehouse/stage.db/Clinical_event/final/clinical_event.txt, which is a single file.

Legend:
Blue: subcommand with option
Red: stdin
Purple: pipe
Code:
hadoop fs -text /user/hive/warehouse/stage.db/CLINICAL_EVENT/CLINICAL_EVENT* | hadoop fs -put - /user/hive/warehouse/stage.db/Clinical_event/final/clinical_event.txt

 

10 More Discussions You Might Find Interesting

1. Programming

Uncompressing a Linux file on HP-UX

Hi!! All, I have a compressed C file made on Linux. On Linux I used to uncompress it using the command: zcat filename -xvf- |tar I am not able to uncompress the same on Hp-UX-10.2. Could someone help me out how to do it ?? Do I have to use any other command to uncompress it?? (2 Replies)
Discussion started by: jyotipg
2 Replies

2. Solaris

uncompressing a .gz file

Hi Friends, i am trying to uncompress a file which is of the following type filename.tar.gz. when i am using gunzip filename.tar.gz to unzip it, i am getting the error as filename .tar.gz: invalid compressed data--format violated what is the mistake that i am doing ? any other way to... (2 Replies)
Discussion started by: sveera
2 Replies

3. AIX

uncompressing file

Hi, how do u uncompress a .Z file? i have a file called software.all.Z and have to change it to software.all. how to do that? thanks! karthik (3 Replies)
Discussion started by: karthikosu
3 Replies

4. Shell Programming and Scripting

extract one file form .tar.gz without uncompressing .tar.gz file

hi all, kindly help me how to extract one file form .tar.gz without uncompressing .tar.gz file. thanks in advance bali (2 Replies)
Discussion started by: balireddy_77
2 Replies

5. Shell Programming and Scripting

unix script to takes the old data from a TXT file and compress them into new file

Hi, I am looking for the unix script which can takes the 2 month old data from a TXT file (there is one txt file in whiche messages are appended on daily basis) and compress them into new file.Please halp me out. (2 Replies)
Discussion started by: vpandey
2 Replies

6. UNIX for Dummies Questions & Answers

Binary txt file received when i use uuencode to send txt file as attachment

Hi, I have already read a lot of posts on sending attachments in unix...but none of them were of help for my problem...so here goes.. i wanna attach a text file and send to a mail id..used the following code : uuencode "$File1" "$File1" ;|mail -s "$Mail_sub" abc@abc.com it works... (2 Replies)
Discussion started by: ash22
2 Replies

7. Shell Programming and Scripting

Uncompressing .zip file specific directory Fedora 14

Hello, I have Fedora 14 installed on my machine I have a .zip file ( some latex package) which I want to unzip to some location in the Latex paths /usr/share.../texmf/.. so I went to super user mode, created the directory for this package over there, mkdir logo and tried... (1 Reply)
Discussion started by: ajayram
1 Replies

8. Windows & DOS: Issues & Discussions

2 Questions: replace text in txt file, add text to end of txt file

so... Lets assume I have a text file. The text file contains multiple "#" symbols. I want to replace all thos "#"s with a STRING using DOS/Batch I want to add a certain TEXT to the end of each line. How can I do this WITHOUT aid of sed, grep or anything linux related ? (1 Reply)
Discussion started by: pasc
1 Replies

9. OS X (Apple)

Uncompressing but not extracting tar.Z file

Hi I have a few hundred files with extension .tar.Z. These files were archived (tar) and compressed (Z) on a UNIX system. I need to unzip them but not extract them. In other words they need to go to .tar extension. I would like to do this on my MAC or on a windows pc. I do not have a UNIX... (3 Replies)
Discussion started by: kalbano
3 Replies

10. Shell Programming and Scripting

Help with Shell Scrip in Masking particular columns in .csv file or .txt file using shell script

Hello Unix Shell Script Experts, I have a script that would mask the columns in .csv file or .txt file. First the script will untar the .zip files from Archive folder and processes into work folder and finally pushes the masked .csv files into Feed folder. Two parameters are passed ... (5 Replies)
Discussion started by: Mahesh G
5 Replies
set_color(1)                                                           fish                                                           set_color(1)

NAME
set_color - set_color - set the terminal color set_color - set the terminal color Synopsis set_color [-v --version] [-h --help] [-b --background COLOR] [COLOR] Description Change the foreground and/or background color of the terminal. COLOR is one of black, red, green, brown, yellow, blue, magenta, purple, cyan, white and normal. o -b, --background Set the background color o -c, --print-colors Prints a list of all valid color names o -h, --help Display help message and exit o -o, --bold Set bold or extra bright mode o -u, --underline Set underlined mode o -v, --version Display version and exit Calling set_color normal will set the terminal color to whatever is the default color of the terminal. Some terminals use the --bold escape sequence to switch to a brighter color set. On such terminals, set_color white will result in a grey font color, while set_color --bold white will result in a white font color. Not all terminal emulators support all these features. This is not a bug in set_color but a missing feature in the terminal emulator. set_color uses the terminfo database to look up how to change terminal colors on whatever terminal is in use. Some systems have old and incomplete terminfo databases, and may lack color information for terminals that support it. Download and install the latest version of ncurses and recompile fish against it in order to fix this issue. Version 1.23.1 Sun Jan 8 2012 set_color(1)
All times are GMT -4. The time now is 01:05 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy