Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

1 command line gunzip -c get 3 output.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting 1 command line gunzip -c get 3 output.
# 1  
Old 03-31-2015
1 command line gunzip -c get 3 output.
Hi everyone----I have big file like 15G to 20G and took really long time to get the output, cause run 3 steps gunzip -c below to get the ouput. Is there is the way can run 1 command line gunzip -c then get all 3 output (Unzipped Bytes, Records, Record, Length) same time? Please any input for this? Thanks

Quote:
for Dtran in $(ls `cat $New_File`)
do
LETTER=`ls | awk -F. '{print $NF}' | sort -u`
case "$LETTER" in
"gz" )
echo "Filename : $Dtran" >> $README_FILE
echo " Data Format : ASCII with carriage returns and linefeeds" >> $README_FILE
echo " Compression : GZIP" >> $README_FILE
echo " GZIP Bytes : `ls -l $Dtran | awk '{print $5}'`" >> $README_FILE
echo " Unzipped Bytes : `gunzip -c $Dtran | wc -c | awk '{print $1}'`" >> $README_FILE
echo " Records : `gunzip -c $Dtran | wc -l | awk '{print $1}'`" >> $README_FILE
echo " Record Length : `gunzip -c $Dtran | awk '{ if ( length > L ) { L=length} }END{ print L}'" >> $README_FILE
printf '%s\n' >> $README_FILE


Here is new improve, but still 2 steps:
Quote:
echo "Filename : $Dtran" >> $README_FILE
echo " Data Format : ASCII with carriage returns and linefeeds" >> $README_FILE
echo " Compression : GZIP" >> $README_FILE
echo " GZIP Bytes : `ls -l $Dtran | awk '{print $5}'`" >> $README_FILE
gunzip -c $Dtran | wc -cl > /tmp/gunzip_${AUTO_JOB_NAME}
echo " Unzipped Bytes : `cat /tmp/gunzip_${AUTO_JOB_NAME} | awk '{print $2}'`" >> $README_FILE
echo " Records : `cat /tmp/gunzip_${AUTO_JOB_NAME} | awk '{print $1}'`" >> $README_FILE
echo " Record Length : `gunzip -c $Dtran | awk ' { if ( length > L ) { L=length} }END{ print L}'" >> $README_FILE
printf '%s\n' >> $README_FILE

Test different command line still not work.
Quote:
gunzip -c $Dtran | awk 'BEGIN { bytes = 0 } { bytes += length } END { print NR" "length" "bytes }' ...Still not get correct....
# 2  
Old 03-31-2015
what's the output of your gunzip -c $Dtran?
A sufficient/representative dozen lines posted using code tags, please.
# 3  
Old 03-31-2015
Quote:
Originally Posted by dotran
$Dtran = filename = test.txt.gz
gunzip -c filename

This is output with command line gunzip -c. I like run in 1 command gunzip -c | etc then get output for (Unzipped Bytes, Records, Record, Length) into a txt file.

Filename : test.txt.gz
Data Format : ASCII with carriage returns and linefeeds
Compression : GZIP
GZIP Bytes : 1253221736
Unzipped Bytes : 35750574720
Records : 26600130
Record Length : 1343
# 4  
Old 03-31-2015
Code:
gunzip -c filename | awk -F: '/^(Unzipped|Records|Record L)/{a[$1]+=$NF}END { for(i in a) print i FS a[i]}'

Last edited by vgersh99; 03-31-2015 at 08:04 PM..
# 5  
Old 04-01-2015
Thanks Vgersh9....but somehow didn't work at all.

Code:
 
 gunzip -c test.txt.gz | awk -F: '/^(Unzipped|Records|Record L)/{a[$1]+=$NF}END { for(i in a) print i FS a[i]}'
awk: a is not an array
 record number 424033

After a lot testing....I got it work. One command line gunzip -c and get 3 output (Unzipped Bytes, Records, Record, Length) same time. Thanks

Code:
 
for Dtran in $(ls `cat $New_File`)
do
LETTER=`ls | awk -F. '{print $NF}' | sort -u`
        case "$LETTER" in
        "gz" )
echo "Filename         : $Dtran" >> $README_FILE
echo " Data Format     : ASCII with carriage returns and linefeeds" >> $README_FILE
echo " Compression     : GZIP" >> $README_FILE
echo " GZIP Bytes      : `ls -l $Dtran | awk '{print $5}'`" >> $README_FILE
gunzip -c $Dtran | awk 'BEGIN { l = 0 } { l = (length)+1 } { b += (length)+1 } END { print NR" "l-1" "b }' > /tmp/gunzip_${AUTO_JOB_NAME}
echo " Unzipped Bytes  : `cat /tmp/gunzip_${AUTO_JOB_NAME} | awk '{print $3}'`" >> $README_FILE
echo " Records         : `cat /tmp/gunzip_${AUTO_JOB_NAME} | awk '{print $1}'`" >> $README_FILE
echo " Record Length   : `cat /tmp/gunzip_${AUTO_JOB_NAME} | awk '{print $2}'`" >> $README_FILE
printf '%s\n' >> $README_FILE
        ;;

# 6  
Old 04-01-2015
Quote:
Originally Posted by dotran
Thanks Vgersh9....but somehow didn't work at all.

Code:
 
 gunzip -c test.txt.gz | awk -F: '/^(Unzipped|Records|Record L)/{a[$1]+=$NF}END { for(i in a) print i FS a[i]}'
awk: a is not an array
 record number 424033

After a lot testing....I got it work. One command line gunzip -c and get 3 output (Unzipped Bytes, Records, Record, Length) same time. Thanks

Code:
 
for Dtran in $(ls `cat $New_File`)
do
LETTER=`ls | awk -F. '{print $NF}' | sort -u`
        case "$LETTER" in
        "gz" )
echo "Filename         : $Dtran" >> $README_FILE
echo " Data Format     : ASCII with carriage returns and linefeeds" >> $README_FILE
echo " Compression     : GZIP" >> $README_FILE
echo " GZIP Bytes      : `ls -l $Dtran | awk '{print $5}'`" >> $README_FILE
gunzip -c $Dtran | awk 'BEGIN { l = 0 } { l = (length)+1 } { b += (length)+1 } END { print NR" "l-1" "b }' > /tmp/gunzip_${AUTO_JOB_NAME}
echo " Unzipped Bytes  : `cat /tmp/gunzip_${AUTO_JOB_NAME} | awk '{print $3}'`" >> $README_FILE
echo " Records         : `cat /tmp/gunzip_${AUTO_JOB_NAME} | awk '{print $1}'`" >> $README_FILE
echo " Record Length   : `cat /tmp/gunzip_${AUTO_JOB_NAME} | awk '{print $2}'`" >> $README_FILE
printf '%s\n' >> $README_FILE
        ;;

I don't understand this at all, but if the OP is happy.....
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to read the output of a command line by line and pass it as a variable?

Hi, I have some 2000 names in a table like below. Java Oracle/SQL ANSI SQL SQL,DWH,DB DB&Java And by using for loop in my code i am able to get a single word but if there is any special character or space then it is considering as a next line. I have to execute the below queries in... (10 Replies)
Discussion started by: Samah
10 Replies

2. Shell Programming and Scripting

How to execute a command on each line of output from another command?

Hello :) new to bash not to programming. I have an on-going need to change the owning group on sets of files and directories from the one they were created with or changed to on update to the one they need to have going forward. find {target_root} -group wrong_group gets me a newline... (4 Replies)
Discussion started by: naftali
4 Replies

3. Shell Programming and Scripting

sed command to replace a line in a file using line number from the output of a pipe.

Sed command to replace a line in a file using line number from the output of a pipe. Is it possible to replace a whole line piped from someother command into a file at paritcular line... here is some basic execution flow.. the line number is 412 lineNo=412 Now i have a line... (1 Reply)
Discussion started by: vivek d r
1 Replies

4. UNIX for Dummies Questions & Answers

Gunzip and tar command

Hi I wanted to tar and gunzip a file named backup tar: backup.tar: Wrote only 2244 of 10240 bytes tar: Error is not recoverable: exiting now Please tell me what I am doing wrong? Please do help. (4 Replies)
Discussion started by: sonia102
4 Replies

5. UNIX for Dummies Questions & Answers

command for gunzip?

Hi All This is very basic query but I have a huge folder named backup that I need to transfer. What is the command to convert the file in format backup.tar.gz so that I could transfer the folder. Is the command gzip filename? Thanks Sonia. :wall: (6 Replies)
Discussion started by: sonia102
6 Replies

6. Shell Programming and Scripting

Can we get output on ls | wc -l command on the same line?

can we get o/p of ls | wc - l command on the same line. # ls nc*010412* | wc -l 23 # ls nc*010412* | wc -l 24 # ls nc*050412* | wc -l 21 # ls nc*040412* | wc -l 23 # ls nc*070412* | wc -l 22 my expectation is grepping specific lines from the file and... (8 Replies)
Discussion started by: sagar_1986
8 Replies

7. Shell Programming and Scripting

Bash - Loading a command's output line by line into an array

I have been trying this a lot of different ways and haven't found too much online. Here's what I've got so far: j=0 declare -a first zero=(`cat $tmpfile`) for i in "${zero}" do command $i >> "${first}" ... (4 Replies)
Discussion started by: Azrael
4 Replies

8. Shell Programming and Scripting

reading ps command's output line by line

hi; as a pseudo; while read psLine do myFunc $psLine done < ps i don't want to redirect ps command's to a file. in fact, my problem is "how can i read stdout line by line in bash, sed, awk or any?" thanks, (5 Replies)
Discussion started by: s. murat
5 Replies

9. UNIX for Dummies Questions & Answers

Command output on ONE line

We want to make a history of errors and append a line to a file each day. To do this we want to display the output of Date and DU (of 1 file) on ONE line. However if i type Date ; DU the output is displayed on two lines. Can Anyone help me please? The output should look like this; ... (1 Reply)
Discussion started by: Robey
1 Replies
Login or Register to Ask a Question