Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Compare output from two commands

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare output from two commands
# 1  
Old 08-12-2013
Compare output from two commands
Hello folks,
I would like ask for a help in one script I'm currently working on.
Main goal is to compare size of file available on remote site and file already downloaded (check if downloaded file is complete and non corrupted).

Code:
# Check if files were downloaded correctly
for FILES in $FILES_TO_DOWN
   do
        LOCAL_SIZE=`ls -la $FILES | awk '{print $5}'`
        REMOTE_SIZE=`$CURL -sI -u user:pass --proxy http://gw:8080 --proxy-user user:pass https://remote_site/$FILES | grep 'Content-Length:' | cut -f 2
 -d" " | tr -d ' '`
        if [ "$LOCAL_SIZE" -eq "$REMOTE_SIZE" ]
                then
                        echo "File is OK"
                else
                        echo "File is corrupted"
        fi
done

In output from script I see this:
Code:
+ for FILES in '$FILES_TO_DOWN'
++ ls -la file_20130715.xls
++ awk '{print $5}'
+ LOCAL_SIZE=89984
++ /opt/sfw/bin/curl -sI -u user:pass --proxy http://gw:8080 --proxy-user user:pass https://remote_site/$FILES/file_20130715.xls
++ grep Content-Length:
++ cut -f 2 '-d '
++ tr -d ' '
+ REMOTE_SIZE=$'89984\r'
+ '[' 89984 -eq $'89984\r' ']'
: integer expression expectedine 53: [: 89984
+ echo 'File is corrupted'
File is corrupted

So it looks that local file size is returned as integer and size of remote file is returned as a string.
Can anybody kick me to the right way how do get rid of it?? I suppose that reasonable is try to get size of remote file as well as integer.

Many thanks,
Stan
Last edited by brusell; 08-12-2013 at 08:49 AM..
# 2  
Old 08-12-2013
You need to remove \r
Try this...
Code:
REMOTE_SIZE=`$CURL -sI -u user:pass --proxy http://gw:8080 --proxy-user user:pass https://remote_site/$FILES | grep 'Content-Length:' | cut -f 2
 -d" " | tr -d ' ' | sed 's/\r//g' `

--ahamed

---------- Post updated at 03:54 AM ---------- Previous update was at 03:50 AM ----------
Or

Code:
REMOTE_SIZE=$( $CURL -sI -u user:pass --proxy http://gw:8080 --proxy-user user:pass https://remote_site/$FILES | 
awk '/Content-Length/{sub(/\r/,"",$2); print $2}' )

--ahamed

---------- Post updated at 04:09 AM ---------- Previous update was at 03:54 AM ----------

I just noticed the $ in the output. Can you paste the output of the curl command?

--ahamed
# 3  
Old 08-12-2013
Quote:
Originally Posted by ahamed101
You need to remove \r
Try this...
Code:
REMOTE_SIZE=`$CURL -sI -u user:pass --proxy http://gw:8080 --proxy-user user:pass https://remote_site/$FILES | grep 'Content-Length:' | cut -f 2
 -d" " | tr -d ' ' | sed 's/\r//g' `

--ahamed

---------- Post updated at 03:54 AM ---------- Previous update was at 03:50 AM ----------
Or

Code:
REMOTE_SIZE=$( $CURL -sI -u user:pass --proxy http://gw:8080 --proxy-user user:pass https://remote_site/$FILES | 
awk '/Content-Length/{sub(/\r/,"",$2); print $2}' )

--ahamed

---------- Post updated at 04:09 AM ---------- Previous update was at 03:54 AM ----------

I just noticed the $ in the output. Can you paste the output of the curl command?

--ahamed
Yep....
Code:
/opt/sfw/bin/curl -sI -u user:pass --proxy http://gw:8080 --proxy-user user:pass https://remote_site/$FILES/file_20130715.xls
HTTP/1.1 200 OK
Date: Mon, 12 Aug 2013 12:00:06 GMT
Content-Type: application/vnd.ms-excel
Last-Modified: Mon, 03 Jun 2013 05:00:30 GMT
Content-Length: 89984
Cache-control: private
Pragma: no-cache
Connection: close
X-CRC: 3A871FE5
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block

What I'm trying to dig out from output is Content-Length as integer, but it seems that I get is as a string.

Code:
/opt/sfw/bin/curl -sI -u user:pass --proxy http://gw:8080 --proxy-user user:pass https://remote_site/$FILES/file_20130715.xls| grep 'Content-Length:' | cut -f 2 -d " "
89984

# 4  
Old 08-12-2013
You can use the command which I gave you, that will work!
Try and let us know. In case, if it doesn't, paste the debug output.

--ahamed
# 5  
Old 08-12-2013
First try with awk '{print $2}' where I should get the same result as from previous try
Code:
++ ls -la file_20130715.xls
++ awk '{print $5}'
+ LOCAL_SIZE=89984
++ /opt/sfw/bin/curl -sI -u user:pass --proxy http://gw:8080 --proxy-user user:pass https://remote_site/$FILES/file_20130715.xls
++ grep Content-Length:
++ awk '{print $2}'
+ REMOTE_SIZE=$'89984\r'
+ '[' 89984 -eq $'89984\r' ']'
: integer expression expected 29: [: 89984
+ echo 'File is corrupted'
File is corrupted


Next with yours first command

Code:
++ ls -la file_20130715.xls
+ LOCAL_SIZE=89984
++ grep Content-Length:
++ /opt/sfw/bin/curl -sI -u user:pass --proxy http://gw:8080 --proxy-user user:pass https://remote_site/$FILES/file_20130715.xls
++ cut -f 2 '-d '
++ tr -d ' '
++ sed 's/\r//g'
+ REMOTE_SIZE=$'89984\r'
+ '[' 89984 -eq $'89984\r' ']'
: integer expression expected 29: [: 89984
+ echo 'File is corrupted'
File is corrupted

Second one from you with awk
Code:
++ ls -la file_20130715.xls
++ awk '{print $5}'
+ LOCAL_SIZE=89984
++ /opt/sfw/bin/curl -sI -u user:pass --proxy http://gw:8080 --proxy-user user:pass https://remote_site/$FILES/file_20130715.xls
++ awk '/Content-Length/{sub(/\r/,"",$2); print $2}'
awk: syntax error near line 1
awk: illegal statement near line 1
+ REMOTE_SIZE=
+ '[' 89984 -eq '' ']'
./download1.sh: line 29: [: : integer expression expected
+ echo 'File is corrupted'
File is corrupted

# 6  
Old 08-12-2013
Are you on solaris? Try using nawk

--ahamed
This User Gave Thanks to ahamed101 For This Post:
brusell
# 7  
Old 08-12-2013
Quote:
Originally Posted by ahamed101
Are you on solaris? Try using nawk

--ahamed
Yep this is Solaris, sorry I forgot to mention it on the beginning since I suppose that grep/sed/awk etc. are having the same syntax.
Anyway this solved my problem.

Many thanks for your help Ahamed, You rock!

Stan
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Arranging output for 2 commands

I am trying to run 2 sets of commands but want their output in a particular format. The 2 commands are : md5sum $WAR_DIR/$war and java -jar $WAR_DIR/$war | grep build.release.version | awk '{print $3}' The first command gives an output of 5f5261a33b92a36f80218cf14e8271ad ... (4 Replies)
Discussion started by: Junaid Subhani
4 Replies

2. Shell Programming and Scripting

How to compare below output?

I'm very new to shell scripting, below is my output in status.txt CM TARGET ACTUAL ---------------------------------- ---------- ---------- Conflict Resolution Manager 1 1 Internal Manager 1 ... (6 Replies)
Discussion started by: dhaawale
6 Replies

3. Shell Programming and Scripting

Compare two files and get output

Hi, I have two files, file1 and file2 and I need to compare them by line (exact match, order of the lines is not important) and get output with lines from file2 that are not found in file1 (not other way around). How do I do that? With grep or otherwise.. Thankyou (2 Replies)
Discussion started by: orp56
2 Replies

4. Shell Programming and Scripting

compare 2 files > output new to third

Hi, I have a question of comparing to files and output the result third file where file1 is the mainfile containing processed dir data and 2nd file grepīs dirīs data again (could be newer dirs comparing file1<file2) now i wanna make shure that output in file3 only contains newer dirs hx... (1 Reply)
Discussion started by: needle
1 Replies

5. Shell Programming and Scripting

Compare two file & output

Hi every body i have a problem need help urgently file 1 (approx 200K entries) aaaaa bbbb cccccc dddd ffff file 2 (approx 2 million entries) aaaaa,1,ee,44,5t,6y, bbbb,3,ff,66,5u,8r, cccccc, ..... dddd, ..... eeeeee, ..... ffff, ...... (5 Replies)
Discussion started by: The_Archer
5 Replies

6. Solaris

Logging commands and output

I'm looking for a CLI utility that will capture all the commands you type at the Solaris CLI (and their output) into a file. I'm sure it's called "scripter", but I can't find anything on a command called scripter. Does anyone know of a such a command? Your help will be greatly... (3 Replies)
Discussion started by: soliberus
3 Replies

7. Shell Programming and Scripting

Combine Two Commands Output

How i can combine output of two commands in one file.......i tried this but it is not working although each command is working good seperately..... head -1 filename | tail -1 filename i think there is problem with command concatenator? (16 Replies)
Discussion started by: 33junaid
16 Replies

8. UNIX for Dummies Questions & Answers

Output of 2 commands into 1 file

How could you put the output of two commands into one file using a single command? For example put the output of a grep command and a sort command into one file together. Here is another rough explanation of what I am trying to do; output of $ grep pattern file1 plus output of $ sort file... (8 Replies)
Discussion started by: enuenu
8 Replies

9. UNIX for Advanced & Expert Users

Output all commands to logfiles ???

Dear Forum, My .cshrc settings are embedded in a massive jungle of code distributed all over the place, where finding anything is a "needle in a haystack" daily pain in the royal backside. Is there anyway, i can dump out every command and file executed to STDOUT after sourcing my .cshrc ??? ... (2 Replies)
Discussion started by: fawqati
2 Replies

10. Shell Programming and Scripting

Compare output from awk to a value

I need to write a shell script which checks the CPU utilization is above 90% for a particular process. So far I have done this prstat 0 1 | grep weblogic | awk '{print $9}' This give an output like this 0.1% 0.0% 0.0% 0.0% 0.0% Now I need to assign this to an array and check... (4 Replies)
Discussion started by: venishjoe
4 Replies
Login or Register to Ask a Question