How to download Images and Json file from server(godaddy) to Local machine (Ubuntu 14.04).?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to download Images and Json file from server(godaddy) to Local machine (Ubuntu 14.04).?
# 8  
Old 01-09-2016
No need to apologize, no one was born with linux/unix skills.

No need to use cmp
This is why we are using wget mirroring based on timestamps.
wget will know if something has changed and copy it if it has (check the log, it will write it).

Try this for entire operation :
Code:
#!/bin/sh
# On Linux systems, /bin/sh is often link to dash or bash, on other unix systems it will be posix shell and stuff here might not work.
LOCKFILE=/home/gav/Desktop/wgetTest/lock/wget.lock
LOGFILE=/home/gav/Desktop/wgetTest/log/wget.log
DDIR=/home/gav/Desktop/wgetTest/
JSONDIR=/home/gav/Desktop/wgetTest/text/new.json
IMAGEDIR=/home/gav/Desktop/wgetTest/display/

if [ -f $LOCKFILE ]; then
printf "%s\n" "Lock file exists on date $(date "+%Y%m%d_%H%M")" >> $LOGFILE
exit 1
fi
while true
do
    cd $DDIR || exit 1
    wget -m -c -nH --cut-dirs=2 --output-file=$LOGFILE --timeout=3 --tries=3 --passive-ftp ftp://usr:pwd@mywesite.com/www/test/images/*
    WEXIT1=$? # we shall catch the exit code of wget, 0 is expected (success)
    wget -m -c -nH --cut-dirs=2 --output-file=$LOGFILE --timeout=3 --tries=3 --passive-ftp ftp://usr:pwd@mywesite.com/www/test/display.json
    WEXIT2=$? # we shall catch the exit code of wget, 0 is expected (success)
    ERROR=$(( $WEXIT1 + $WEXIT2 )) # if the sum of both exit codes
    if [ $ERROR -eq 0 ]; then # is 0 (success both, expected), proceed with mv
         mv images/* $IMAGEDIR # we are not checking for mv status, we hope filesystem will work :) (you could do a $? checking here if required)
         mv display.json $JSONDIR  # we are not checking for mv status, we hope filesystem will work :) (you could do a $? checking here if required)
         rm $LOCKFILE # remove the $LOCKFILE for next itteration since everything succeded.
    else
    printf "%s\n" "A wget error has happend, check $LOGFILE, remove $LOCKFILE by hand and rerun"
   exit 1
   fi
sleep 60
done

I would rather recommend using cron every minute with less shell code and no while loops.
Also i haven't ran this code i just wrote it directly on forums, so give it a try.

Hope this is what you are looking for, but again, can you use SSH with rsync ?
With that and exchanged ssh keys this entire script would be a couple of lines.

Best regards
Peasant.
This User Gave Thanks to Peasant For This Post:
# 9  
Old 01-09-2016
I will test this code and let you know the results soon Smilie Thanks

I can use ssh with rsync but I dont know much about how to set it up on godaddy webhosting.

I am using while loops because I want to create a service(upstart) for this later that initiate this script during boot and keep it always running at background. Is it a good idea or you prefer cron job more?

---------- Post updated at 04:03 AM ---------- Previous update was at 03:53 AM ----------

One question, can i put
Code:
rm $LOCKFILE

in the else section just after the error message so that I dont need to remove it manually
# 10  
Old 01-09-2016
I would prefer cron job.
If you use cron to run every minute, there is no need for while loop.

Regarding LOCKFILE..
It is there for the sole purpose for a human to check what went wrong and, when fixed, to remove it and to stop script starting when another instance of the same script is running.

I have no experience with upstart so far so i cannot advise on that subject, but it looks like an overhead to me.

I'm not acquainted with godaddy and their limits or features, you will have to examine that yourself using google.
This User Gave Thanks to Peasant For This Post:
# 11  
Old 01-09-2016
Thanks a lot Peasant for your help Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Red Hat

iptables applied in local machine, can't ssh remote machine after chain changed to DROP

I want to SSH to 192.168.1.15 Server from my machine, my ip was 192.168.1.99 Source Destination was UP, with IP 192.168.1.15. This is LAN Network there are 30 Machine's Connected to the network and working fine, I'm Playing around the local machine's because I need to apply the same rules in... (2 Replies)
Discussion started by: babinlonston
2 Replies

2. UNIX for Dummies Questions & Answers

Copy files from Linux server local windows machine using a shell script

Hello, I need to create a shell script which will copy files - which are created on particular date and starting with particular name - to local windows XP machine. Is this possible.? Currently it is being done manually using winscp (1 Reply)
Discussion started by: NarayanaPrakash
1 Replies

3. UNIX for Advanced & Expert Users

FTP While transfering files to local machine to remote machine

Hi Am using unix Ksh Am getting the problem while transferring zero size files through the script . When i transfer zero size files from local machine to remote machine manually i can able to do it . My question its beause of zero size files am not able to transfer through script ? or its... (2 Replies)
Discussion started by: Venkatesh1
2 Replies

4. Solaris

Copying from server to a local machine

Hi, I am trying to copy a file from oracle home on a unix platform to my local machine. I think i can use ftp but i am not sure of the exact syntax and way i should write this. Can someone help please? (2 Replies)
Discussion started by: dollypee
2 Replies

5. IP Networking

How to create Local IPv6 and IPv4 networks by using ubuntu server as router

Hi, I have two Cent OS Linux systems and one Ubuntu OS system contains two Ethernet Cards. I need to create two networks by using Ubuntu sys as router. One network need to configure in IPv4 addresses and another network should configure in IPv6 addresses. Please help me how to configure two... (0 Replies)
Discussion started by: hanuma614
0 Replies

6. Shell Programming and Scripting

How to transfer files from unix machine to local machine using shell script?

Hi All.. Am new to Unix!! Am creating a shell script in which a scenario is like i have transfer the output file from unix machine (Server) to local directory (Windows xp). And also i have to transfer the input file from the local directory to Unix machine (Server) Any help from you... (1 Reply)
Discussion started by: vidhyaS
1 Replies

7. Shell Programming and Scripting

check web server running on local and on remote machine

Hi , How to check whether web server is running from remote machine How to check whether web server is running on web server itself Can any one help me soon (1 Reply)
Discussion started by: satheeshkr_cse
1 Replies

8. Red Hat

To find the LATEST file from a dir on REMOTE machine and SCP to local machine?

Hi All, URGENT - Please help me form a scipt for this: I need the LATEST file from a dir on REMOTE machine to be SCP'd to a dir on local machine. (and I need to execute this from local server) I know that the below cmd is used to find the LATEST file from a dir. But this command is not... (3 Replies)
Discussion started by: me_ub
3 Replies

9. UNIX for Dummies Questions & Answers

Copy file from local machine to server?

I am logged into a server via SSH. There is a file on my desktop of my Windows PC I would like to put on that server. How do I do it? SCP looks the likely method but how does the server know where the file is on the local machine? Thanks. (2 Replies)
Discussion started by: Sepia
2 Replies

10. UNIX for Advanced & Expert Users

Using SCP command in IBM AIX to download file from remote to local system

Hi, When i run the code in solaris unix machine, the file from remote server is getting downloaded. but when i use the same code in IBM AIX remote machine, it is not running. It is saying "Erro during scp transfer." Below is the code. Please give some resolution. SCPClient client = new... (1 Reply)
Discussion started by: gravi2020
1 Replies
Login or Register to Ask a Question