👤


UNIX for Beginners Questions & Answers

If you're not sure where to post a Unix or Linux question, post it here. All unix and Linux beginners welcome in this forum!

Mirror a folder from the Internet to local machine

👤 Login to reply
 
Thread Tools Search this Thread Display Modes
    #1  
Old 01-02-2018
atux_null atux_null is offline
Registered User
 
Join Date: Jan 2018
Last Activity: 2 January 2018, 4:09 AM EST
Posts: 2
Thanks: 0
Thanked 0 Times in 0 Posts
Mirror a folder from the Internet to local machine

I have a machine in my LAN that runs Ubuntu and I have full access in it.
There is another PC on the Internet that has a public IP eg 3.3.3.3 and it servers as https server and access to it is by username and password. On the remote PC all i have is access through https using username/passwd.

At the moment I'm logging to the ubuntu and using wget to get folders that I want.
I would like to automate the boring stuff and every 1 hour to check if the server has new content compared to the ubuntu's /var/www/html/x3 and /var/www/html/y7 local folders.
The Internet server has as series of folders eg x1, x2,...,xn, y1,y2,...,yn. i do need to get only 2 particular folder and its contents and not the whole server.
If those 2 folders have different content, then download the new content and delete the old ones in the corresponding folder.
It is a one way sync. is there any script that could do the job with cron, please?
Sponsored Links
    #2  
Old 01-02-2018
vbe's Unix or Linux Image
vbe vbe is offline Forum Staff  
Moderator
 
Join Date: Sep 2005
Last Activity: 16 July 2018, 9:56 AM EDT
Location: Switzerland - GE
Posts: 6,550
Thanks: 311
Thanked 564 Times in 526 Posts
I think looking at the man pages of rsync command would be a good start, as rsync is very versatile and may just do the job you want, I used to use it to synchronise /etc/passwd and other changing stuff in clusters...
Sponsored Links
    #3  
Old 01-02-2018
atux_null atux_null is offline
Registered User
 
Join Date: Jan 2018
Last Activity: 2 January 2018, 4:09 AM EST
Posts: 2
Thanks: 0
Thanked 0 Times in 0 Posts
Hi. Thanks a lot for the quick reply. I thought of rsync, but the only access to the remote PC is through https.
as i see it i need a script that checks the current directory against the remote. if there is difference then wget the new and delete the old ones.
    #4  
Old 02-09-2018
abdulbadii abdulbadii is offline
Registered User
 
Join Date: Jan 2018
Last Activity: 23 April 2018, 1:31 AM EDT
Posts: 47
Thanks: 3
Thanked 7 Times in 7 Posts
There's only one way synchronizing with wget -m <URL>
which will copy what in server to your local storage. Try
wget --spider .. read its official man; GNU Wget 1.18 Manual
Sponsored Links
👤 Login to reply

« Previous Thread | Next Thread »
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Help with fetching the data from remote machine from my jumpbox(local machine) whizkidash Shell Programming and Scripting 20 08-15-2014 11:42 AM
iptables applied in local machine, can't ssh remote machine after chain changed to DROP babinlonston Red Hat 2 06-05-2014 12:57 AM
How to transfer files from unix machine to local machine using shell script? vidhyaS Shell Programming and Scripting 1 08-08-2011 10:27 AM
To find the LATEST file from a dir on REMOTE machine and SCP to local machine? me_ub Red Hat 3 07-08-2009 12:57 AM
Copying multiple folders to local machine (don't know folder names) leenyburger UNIX for Dummies Questions & Answers 5 06-12-2008 07:38 AM



All times are GMT -4. The time now is 11:03 PM.

Unix & Linux Forums Content Copyrightę1993-2018. All Rights Reserved.
×
UNIX.COM Login
Username:
Password:  
Show Password





Not a Forum Member?
Forgot Password?