Sponsored Content
Special Forums UNIX and Linux Applications How can I make multiple mirror websites on the fly? Post 302312849 by eurusd on Sunday 3rd of May 2009 09:53:24 PM
Old 05-03-2009
How can I make multiple mirror websites on the fly?

-How can I make multiple mirror websites on the fly?
How can I make multiple mirror websites on the fly, transparently, seemingly created, from multiple donor sites?
For example hxxp://website.that-i-want-to-mirror.net/ to my hxxp://mymirror.website.com on dedicated server with WINDOWS or LINUX?
Without IFRAME, FRAME, etc.
So it looks like my domain, my html code output when “view source” done.
Is there any software proxy that sits in the process and when accessed to my specific domain creates WebPages on the fly from specific domain I pointed at and attaches codes or replaces code to output WebPages, let's say to the </body> tag or any other tag or string or regexp, etc.
For now I use offline copier like teleport pro, or offline browser but this is time consuming, space, effort, traffic etc and old-school and inefficient.
I make offline copy of a desired website, for example it takes 600 MB space and 235 000 files and many hours of downloading and uploading and tweaking and it's already outdated. Then upload it to my server and show it through hxxp://mymirror.website.com and then show it. And if I want to make 20 mirrored websites this tasks become impossible to do....
I want it seemingly output through my domain name, may be by CURL, SOCKET, or any other means....
Any ideas and suggestions, guys?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to make the same change in multiple shell scripts?

I would like to make the same change in multiple shell script files and would like to know if anyone can be of some help? I would appreciate it. (4 Replies)
Discussion started by: rdakhan
4 Replies

2. Shell Programming and Scripting

Moving multiple files and renaming them on the fly

Hi All, Being new to scripting I am facing a new situation. We have an application that generates a file lets say dumpfile for each user under the users home directory when they execute the application. This is quite a huge file and imagine having that for over 40 users on a daily basis. The... (1 Reply)
Discussion started by: daemongk
1 Replies

3. Solaris

ZFS Mirror versus Hardware Mirror

I've looked a little but haven't found a solid answer, assuming there is one. What's better, hardware mirroring or ZFS mirroring? Common practice for us was to use the raid controllers on the Sun x86 servers. Now we've been using ZFS mirroring since U6. Any performance difference? Any other... (3 Replies)
Discussion started by: Lespaul20
3 Replies

4. Solaris

What is mirror and sub mirror in RAID -1 SVM

Hi , I am new to SVM .when i try to learn RAID 1 , first they are creating two RAID 0 strips through metainit d51 1 1 c0t0d0s2 metainit d52 1 1 c1t0d0s2 In the next step metainit d50 -m d51 d50: Mirror is setup next step is metaattach d50 d52 d50 : submirror d52 is... (7 Replies)
Discussion started by: vr_mari
7 Replies

5. UNIX for Dummies Questions & Answers

sed command, make multiple lines into one

:confused:Hi, I'm relativley new at unix so am having difficulties at the most basic of areas. I am trying using sed to make multiple lines into one line. For example, i would like: Mary had a little lamb to look like this Maryhadalittlelamb so far i have tried sed... (1 Reply)
Discussion started by: cavanac2
1 Replies

6. Solaris

Solaris Volume Manager: Break Mirror and use to make a 2nd machine

Hello: I have a machine built on 2 drives and mirrors have been created to a second set of 2 drives (4 total) on the same platform. SVM. Might anyone provide guidance to create a second machine by breaking the mirrored set, moving the two mirrors (2) to another machine (same platform type),... (1 Reply)
Discussion started by: 4dailyrunner
1 Replies

7. Shell Programming and Scripting

How to make this run in multiple threads

Hi, I have a list of URLs in a csv file which I'm checking for page status. It just prints the URL and the status as output. This works perfectly fine. I'm looking to run this in multiple threads to make this process faster. I'm pretty new to Perl and I managed to complete this. It would be... (9 Replies)
Discussion started by: kzenthil
9 Replies

8. Shell Programming and Scripting

Make multiple files of equal length

I have 150 files with 4 columns each but variable row lengths that I need to combine by column. I do not have any common column. I want to use "paste " command in unix to do it but before that I have to get all my files to be of equal length. Is there a way using awk or sed to fill up n no. of... (7 Replies)
Discussion started by: manishabh
7 Replies

9. Shell Programming and Scripting

How to make multiple copies?

At work I have to create multiple copies of a file all the time. Example: I have a file called Sec30p01.txt I need thirty of these, then I edit one line in each to make 30 different control files. So I end up with Sec30p02.txt, Sec30p03.txt and so on up to 30 Currently I copy the first file... (8 Replies)
Discussion started by: faaslave
8 Replies

10. HP-UX

What is the difference between DRD and Root Mirror Disk using LVM mirror ?

what is the difference between DRD and Root Mirror Disk using LVM mirror ? (3 Replies)
Discussion started by: maxim42
3 Replies
MM(1L)																	    MM(1L)

NAME
mm - mirror master SYNOPSIS
mm [flags] [package-files] DESCRIPTION
mm is the mirror master. It runs multiple calls to mirror in parallel and handles locking and minimal retry times. As with mirror, mm is a package written in Perl. mm presumes that all the package details are stored in a directory called packages. All the packages for a given site must be in a file whose name is the same as that of the site. So in packages/sunsite.org.uk will be all the package details for the host sunsite.org.uk. OPTIONS
-t Ignore the minimal retry timers. -o site:package Only mirror the given site:package. -debug Enable debugging. If this argument is given more than once the debugging level will increase. Currently the maximum useful level is four. -s Enable status debugging. This will print out the status of any subprocess as it is spawned or as it exits. CONFIGURATION FILE The configuration file is parsed as a series of statements. Blank lines and lines beginning with a hash are ignored. There are are two kinds of statements, controls and site:package details. Control statements are of the form: keyword=value You can add whitespace around the equals. Here is a list of the keywords and their values, any defaults are given inside square brackets: home Directory that mm chdirs to before doing any work. [''] max Maximum number of mirrors that can be running in parallel at any one time ['6'] mirror This is the way to call mirror. Any occurence of $args, $package or $site is replace with the appropriate entry from the package. Any $pkg is replaced with a version of $package with all characters that cause problems in filenames converted to underscore. The default is: exec ./mirror $args -p'$package' packages/$site > logs/$site:$pkg 2>&1 [''] cmd Run the given command. This can occur multiple times. [''] cmdin Run the given command and use its output as mm input. This can occur multiple times. [''] skip Skip the given site:package when it occurs. [''] In addition to control statements there are also package details. Each package details statement is of the form: site:package min-restart-last-ok min-restart-last-notok mirror-args The site and package fields are matched against those in the packages. A mirror will only be run to this site:package if the last mirror run was more than min hours out. The min-restart-last-ok limit will be used if the last call to mirror was successful. If the mirror call failed then min-restart-last-notok will be used. EXAMPLES
Here is a simple mm input file to manage mirroring two packages. home=/public/mirror ftp.cl.cam.ac.uk:JIPS-nrs 20 20 hcrl.open.ac.uk:hcrl-psion 20 20 More usually the auxilly program pkgs_to_mmin will be used to automatically convert the contents of the packages directory into the site:package statements. home=/public/mirror cmdin=./pkgs_to_mmin packages/* AUTHOR
Written by Lee McLoughlin <lmjm@icparc.ic.ac.uk>. 21 April 1993 MM(1L)
All times are GMT -4. The time now is 11:35 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy