How can I make multiple mirror websites on the fly?

 
Thread Tools Search this Thread
Special Forums UNIX and Linux Applications How can I make multiple mirror websites on the fly?
# 1  
Old 05-03-2009
How can I make multiple mirror websites on the fly?

-How can I make multiple mirror websites on the fly?
How can I make multiple mirror websites on the fly, transparently, seemingly created, from multiple donor sites?
For example hxxp://website.that-i-want-to-mirror.net/ to my hxxp://mymirror.website.com on dedicated server with WINDOWS or LINUX?
Without IFRAME, FRAME, etc.
So it looks like my domain, my html code output when “view source” done.
Is there any software proxy that sits in the process and when accessed to my specific domain creates WebPages on the fly from specific domain I pointed at and attaches codes or replaces code to output WebPages, let's say to the </body> tag or any other tag or string or regexp, etc.
For now I use offline copier like teleport pro, or offline browser but this is time consuming, space, effort, traffic etc and old-school and inefficient.
I make offline copy of a desired website, for example it takes 600 MB space and 235 000 files and many hours of downloading and uploading and tweaking and it's already outdated. Then upload it to my server and show it through hxxp://mymirror.website.com and then show it. And if I want to make 20 mirrored websites this tasks become impossible to do....
I want it seemingly output through my domain name, may be by CURL, SOCKET, or any other means....
Any ideas and suggestions, guys?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. HP-UX

What is the difference between DRD and Root Mirror Disk using LVM mirror ?

what is the difference between DRD and Root Mirror Disk using LVM mirror ? (3 Replies)
Discussion started by: maxim42
3 Replies

2. Shell Programming and Scripting

How to make multiple copies?

At work I have to create multiple copies of a file all the time. Example: I have a file called Sec30p01.txt I need thirty of these, then I edit one line in each to make 30 different control files. So I end up with Sec30p02.txt, Sec30p03.txt and so on up to 30 Currently I copy the first file... (8 Replies)
Discussion started by: faaslave
8 Replies

3. Shell Programming and Scripting

Make multiple files of equal length

I have 150 files with 4 columns each but variable row lengths that I need to combine by column. I do not have any common column. I want to use "paste " command in unix to do it but before that I have to get all my files to be of equal length. Is there a way using awk or sed to fill up n no. of... (7 Replies)
Discussion started by: manishabh
7 Replies

4. Shell Programming and Scripting

How to make this run in multiple threads

Hi, I have a list of URLs in a csv file which I'm checking for page status. It just prints the URL and the status as output. This works perfectly fine. I'm looking to run this in multiple threads to make this process faster. I'm pretty new to Perl and I managed to complete this. It would be... (9 Replies)
Discussion started by: kzenthil
9 Replies

5. Solaris

Solaris Volume Manager: Break Mirror and use to make a 2nd machine

Hello: I have a machine built on 2 drives and mirrors have been created to a second set of 2 drives (4 total) on the same platform. SVM. Might anyone provide guidance to create a second machine by breaking the mirrored set, moving the two mirrors (2) to another machine (same platform type),... (1 Reply)
Discussion started by: 4dailyrunner
1 Replies

6. UNIX for Dummies Questions & Answers

sed command, make multiple lines into one

:confused:Hi, I'm relativley new at unix so am having difficulties at the most basic of areas. I am trying using sed to make multiple lines into one line. For example, i would like: Mary had a little lamb to look like this Maryhadalittlelamb so far i have tried sed... (1 Reply)
Discussion started by: cavanac2
1 Replies

7. Solaris

What is mirror and sub mirror in RAID -1 SVM

Hi , I am new to SVM .when i try to learn RAID 1 , first they are creating two RAID 0 strips through metainit d51 1 1 c0t0d0s2 metainit d52 1 1 c1t0d0s2 In the next step metainit d50 -m d51 d50: Mirror is setup next step is metaattach d50 d52 d50 : submirror d52 is... (7 Replies)
Discussion started by: vr_mari
7 Replies

8. Solaris

ZFS Mirror versus Hardware Mirror

I've looked a little but haven't found a solid answer, assuming there is one. What's better, hardware mirroring or ZFS mirroring? Common practice for us was to use the raid controllers on the Sun x86 servers. Now we've been using ZFS mirroring since U6. Any performance difference? Any other... (3 Replies)
Discussion started by: Lespaul20
3 Replies

9. Shell Programming and Scripting

Moving multiple files and renaming them on the fly

Hi All, Being new to scripting I am facing a new situation. We have an application that generates a file lets say dumpfile for each user under the users home directory when they execute the application. This is quite a huge file and imagine having that for over 40 users on a daily basis. The... (1 Reply)
Discussion started by: daemongk
1 Replies

10. Shell Programming and Scripting

How to make the same change in multiple shell scripts?

I would like to make the same change in multiple shell script files and would like to know if anyone can be of some help? I would appreciate it. (4 Replies)
Discussion started by: rdakhan
4 Replies
Login or Register to Ask a Question
MONBOOB(1)						      General Commands Manual							MONBOOB(1)

NAME
monboob - Daemon allowing to regularly check for new messages on various websites SYNOPSIS
monboob [-dqv] [-b backends] [-cnfs] [command [arguments..]] monboob [--help] [--version] DESCRIPTION
Daemon allowing to regularly check for new messages on various websites, and send an email for each message, and post a reply to a message on a website. Supported websites: * aum ("Adopte un Mec" French dating website) * bnporc (BNP Paribas French bank website) * bouygues (Bouygues Telecom French mobile phone provider) * dlfp (Da Linux French Page news website) * ecrans (Ecrans French news website) * fourchan (4chan image board) * hds (Histoires de Sexe French erotic novels) * inrocks (Les Inrocks French news website) * lefigaro (Le Figaro French newspaper website) * minutes20 (2 Minutes French newspaper website) * newsfeed (Loads RSS and Atom feeds from any website) * okc (OkCupid dating website) * orange (Orange French mobile phone provider) * phpbb (phpBB forum) * presseurop (Presseurop website) * sfr (SFR French mobile phone provider) * taz (Taz newspaper website) MONBOOB COMMANDS
once Send mails only once, then exit. post Pipe with a mail to post message. run Run the fetching daemon. OPTIONS
--version show program's version number and exit -h, --help show this help message and exit -b BACKENDS, --backends=BACKENDS what backend(s) to enable (comma separated) MONBOOB OPTIONS
-S SMTPD, --smtpd=SMTPD run a fake smtpd server and set the port LOGGING OPTIONS
-d, --debug display debug messages -q, --quiet display only error messages -v, --verbose display info messages --logging-file=LOGGING_FILE file to save logs -a, --save-responses save every response RESULTS OPTIONS
-c CONDITION, --condition=CONDITION filter result items to display given a boolean expression -n COUNT, --count=COUNT get a maximum number of results (all backends merged) -s SELECT, --select=SELECT select result item keys to display (comma separated) FORMATTING OPTIONS
-f FORMATTER, --formatter=FORMATTER select output formatter (csv, htmltable, multiline, simple, table, webkit) --no-header do not display header --no-keys do not display item keys -O OUTFILE, --outfile=OUTFILE file to export result COPYRIGHT
Copyright(C) 2010-2011 Romain Bignon For full COPYRIGHT see COPYING file with weboob package. FILES
"~/.config/weboob/backends" "~/.config/weboob/monboob" SEE ALSO
Home page: http://weboob.org/applications/monboob monboob 0.c 04 May 2012 MONBOOB(1)