I have a list of 10 million page urls. I want those pages scraped and saved in the mysql database as raw html.
I own a Linux VPS server with 1GB RAM and WHM/cPanel.
I would like to scrape at least 100,000 urls in 24 hours.
So can anyone give me some sample shell scripting code? (4 Replies)
I am using an html form and a php upload script to upload files.
HTML form
<table width="500" border="0" align="center" cellpadding="0" cellspacing="1" bgcolor="#CCCCCC">
<tr>
<form action="upload_ac.php" method="post" enctype="multipart/form-data" name="form1" id="form1">
<td>
<table... (1 Reply)
Hi
I would like to convert standard online man pages from my solaris10 system into html form to publish it on my webpage.
How this can be done in Sol10 ?
thx for help. (2 Replies)
I am sure this is easy but I can't figure it out...
Here is the form.
<?php
$searchString = $_POST;
if (!isset($_POST))
?>
<html>
<head>
<title>Personal INFO</title>
</head>
<body>
<form method="post" action="search.php">
<input type="text" size="20" maxlength="20" name="search">... (1 Reply)
I have an HTML form that sends email to a large list of users one at a time by matching an email address in peoplesoft to their username. It works great, except that special characters are converted to %## format. Is there a library of these I can use to sed them back (yes this is a crappy UNIX... (1 Reply)
I wrote a script to automate user account verification against peoplesoft. Now I want to make it available to my peers via the web. It is running on Solaris.
I have the form written, but am not sure how to make it work. I think the form should call a perl cgi when submitted. The cgi should call... (7 Replies)
I am currently able to use the $QUERY_STRING variable and simply cut out what I need to be assigned as variables within the shell script. However, I've been able to use the "name" value assigned within the FORM(HTML) as a variable when I use perl. Why is it that ksh doesn't read the "name" in as... (1 Reply)
MYSQL_TZINFO_TO_S(1) MySQL Database System MYSQL_TZINFO_TO_S(1)NAME
mysql_tzinfo_to_sql - load the time zone tables
SYNOPSIS
mysql_tzinfo_to_sql arguments
DESCRIPTION
The mysql_tzinfo_to_sql program loads the time zone tables in the mysql database. It is used on systems that have a zoneinfo database (the
set of files describing time zones). Examples of such systems are Linux, FreeBSD, Solaris, and Mac OS X. One likely location for these
files is the /usr/share/zoneinfo directory (/usr/share/lib/zoneinfo on Solaris). If your system does not have a zoneinfo database, you can
use the downloadable package described in Section 10.6, "MySQL Server Time Zone Support".
mysql_tzinfo_to_sql can be invoked several ways:
shell> mysql_tzinfo_to_sql tz_dir
shell> mysql_tzinfo_to_sql tz_file tz_name
shell> mysql_tzinfo_to_sql --leap tz_file
For the first invocation syntax, pass the zoneinfo directory path name to mysql_tzinfo_to_sql and send the output into the mysql program.
For example:
shell> mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql
mysql_tzinfo_to_sql reads your system's time zone files and generates SQL statements from them. mysql processes those statements to load
the time zone tables.
The second syntax causes mysql_tzinfo_to_sql to load a single time zone file tz_file that corresponds to a time zone name tz_name:
shell> mysql_tzinfo_to_sql tz_file tz_name | mysql -u root mysql
If your time zone needs to account for leap seconds, invoke mysql_tzinfo_to_sql using the third syntax, which initializes the leap second
information. tz_file is the name of your time zone file:
shell> mysql_tzinfo_to_sql --leap tz_file | mysql -u root mysql
After running mysql_tzinfo_to_sql, it is best to restart the server so that it does not continue to use any previously cached time zone
data.
COPYRIGHT
Copyright (C) 1997, 2014, Oracle and/or its affiliates. All rights reserved.
This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as
published by the Free Software Foundation; version 2 of the License.
This documentation is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with the program; if not, write to the Free Software Foundation,
Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA or see http://www.gnu.org/licenses/.
SEE ALSO
For more information, please refer to the MySQL Reference Manual, which may already be installed locally and which is also available online
at http://dev.mysql.com/doc/.
AUTHOR
Oracle Corporation (http://dev.mysql.com/).
MySQL 5.5 01/30/2014 MYSQL_TZINFO_TO_S(1)