Sponsored Content
Full Discussion: sh and MySQL LOAD DATA
Top Forums Shell Programming and Scripting sh and MySQL LOAD DATA Post 302296472 by worchyld on Wednesday 11th of March 2009 07:02:52 AM
Old 03-11-2009
sh and MySQL LOAD DATA

I have a csv file in a folder that is roughly 500,000 rows long.

Rather than using PHP, I would like to use SH to run a MYSQL LOAD DATA command to load the data in, as I think it would be much faster and would not cause any memory problems associated with PHP.

But the problem is, I am not sure how to combine sh and my MySQL LOAD DATA command as it's on several lines and is very, very long.

The following is a sample of the actual load data I use.

How do I make sh to connect to mysql, and run this following command?

Any help on this would be great!

Code:
LOAD DATA LOCAL INFILE '/path/to/traveldata/traveldata.csv' 
INTO TABLE `offers` 
FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 2 LINES
(
OfferProcessType,
TradingName,
OfferLabelDesc,
DestinationName,
DepAirportCode,
TelephoneNumber,
ReturnAirportCode,
ArrivalAirportCode,
ABTA,
ATOL,
OtherBonding,
BoardTypesShort,
AccomodationShort,
AllocationShort,
OperatorShort,
@DepartureDate,
Price,
ShareBasisAdult,
BulkLoaderRef,
Email,
PublicNotes,
CCC,
TOD,
DCC,
AMEX,
CDW,
BookingFee,
AccomodationName,
Duration,
ShareBasisChild,
Rating,
OfferLink,
AccomodationRef,
Transfers,
OperatorsRating,
QuoteRef,
Url,
OutDepTime,
OutArrTime,
InDepTime,
InArrTime
)
SET
DepartureDate = (str_to_date(@DepartureDate, '%d-%M-%y'))
;

 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need help in wrting Load Script for a Load-Resume type of load.

hi all need your help. I am wrting a script that will load data into the table. then on another load will append the data into the existing table. Regards Ankit (1 Reply)
Discussion started by: ankitgupta
1 Replies

2. Programming

Load text file into a MySQL field

Hello, maybe this post is offtopic, sorry for the inconveniencies. I found examples about how to populate different fields from a text file (with MySQL, the LOAD DATA INFILE sentece), but not about how to load a complete plain text file into a concrete database field. Could you please tell me if... (1 Reply)
Discussion started by: aristegui
1 Replies

3. Shell Programming and Scripting

Automatically Load data from all files in directory

I'm new in Unix shell scripting and i need someone to help me to make Script that run all time watching my directory that files uploaded to it via FTP (/mydir/incoming_files), if any files exists in it then (if many files exists, then sort files and load them ascending) it‘ll checks the size of the... (1 Reply)
Discussion started by: m_fighter
1 Replies

4. Shell Programming and Scripting

How to load different type of data in a file to two arrays

Hi, I have tried to find some sort of previous similar thread on this but not quite close to what I want to achieve. Basically I have two class of data in my file..e.g 1,1,1,1,1,2,yes 1,2,3,4,5,5,yes 2,3,4,5,5,5,no 1,2,3,4,4,2,no 1,1,3,4,5,2,no I wanted to read the "yes" entry to an... (5 Replies)
Discussion started by: ahjiefreak
5 Replies

5. Shell Programming and Scripting

executing mysql load statement from shell script

Hi, I have a piece of shell script which will connect to mysql database and execute a load statement(which will load datas in a file to the database table).The code is working and the data is in the tables. Now my requirement is, i need to grab the output from the load statement... (4 Replies)
Discussion started by: DILEEP410
4 Replies

6. Web Development

script to load data from csv file

hello i want a script to load the data line by line from a csv file into a mysql table (3 Replies)
Discussion started by: srpa01red
3 Replies

7. Shell Programming and Scripting

Load data from a flat file to oracle.

I have a flat file with records like Header 123 James Williams Finance2000 124 Pete Pete HR 1500 125 PatrickHeather Engg 3000 Footer The structure is: Eno:4 characters Name:8 characters Surname : 9 characters Dept:7 characters Sal:4characters These are sample... (1 Reply)
Discussion started by: Shivdatta
1 Replies

8. UNIX for Dummies Questions & Answers

Load UNIX data into excel sheet

Hi, i have some data in a temporary file in Unix (the data is taken from the result of an SQL query). Now i want to dump that data into an excel sheet. How to do that. Someone please advise. Thanks Regards, Vinit (3 Replies)
Discussion started by: vinit raj
3 Replies

9. Shell Programming and Scripting

Parallel sqlldr to load data.

I am using SQLLDR to load data in DB.For parallel loading I'm using nohup command. The requirement is: I have different files within a directories. Ex: 1) Dir/folder_A/AE.txt 2) Dir/folder_A/DM.txt 3) Dir/folder_B/CM.txt I need to loop through directories and load the data... (1 Reply)
Discussion started by: Pratiksha Mehra
1 Replies
NDB_SHOW_TABLES 					       MySQL Database System						   NDB_SHOW_TABLES

NAME
ndb_show_tables - display list of NDB tables SYNOPSIS
ndb_show_tables [options] DESCRIPTION
ndb_show_tables displays a list of all NDB database objects in the cluster. By default, this includes not only both user-created tables and NDB system tables, but NDB-specific indexes, internal triggers, and MySQL Cluster Disk Data objects as well. The following table includes options that are specific to the MySQL Cluster native backup restoration program ndb_show_tables. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_show_tables), see Options Common to MySQL Cluster Programs(1). Table 17.26. ndb_show_tables Options and Variables: MySQL Cluster NDB 7.2 +--------------------------+--------------------------------------+-------------------------------------+ |Format | Description | Added / Removed | +--------------------------+--------------------------------------+-------------------------------------+ | | Specifies the database in which the | | | --database=string, | table is found | All MySQL 5.5 based releases | | | | | | -d | | | +--------------------------+--------------------------------------+-------------------------------------+ | | Number of times to repeat output | | | --loops=#, | | All MySQL 5.5 based releases | | | | | | -l | | | +--------------------------+--------------------------------------+-------------------------------------+ | | Limit output to objects of this type | | | --type=#, | | All MySQL 5.5 based releases | | | | | | -t | | | +--------------------------+--------------------------------------+-------------------------------------+ | | Do not qualify table names | | | --unqualified, | | All MySQL 5.5 based releases | | | | | | -u | | | +--------------------------+--------------------------------------+-------------------------------------+ | | Return output suitable for MySQL | | | --parsable, | LOAD DATA INFILE statement | All MySQL 5.5 based releases | | | | | | -p | | | +--------------------------+--------------------------------------+-------------------------------------+ | | Show table temporary flag | | | --show-temp-status | | All MySQL 5.5 based releases | +--------------------------+--------------------------------------+-------------------------------------+ Usage ndb_show_tables [-c connect_string] o --database, -d Specifies the name of the database in which the tables are found. o --loops, -l Specifies the number of times the utility should execute. This is 1 when this option is not specified, but if you do use the option, you must supply an integer argument for it. o --parsable, -p Using this option causes the output to be in a format suitable for use with LOAD DATA INFILE. o --show-temp-status If specified, this causes temporary tables to be displayed. o --type, -t Can be used to restrict the output to one type of object, specified by an integer type code as shown here: o 1: System table o 2: User-created table o 3: Unique hash index Any other value causes all NDB database objects to be listed (the default). o --unqualified, -u If specified, this causes unqualified object names to be displayed. Note Only user-created MySQL Cluster tables may be accessed from MySQL; system tables such as SYSTAB_0 are not visible to mysqld. However, you can examine the contents of system tables using NDB API applications such as ndb_select_all (see ndb_select_all(1)). COPYRIGHT
Copyright (C) 1997, 2014, Oracle and/or its affiliates. All rights reserved. This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. This documentation is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with the program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA or see http://www.gnu.org/licenses/. SEE ALSO
For more information, please refer to the MySQL Reference Manual, which may already be installed locally and which is also available online at http://dev.mysql.com/doc/. AUTHOR
Oracle Corporation (http://dev.mysql.com/). MySQL 5.5 01/30/2014 NDB_SHOW_TABLES
All times are GMT -4. The time now is 03:08 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy