Sponsored Content
Full Discussion: cat or cut to delimit csv?
Top Forums Shell Programming and Scripting cat or cut to delimit csv? Post 302312604 by newtony on Saturday 2nd of May 2009 06:24:13 AM
Old 05-02-2009
absolutely Smilie

here is one line of delimited info:

Code:
"product name"|"http://example.com/productlink.htm"|"http://example.com/productimage.jpg"|"price $80"

The code I am sending to mysql slurp: (this converts pipes to tabs) I was hoping not to do any conversion and just to use cat but it has no delimiter feature. This also doesnt get rid of the quotes enclosing each field.

Code:
cat myfile.csv | tr '|' '\t' | mysqlslurp --database=mydb --table=mytable

I cant seem to work out how to use the --mysqlimport options. It mentions on the mysqlslurp site that its possible to use them. I was thinking if i can use the options i can use --fields-terminated-by and --fields-enclosed-by etc

from mysqlslurp: http://search.cpan.org/dist/MySQL-Sl...ipt/mysqlslurp
Code:
  mysqlslurp 
      -D | --database                   Database    
      --table                           Table
      [--tmp | --tmpdir | --temp ]      Temporary Directory

      [ --usage ]                       Prints this synopsis
      [ --man ]                         Prints full man page

      [options passed to mysqlimport] 
      ( see mysqlimport --help for a list of options )


  Example: 
    cat file | mysqlslurp --database=my_db --table=my_table

I have a huge amount of these files (millions), so i was hoping to do it all with cat with as little text operation as possible from awk or perl etc

I guess I am looking for the fastest method to upload this to the db.

Cheers for any help Smilie

Last edited by newtony; 05-02-2009 at 07:34 AM..
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Difference between cat , cat > , cat >> and touch !!!

Hi Can anybody tell the difference between Difference between cat , cat > , cat >> and touch command in UNIX? Thanks (6 Replies)
Discussion started by: skyineyes
6 Replies

2. Shell Programming and Scripting

How to delimit a flat file with records with SED

Hi gurus, hoping someone can help with a sed line that can do the following... I have a flat file with about 1000 records, but in order to import into openoffice spreadsheet, I need to create a delimited file. I'd like to do 2 things with the SED command: 1- add a pipe character "|" at the end... (4 Replies)
Discussion started by: RogCor
4 Replies

3. Shell Programming and Scripting

cat in the command line doesn't match cat in the script

Hello, So I sorted my file as I was supposed to: sort -n -r -k 2 -k 1 file1 | uniq > file2 and when I wrote > cat file2 in the command line, I got what I was expecting, but in the script itself ... sort -n -r -k 2 -k 1 averages | uniq > temp cat file2 It wrote a whole... (21 Replies)
Discussion started by: shira
21 Replies

4. Solaris

how to delimit a colon

I want to know how to delimit a colon in a file name while scp ing from one host to other host. I used forward slash(\) but that is not working. (1 Reply)
Discussion started by: rogerben
1 Replies

5. UNIX for Advanced & Expert Users

Delimit the Folder Size come under Webapps dir

Hi all, Great thanks to all for support till today..today i came here for 1 new issue :-( in our organization we are developing a job portal web application for a client. using Apache-tomcat we are hosting this application, now i need to delimit the applications directory comer under the webapps... (1 Reply)
Discussion started by: anishkumarv
1 Replies

6. Shell Programming and Scripting

CSV with commas in field values, remove duplicates, cut columns

Hi Description of input file I have: ------------------------- 1) CSV with double quotes for string fields. 2) Some string fields have Comma as part of field value. 3) Have Duplicate lines 4) Have 200 columns/fields 5) File size is more than 10GB Description of output file I need:... (4 Replies)
Discussion started by: krishnix
4 Replies

7. Shell Programming and Scripting

Perl script to delimit size of strings

Hello, I have a huge file of over 2,00,00,00 strings in UTF8 format. I have managed to write a script in Perl which sorts them neatly as per their Unicode ranges. However I am now stuck with a script which will pipe out all strings between 3 and 20 letters/characters. I am not very good at... (2 Replies)
Discussion started by: gimley
2 Replies

8. UNIX for Dummies Questions & Answers

Delimit the out of wc -l command.

Hi All, Greetings !!! :) Am having issues with wc -l command. My requirement is to delimit the output with comma seperated values and use that file as input in informatica. wc -l *.ksh Output : 16 espress_raise.ksh 173 espresso_monitoring.ksh 189 total ... (6 Replies)
Discussion started by: srivignesh.kn
6 Replies

9. Shell Programming and Scripting

How to remove space and delimit a csv file?

Hi All , I am facing a small challenge i need unix command to remove space as well as replace "|" to "," in a csv file . original file : A | B | c | D E | F | G | H I | J | K | L P | M | N | O Expected o/p: A,B,c,D E,F,G,H I,J,K,L P,M,N,O (4 Replies)
Discussion started by: Sweety532
4 Replies
MYSQL_TZINFO_TO_S(1)					       MySQL Database System					      MYSQL_TZINFO_TO_S(1)

NAME
mysql_tzinfo_to_sql - load the time zone tables SYNOPSIS
mysql_tzinfo_to_sql arguments DESCRIPTION
The mysql_tzinfo_to_sql program loads the time zone tables in the mysql database. It is used on systems that have a zoneinfo database (the set of files describing time zones). Examples of such systems are Linux, FreeBSD, Solaris, and Mac OS X. One likely location for these files is the /usr/share/zoneinfo directory (/usr/share/lib/zoneinfo on Solaris). If your system does not have a zoneinfo database, you can use the downloadable package described in Section 10.6, "MySQL Server Time Zone Support". mysql_tzinfo_to_sql can be invoked several ways: shell> mysql_tzinfo_to_sql tz_dir shell> mysql_tzinfo_to_sql tz_file tz_name shell> mysql_tzinfo_to_sql --leap tz_file For the first invocation syntax, pass the zoneinfo directory path name to mysql_tzinfo_to_sql and send the output into the mysql program. For example: shell> mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql mysql_tzinfo_to_sql reads your system's time zone files and generates SQL statements from them. mysql processes those statements to load the time zone tables. The second syntax causes mysql_tzinfo_to_sql to load a single time zone file tz_file that corresponds to a time zone name tz_name: shell> mysql_tzinfo_to_sql tz_file tz_name | mysql -u root mysql If your time zone needs to account for leap seconds, invoke mysql_tzinfo_to_sql using the third syntax, which initializes the leap second information. tz_file is the name of your time zone file: shell> mysql_tzinfo_to_sql --leap tz_file | mysql -u root mysql After running mysql_tzinfo_to_sql, it is best to restart the server so that it does not continue to use any previously cached time zone data. COPYRIGHT
Copyright (C) 1997, 2014, Oracle and/or its affiliates. All rights reserved. This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. This documentation is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with the program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA or see http://www.gnu.org/licenses/. SEE ALSO
For more information, please refer to the MySQL Reference Manual, which may already be installed locally and which is also available online at http://dev.mysql.com/doc/. AUTHOR
Oracle Corporation (http://dev.mysql.com/). MySQL 5.5 01/30/2014 MYSQL_TZINFO_TO_S(1)
All times are GMT -4. The time now is 07:15 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy