Sponsored Content
Full Discussion: cat or cut to delimit csv?
Top Forums Shell Programming and Scripting cat or cut to delimit csv? Post 302312604 by newtony on Saturday 2nd of May 2009 06:24:13 AM
Old 05-02-2009
absolutely Smilie

here is one line of delimited info:

Code:
"product name"|"http://example.com/productlink.htm"|"http://example.com/productimage.jpg"|"price $80"

The code I am sending to mysql slurp: (this converts pipes to tabs) I was hoping not to do any conversion and just to use cat but it has no delimiter feature. This also doesnt get rid of the quotes enclosing each field.

Code:
cat myfile.csv | tr '|' '\t' | mysqlslurp --database=mydb --table=mytable

I cant seem to work out how to use the --mysqlimport options. It mentions on the mysqlslurp site that its possible to use them. I was thinking if i can use the options i can use --fields-terminated-by and --fields-enclosed-by etc

from mysqlslurp: http://search.cpan.org/dist/MySQL-Sl...ipt/mysqlslurp
Code:
  mysqlslurp 
      -D | --database                   Database    
      --table                           Table
      [--tmp | --tmpdir | --temp ]      Temporary Directory

      [ --usage ]                       Prints this synopsis
      [ --man ]                         Prints full man page

      [options passed to mysqlimport] 
      ( see mysqlimport --help for a list of options )


  Example: 
    cat file | mysqlslurp --database=my_db --table=my_table

I have a huge amount of these files (millions), so i was hoping to do it all with cat with as little text operation as possible from awk or perl etc

I guess I am looking for the fastest method to upload this to the db.

Cheers for any help Smilie

Last edited by newtony; 05-02-2009 at 07:34 AM..
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Difference between cat , cat > , cat >> and touch !!!

Hi Can anybody tell the difference between Difference between cat , cat > , cat >> and touch command in UNIX? Thanks (6 Replies)
Discussion started by: skyineyes
6 Replies

2. Shell Programming and Scripting

How to delimit a flat file with records with SED

Hi gurus, hoping someone can help with a sed line that can do the following... I have a flat file with about 1000 records, but in order to import into openoffice spreadsheet, I need to create a delimited file. I'd like to do 2 things with the SED command: 1- add a pipe character "|" at the end... (4 Replies)
Discussion started by: RogCor
4 Replies

3. Shell Programming and Scripting

cat in the command line doesn't match cat in the script

Hello, So I sorted my file as I was supposed to: sort -n -r -k 2 -k 1 file1 | uniq > file2 and when I wrote > cat file2 in the command line, I got what I was expecting, but in the script itself ... sort -n -r -k 2 -k 1 averages | uniq > temp cat file2 It wrote a whole... (21 Replies)
Discussion started by: shira
21 Replies

4. Solaris

how to delimit a colon

I want to know how to delimit a colon in a file name while scp ing from one host to other host. I used forward slash(\) but that is not working. (1 Reply)
Discussion started by: rogerben
1 Replies

5. UNIX for Advanced & Expert Users

Delimit the Folder Size come under Webapps dir

Hi all, Great thanks to all for support till today..today i came here for 1 new issue :-( in our organization we are developing a job portal web application for a client. using Apache-tomcat we are hosting this application, now i need to delimit the applications directory comer under the webapps... (1 Reply)
Discussion started by: anishkumarv
1 Replies

6. Shell Programming and Scripting

CSV with commas in field values, remove duplicates, cut columns

Hi Description of input file I have: ------------------------- 1) CSV with double quotes for string fields. 2) Some string fields have Comma as part of field value. 3) Have Duplicate lines 4) Have 200 columns/fields 5) File size is more than 10GB Description of output file I need:... (4 Replies)
Discussion started by: krishnix
4 Replies

7. Shell Programming and Scripting

Perl script to delimit size of strings

Hello, I have a huge file of over 2,00,00,00 strings in UTF8 format. I have managed to write a script in Perl which sorts them neatly as per their Unicode ranges. However I am now stuck with a script which will pipe out all strings between 3 and 20 letters/characters. I am not very good at... (2 Replies)
Discussion started by: gimley
2 Replies

8. UNIX for Dummies Questions & Answers

Delimit the out of wc -l command.

Hi All, Greetings !!! :) Am having issues with wc -l command. My requirement is to delimit the output with comma seperated values and use that file as input in informatica. wc -l *.ksh Output : 16 espress_raise.ksh 173 espresso_monitoring.ksh 189 total ... (6 Replies)
Discussion started by: srivignesh.kn
6 Replies

9. Shell Programming and Scripting

How to remove space and delimit a csv file?

Hi All , I am facing a small challenge i need unix command to remove space as well as replace "|" to "," in a csv file . original file : A | B | c | D E | F | G | H I | J | K | L P | M | N | O Expected o/p: A,B,c,D E,F,G,H I,J,K,L P,M,N,O (4 Replies)
Discussion started by: Sweety532
4 Replies
mysql(1)							  MySQL database							  mysql(1)

NAME
mysql_fix_privilege_tables - Fixes MySQL privilege tables. SYNOPSIS
mysql_fix_privilege_tables [mysql_root_password] DESCRIPTION
This scripts updates the mysql.user, mysql.db, mysql.host and the mysql.func tables to MySQL 3.22.14 and above. This is needed if you want to use the new GRANT functions, CREATE AGGREGATE FUNCTION or want to use the more secure passwords in 3.23 If you get 'Access denied' errors, run the script again and give the MySQL root user password as an argument. SEE ALSO
isamchk(1), isamlog(1), mysql(1), mysqlaccess(1), mysqladmin(1), mysqld(1), mysqld_multi(1), mysqld_safe(1), mysqldump(1), mysqlshow(1), mysql_zap(1), perror(1), replace(1) For more information please refer to the MySQL reference manual, which may already be installed locally and which is also available online at http://www.mysql.com/doc/en/ BUGS
Please refer to http://bugs.mysql.com/ to report bugs. AUTHOR
This manpage was written by Christian Hammers <ch@debian.org>. MySQL is available at http://www.mysql.com/. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +--------------------+------------------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +--------------------+------------------------------------+ |Availability | SUNWmysqlr, SUNWmysqlu, SUNWmysqlt | +--------------------+------------------------------------+ |Interface Stability | External | +--------------------+------------------------------------+ NOTES
Source for mysql is available on http://opensolaris.org. MySQL 4.0 17 March 2003 mysql(1)
All times are GMT -4. The time now is 08:42 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy