Sponsored Content
Full Discussion: cat or cut to delimit csv?
Top Forums Shell Programming and Scripting cat or cut to delimit csv? Post 302312604 by newtony on Saturday 2nd of May 2009 06:24:13 AM
Old 05-02-2009
absolutely Smilie

here is one line of delimited info:

Code:
"product name"|"http://example.com/productlink.htm"|"http://example.com/productimage.jpg"|"price $80"

The code I am sending to mysql slurp: (this converts pipes to tabs) I was hoping not to do any conversion and just to use cat but it has no delimiter feature. This also doesnt get rid of the quotes enclosing each field.

Code:
cat myfile.csv | tr '|' '\t' | mysqlslurp --database=mydb --table=mytable

I cant seem to work out how to use the --mysqlimport options. It mentions on the mysqlslurp site that its possible to use them. I was thinking if i can use the options i can use --fields-terminated-by and --fields-enclosed-by etc

from mysqlslurp: http://search.cpan.org/dist/MySQL-Sl...ipt/mysqlslurp
Code:
  mysqlslurp 
      -D | --database                   Database    
      --table                           Table
      [--tmp | --tmpdir | --temp ]      Temporary Directory

      [ --usage ]                       Prints this synopsis
      [ --man ]                         Prints full man page

      [options passed to mysqlimport] 
      ( see mysqlimport --help for a list of options )


  Example: 
    cat file | mysqlslurp --database=my_db --table=my_table

I have a huge amount of these files (millions), so i was hoping to do it all with cat with as little text operation as possible from awk or perl etc

I guess I am looking for the fastest method to upload this to the db.

Cheers for any help Smilie

Last edited by newtony; 05-02-2009 at 07:34 AM..
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Difference between cat , cat > , cat >> and touch !!!

Hi Can anybody tell the difference between Difference between cat , cat > , cat >> and touch command in UNIX? Thanks (6 Replies)
Discussion started by: skyineyes
6 Replies

2. Shell Programming and Scripting

How to delimit a flat file with records with SED

Hi gurus, hoping someone can help with a sed line that can do the following... I have a flat file with about 1000 records, but in order to import into openoffice spreadsheet, I need to create a delimited file. I'd like to do 2 things with the SED command: 1- add a pipe character "|" at the end... (4 Replies)
Discussion started by: RogCor
4 Replies

3. Shell Programming and Scripting

cat in the command line doesn't match cat in the script

Hello, So I sorted my file as I was supposed to: sort -n -r -k 2 -k 1 file1 | uniq > file2 and when I wrote > cat file2 in the command line, I got what I was expecting, but in the script itself ... sort -n -r -k 2 -k 1 averages | uniq > temp cat file2 It wrote a whole... (21 Replies)
Discussion started by: shira
21 Replies

4. Solaris

how to delimit a colon

I want to know how to delimit a colon in a file name while scp ing from one host to other host. I used forward slash(\) but that is not working. (1 Reply)
Discussion started by: rogerben
1 Replies

5. UNIX for Advanced & Expert Users

Delimit the Folder Size come under Webapps dir

Hi all, Great thanks to all for support till today..today i came here for 1 new issue :-( in our organization we are developing a job portal web application for a client. using Apache-tomcat we are hosting this application, now i need to delimit the applications directory comer under the webapps... (1 Reply)
Discussion started by: anishkumarv
1 Replies

6. Shell Programming and Scripting

CSV with commas in field values, remove duplicates, cut columns

Hi Description of input file I have: ------------------------- 1) CSV with double quotes for string fields. 2) Some string fields have Comma as part of field value. 3) Have Duplicate lines 4) Have 200 columns/fields 5) File size is more than 10GB Description of output file I need:... (4 Replies)
Discussion started by: krishnix
4 Replies

7. Shell Programming and Scripting

Perl script to delimit size of strings

Hello, I have a huge file of over 2,00,00,00 strings in UTF8 format. I have managed to write a script in Perl which sorts them neatly as per their Unicode ranges. However I am now stuck with a script which will pipe out all strings between 3 and 20 letters/characters. I am not very good at... (2 Replies)
Discussion started by: gimley
2 Replies

8. UNIX for Dummies Questions & Answers

Delimit the out of wc -l command.

Hi All, Greetings !!! :) Am having issues with wc -l command. My requirement is to delimit the output with comma seperated values and use that file as input in informatica. wc -l *.ksh Output : 16 espress_raise.ksh 173 espresso_monitoring.ksh 189 total ... (6 Replies)
Discussion started by: srivignesh.kn
6 Replies

9. Shell Programming and Scripting

How to remove space and delimit a csv file?

Hi All , I am facing a small challenge i need unix command to remove space as well as replace "|" to "," in a csv file . original file : A | B | c | D E | F | G | H I | J | K | L P | M | N | O Expected o/p: A,B,c,D E,F,G,H I,J,K,L P,M,N,O (4 Replies)
Discussion started by: Sweety532
4 Replies
MySQL::Diff::Database(3pm)				User Contributed Perl Documentation				MySQL::Diff::Database(3pm)

NAME
MySQL::Diff::Database - Database Definition Class SYNOPSIS
use MySQL::Diff::Database; my $db = MySQL::Diff::Database->new(%options); my $source = $db->source_type(); my $summary = $db->summary(); my $name = $db->name(); my @tables = $db->tables(); my $table_def = $db->table_by_name($table); my @dbs = MySQL::Diff::Database::available_dbs(); DESCRIPTION
Parses a database definition into component parts. METHODS
Constructor new( %options ) Instantiate the objects, providing the command line options for database access and process requirements. Public Methods o source_type() Returns 'file' if the data source is a text file, and 'db' if connected directly to a database. o summary() Provides a summary of the database. o name() Returns the name of the database. o tables() Returns a list of tables for the current database. o table_by_name( $name ) Returns the table definition (see MySQL::Diff::Table) for the given table. FUNCTIONS
Public Functions o available_dbs() Returns a list of the available databases. Note that is used as a function call, not a method call. COPYRIGHT AND LICENSE
Copyright (c) 2000-2011 Adam Spiers. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO
mysqldiff, MySQL::Diff, MySQL::Diff::Table, MySQL::Diff::Utils AUTHOR
Adam Spiers <mysqldiff@adamspiers.org> perl v5.14.2 2012-04-06 MySQL::Diff::Database(3pm)
All times are GMT -4. The time now is 07:56 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy