Sponsored Content
Top Forums Shell Programming and Scripting CSV to SQL insert: Awk for strings with multiple lines in csv Post 302581242 by Corona688 on Monday 12th of December 2011 12:32:50 PM
Old 12-12-2011
How should I tell apart the files with headers from the files without?

Put this in a text file:

Code:
BEGIN { FS="|"; OFS="," }
# Append 'short' lines into one longer line
NF<5 { if(S){ $0="\\n" $0; } S=S $0; $0=S }
# When it fially ends in "NEWLINE", we know we have an entire record
$NF == "NEWLINE" {
        for(N=1; N<=NF; N++)
        # Add single quotes to things lacking double quotes
        if(substr($N,1,1) != "\"")
                $N = "'" $N "'";

        print "insert into MyTBI values("$1,$2,$3,$4,$5");";
        S="";
}

and use it like this:

Code:
$ awk -f myawk.txt < data
insert into MyTBI values('Column1','Column2','Column3','Column4','NEWLINE');
insert into MyTBI values('Address Type','some descriptive text',"Mailing\nPhysical\nShipping",'Unchanged from Version 1','NEWLINE');
insert into MyTBI values('Street Number/Name','some description for rec 2','None','some text here','NEWLINE');
insert into MyTBI values('City','description for rec 3','None','some text here too','NEWLINE');
insert into MyTBI values('Zip','description for rec 4','text here','sdfla sdflsdhlsdfsk','NEWLINE');
insert into MyTBI values('Record 5','description',"Main - Main record\nAdministrative - Administrative phone number\nHR - HR phone number\nServices - Services line\nFax - Facsimile phone number\nOther",'Unchanged from Version 1','NEWLINE');
insert into MyTBI values('Grades Offered','description here',"IT - Infant/toddler\nPR - Preschool\nPK - Prekindergarten\nTK - Transitional Kindergarten\nKG - Kindergarten\n01 - First grade\n02 - Second grade\nUG - Ungraded\nOther",'Unchanged from Version 1','NEWLINE');
$

Does | ever happen inside double-quotes? That won't be handled properly.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to (n)awk lines of CSV with certain number of fields?

I have a CSV file with a variable number of fields per record. How do I print lines of a certain number of fields only? Several permutations of the following (including the use of escape characters) have failed to retrieve the line I'm after (1,2,3,4)... $ cat myfile 1,2,3,4 1,2,3 $ # Print... (1 Reply)
Discussion started by: cs03dmj
1 Replies

2. Shell Programming and Scripting

Matching lines across multiple csv files and merging a particular field

I have about 20 CSV's that all look like this: "","","","","","","","","","","","","","","",""What I've been told I need to produce is the exact same thing, but with each file now containing the start_code from every other file where the email matches. It doesn't matter if any of the other... (1 Reply)
Discussion started by: Demosthenes
1 Replies

3. Shell Programming and Scripting

insert data into specific lines of a CSV

So I work in a 1 to 1 laptop deployment and sometimes we need to mass order parts. The vendor will send us a text file and we have to manually input serial numbers. Well I have a full blown web based inventory system which I can pull serial number reports from. I then have to input the part... (4 Replies)
Discussion started by: tlarkin
4 Replies

4. Shell Programming and Scripting

how to give multiple csv files as input in awk

Hi All, I am new to shell scripting..My problem is i want to give multiple csv files as input to awk script and process the data into one file.. My input file is File1 File2 File3 Product Location Period SalesPrice A x 8/11/2010 ... (7 Replies)
Discussion started by: kvth
7 Replies

5. Shell Programming and Scripting

Extract strings from multiple lines into one csv file

Hi all, Please go through my requirement. I have a log file in the location /opt/WebSphere61/AppServer/profiles/EMQbatchprofile/logs/EMQbatch This file contains the follwing pattern data <af type="tenured" id="42" timestamp="May 14 13:44:13 2011" intervalms="955.624"> <minimum... (8 Replies)
Discussion started by: satish.vampire
8 Replies

6. Shell Programming and Scripting

Perl search csv fileA where two strings exist on another csv fileB

Hi I have two csv files, with the following formats: FileA.log: Application, This occured blah Application, That occured blah Application, Also this AnotherLog, Bob did this AnotherLog, Dave did that FileB.log: Uk, London, Application, datetime, LaterDateTime, Today it had'nt... (8 Replies)
Discussion started by: PerlNewbRP
8 Replies

7. Shell Programming and Scripting

awk read column csv and search in other csv

hi, someone to know how can i read a specific column of csv file and search the value in other csv columns if exist the value in the second csv copy entire row with all field in a new csv file. i suppose that its possible using awk but i m not expertise thanks in advance (8 Replies)
Discussion started by: giankan
8 Replies

8. UNIX for Dummies Questions & Answers

Need help combining txt files w/ multiple lines into csv single cell - also need data merge

:confused:Hello -- i just joined the forums. I am a complete noob -- only about 1 week into learning how to program anything... and starting with linux. I am working in Linux terminal. I have a folder with a bunch of txt files. Each file has several lines of html code. I want to combine... (2 Replies)
Discussion started by: jetsetter
2 Replies

9. Shell Programming and Scripting

Multiple carriage returns within quotation marks causing new lines in csv

There is a closed thread called "carriage returns within quotation marks causing new lines in csv" that I am unable to post to, so I am starting a new thread. The awk solution worked perfectly in most cases. We have some cases where there are multiple carriage returns within a single quoted... (9 Replies)
Discussion started by: Mary Roberts
9 Replies

10. Shell Programming and Scripting

awk - CSV file - field with single or multiple spaces

Hi, In a csv file, I want to select records where first column has zero or multiple spaces. Eg: abc.csv ,123,a ,22,b ,11,c a,11,d So output should be: ,123,a ,22,b ,11,c Please advise (5 Replies)
Discussion started by: vegasluxor
5 Replies
pocount(1)						      Translate Toolkit 1.3.0							pocount(1)

NAME
pocount - Produces word counts and other statistics from a PO file. SYNOPSIS
pocount [--csv] [directory|file(s)] DESCRIPTION
pocount will count the number of strings and words in a PO file. If no files or directories argument are provided, pocount will recurse through all files from the current directory. Otherwise, it will recurse and count all files in the specified directory or in the specified PO files. OPTIONS
--csv changes the output format to CSV (Comma Seperated Values) for import into a spreadsheet. OUTPUT
In normal mode the following output is given: avmedia/source/viewer.po type strings words (source) words (translation) translated: 1 3 3 fuzzy: 0 0 n/a untranslated: 4 22 n/a Total: 5 25 3 review 1 3 n/a In CSV mode the following outut is shown: Filename, Translated Messages, Translated Source Words, Translated Target Words, Fuzzy Messages, Fuzzy Source Words, Untranslated Mes- sages, Untranslated Source Words, Review Messages, Review Source Words avmedia/source/viewer.po, 1, 3, 3, 0, 0, 4, 22, 1, 3 Totals are not provided in CSV mode. In normal mode a grand total and file count is provided if the number of files is greater than one. BUGS
There are some miscounts related to word breaks. Translate Toolkit 1.3.0 pocount(1)
All times are GMT -4. The time now is 09:32 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy