Sponsored Content
Top Forums Shell Programming and Scripting Wrap lines with awk to create SQL script Post 302531079 by agama on Wednesday 15th of June 2011 10:36:13 PM
Old 06-15-2011
Yep, it's possible. Here is one way:

Code:
#!/usr/bin/env ksh

# if you have a lot of these files, this method will
# prevent arg length errors to the for
find . -name "R*VER" -mtime +1 | while read fname
do
    ( grep "^err" $fname || echo "No error" ) | awk  -v base=${fname##*/} '
        {
            split( base, a, "~" );          # split filename into components
            printf( "%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n", base, $0, a[1], a[2], a[7], a[8], a[9], a[10] );
        }'
done

With some minor changes, it will generate your SQL (based on what you provided; I don't do SQL so I don't know if it's exactly what you need, but you'll get the drift):

Code:
#!/usr/bin/env ksh

# if you have a lot of these files, this method will
# prevent arg length errors to the for
find . -name "R*VER" -mtime +1 | while read fname
do
    ( grep "^err" $fname || echo "No error" ) | awk -v squote="'" -v base=${fname##*/} '
        {
            split( base, a, "~" );          # split filename into components
            printf( "INSERT INTO MYTABLE (COL1,COL2,COL3,COL4,COL5,COL6,COL7,COL8)\n" );
            str = sprintf( "VALUES (\"%s\",\"%s\",\"%s\",\"%s\",\"%s\",\"%s\",\"%s\",\"%s\")", base, $0, a[1], a[2], a[7], a[8], a[9], a[10] );
            gsub( "\"", squote, str );          # replace double quotes with singles
            print str;
        }'
done

The trick with the single quote, and substitution into str gets round the difficulty of using a single quote inside of an awk programme. It's possible, very messy, and makes for code that is difficult to maintain. There are other ways to do the quoting that are more efficient, but I think this is the most readable.

Other things that might not be too obvious:
${fname##*/} strips the leading path (./tmp/xxx/) from the filename returned by find.

The parens round this statement( grep "^err" $fname || echo "No error" ) cause the statement to be executed in a subshell. This is necessary to cause output from the grep and the echo to pipe properly into the awk.

Hope this helps.

Last edited by agama; 06-15-2011 at 11:37 PM.. Reason: clarification
This User Gave Thanks to agama For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Should I use sed/ grep/awk for wrap file?

Hi, This is my first time post a new thread. I have been trying to work on this for the past 2 days and could not find any good solution. I have 1 long long line ( EDI wrapped file) like below: NEW*SR*04411763447*279*278*Q~*ZR*AAV*SR*04511763460*SQ*21B37F04~HL*305*304*Q~K~SN1*1*1*SR*05511763461*... (6 Replies)
Discussion started by: vanda_25
6 Replies

2. Shell Programming and Scripting

Wrap Interactive Script

Does anyone know of a program to wrap an interactive script into an application..I tried using platypus but i want a utility to allow interactive scripts to be run in a stand-alone window to avoid .profile settings on multiple computers...platypus provides the option of a text window output but... (0 Replies)
Discussion started by: meskue
0 Replies

3. Programming

create a spool file based on values passed from korn shell to sql script

this is my issue. 4 parameters are passed from korn shell to sql script. parameter_1= varchar2 datatype or no value entered my user. parameter_2= number datatype or no value entered my user. parameter_3= number datatype or no value entered my user. parameter_4= number datatype or no... (5 Replies)
Discussion started by: megha2525
5 Replies

4. Shell Programming and Scripting

Create SQL DML insert statements from file using AWK or similar

Hi all. This is my first post on this forum. I've previously found great help in the huge knowledgebase that is here, but this time I have not been able to find a solution to my problem. I have a large text file that looks like this: typedef struct ABC_struct_nbr1_ { char attr1; /*... (0 Replies)
Discussion started by: Yagi Uda
0 Replies

5. UNIX for Advanced & Expert Users

SQL script with 86000 lines: new files with only 10000 lines (per file)

Hi this is my SQL script $ wc -l insert_into_customers.sql 85601 insert_into_customers.sqlI wish to cut this file into 9 files each 10000 lines (the last one less) $ wc -l insert_into_customers_00*.sql 10000 insert_into_customers_001.sql 10000 insert_into_customers_002.sql ... (1 Reply)
Discussion started by: slashdotweenie
1 Replies

6. UNIX for Dummies Questions & Answers

Bash does not wrap long lines correctly

Ksh is my default shell, but I want use the bash shell since its convenient to me. When I type a long command line in a terminal, it does not wrap to the next line when I reach the end of the line and it wraps onto the same line, overwriting my prompt and the rest of what I typed. $... (5 Replies)
Discussion started by: senthil.ak
5 Replies

7. Shell Programming and Scripting

Please help, need to create script to remove lines by date in file

Please Help (novice to PERL and SHELL scripting)…. Need to create a script which removes all lines in $filename = "cycle_calendar_ftp_out" older than current date – a variable which will be a number of days passed to script. For Ex it will look at the end date which is the last field (4) and... (2 Replies)
Discussion started by: m3pwr
2 Replies

8. Shell Programming and Scripting

Using awk and sed to modify a create sql script

Hi, I have a file which contains the following data claim_src|clm_id,typ_id pat_src|pat_id prov_src|prov_id,clm_id,prov_name The first field is table name and second field is primary keys of the table Now I have three files which contain ddl of each table. clam_src.sql... (4 Replies)
Discussion started by: wahi80
4 Replies

9. Shell Programming and Scripting

Perl one liner to wrap comment lines

Greetings fellow scripters. I find myself editing multiple files, sometimes with the same bits of information. My bash script, a changelog, and a plist file (OS X). Once I realized this, I thought why not script part of this process (and so it begins). In any case, I've solved several of the... (1 Reply)
Discussion started by: reid
1 Replies

10. Shell Programming and Scripting

How to create SQL statement out of data using shell script?

Table TAB1 contains following example data (its a tree sitting in table data format & its driven based CHILD & PARENT column pick the RULE condition to generate the below SQL: CHILD PARENT SS MID MNM VNM RULE FLG 1 ? S1 ? ? V1 rule004 I 2 1 S1 ? ? V1 0 Z 3 1 S1 ? ? V1 1 Z ... (6 Replies)
Discussion started by: gksenthilkumar
6 Replies
All times are GMT -4. The time now is 03:10 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy