Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Formation of sql files
Top Forums Shell Programming and Scripting Formation of sql files Post 302580448 by Corona688 on Thursday 8th of December 2011 02:08:55 PM
Old 12-08-2011
I'm not surprised... That's a dangerous use of backticks and useless use of cat. It works as a quick fix only but when you use it for anything big, as you've discovered, it has major problems. Like storing the entire file in memory while it's working on it, and possibly truncating the end if it's just too large.

It also produces 5 update statements when you only needed 1, and reopens the query file 5000 times to process 1000 lines when you only needed to open it once for all of them.

Code:
SPLIT="$(printf "\r\n");

while IFS="${SPLIT}" read i
do
        echo "update table set id1=0,id2=0,id3=0,id4=0,id5=0 where no='$i';"
done > output < input

Or a version in awk which may be faster yet:

Code:
awk -v RS="\r\n" '$1 { printf("update table set id1=0, id2=0, id3=0,id4=0,id5=0 where no=\"%s\";", $1); }' filename > output

If you're not on Linux, use gawk or nawk.

---------- Post updated at 01:08 PM ---------- Previous update was at 01:08 PM ----------

Depending on what your database is, it's probably also possible to pipe that output directly into your database, so it can process it as it happens, instead of having to store it in a gigantic temp file in the meantime then start over from the beginning when done.
This User Gave Thanks to Corona688 For This Post:
Naga06
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

date command op formation

Dear All I am facing below mention problem plz suggest me solution. Op of date command: > date Tue Jan 1 12:17:52 IST 2008 Now i want Jan 1 12: or Jan 1 12 op in some another variable. I had tried awk but if give me problem when there is date comes in 2 digit i.e. greater that 9.... (1 Reply)
Discussion started by: jaydeep_sadaria
1 Replies

2. Shell Programming and Scripting

Need to extract .sql files

Hi, I am trying to extract all .sql files present in a particular directory and its sub directories. How can i do this using shell script or awk? any help would be earnestly appreciated. thanks. (1 Reply)
Discussion started by: sprinleo
1 Replies

3. Shell Programming and Scripting

How to retrieve all the linked script files/ctl files/sql files?

Hi I am going to migrate our datawarehouse system from HP Tru 64 Unix to the Red Hat Linux. Inside the box, it is running around 40 cron jobs; inside each cron job, it is calling other shell script files, and the shell script files may again call other shell script files or ctl files(for... (1 Reply)
Discussion started by: franksubramania
1 Replies

4. Shell Programming and Scripting

Help to merge multiple .sql files

Hello all, I have a shell script that uses multiple .sql files. These .sql files mainly contain Oracle SQL queries to pull fields from the database. I want to place all the contents of these .sql files in one .sql file and have some parameter sent based on which the respective block or query... (6 Replies)
Discussion started by: snvniranjanrao
6 Replies

5. Shell Programming and Scripting

Perl string formation from array

HI I ma using perl programming my perl is like this $InputFile = $ENV{UDE_TMP} . "/" ."cre_fmr_gen.temp_data_file_gen.dat"; @duplicates = `cat $InputFile | cut -d "|" -f 1,1 | sort | uniq -c | awk '{ if(\$1>1) {print \$2;}}'`; my $cusiplist ; foreach $cusip (@duplicates) {... (1 Reply)
Discussion started by: ptappeta
1 Replies

6. UNIX for Advanced & Expert Users

Data formation

I have a data like as follows, I need to format it as shown in as below. Request you to help me here ? I/P aa|3|1 aa|4|2 bb|3|1 bb|4|1 cc|3|26 cc|4|1 O/P aa|3|1|4|2 bb|3|1|4|1 cc|3|26|4|1 Thanks, Srikanth (5 Replies)
Discussion started by: srikanth38
5 Replies

7. Shell Programming and Scripting

How to get sql command with output in files?

I tried executing set of queries from shell script but not able to capture the input query in the log file,but the input query is not displayed along with output. Only the outputs are being captured in the log file. Is there a way to capture the input query along with the corresponding input?? (8 Replies)
Discussion started by: pallvi_mahajan
8 Replies

8. Programming

Perl script to create football formation

I need help to create varieties of football formation. The available positions are: GK SW DR DC DL WBR DM WBL MR MC ML AMR AMC AML ST But the conditions are: a. the maximum number in 1 formation: GK is 1 SW is 1 (1 Reply)
Discussion started by: Tzeronone
1 Replies

9. Shell Programming and Scripting

Football formation

I'm trying to create a perl script that will automatically fit the player's name to the formation. For example, in my: DR.txt Arbeloa 200 Carvajal 190 Ramos 180 DC.txt Ramos 200 Pepe 190 Varane 180 DL.txt Marcelo 200 Coentrao 190 Arbeloa 180 formation.txt: DR DC DC DL DR (7 Replies)
Discussion started by: Tzeronone
7 Replies

10. Programming

Not a repeated question (Perl Script Create Football Formation)

https://www.unix.com/programming/252468-perl-script-create-football-formation.html https://www.unix.com/members/43551.html, it is not repeated question. please read it before u block my question. Unblock it for me. Thanks:mad: (0 Replies)
Discussion started by: Tzeronone
0 Replies

LEARN ABOUT SUSE

split

SPLIT(1)							   User Commands							  SPLIT(1)

NAME

       split - split a file into pieces

SYNOPSIS

       split [OPTION]... [INPUT [PREFIX]]

DESCRIPTION

       Output  fixed-size  pieces  of  INPUT to PREFIXaa, PREFIXab, ...; default size is 1000 lines, and default PREFIX is `x'.  With no INPUT, or
       when INPUT is -, read standard input.

       Mandatory arguments to long options are mandatory for short options too.

       -a, --suffix-length=N
	      use suffixes of length N (default 2)

       -b, --bytes=SIZE
	      put SIZE bytes per output file

       -C, --line-bytes=SIZE
	      put at most SIZE bytes of lines per output file

       -d, --numeric-suffixes
	      use numeric suffixes instead of alphabetic

       -l, --lines=NUMBER
	      put NUMBER lines per output file

       --verbose
	      print a diagnostic just before each output file is opened

       --help display this help and exit

       --version
	      output version information and exit

       SIZE may have a multiplier suffix: b 512, kB 1000, K 1024, MB 1000*1000, M 1024*1024, GB 1000*1000*1000, G 1024*1024*1024, and so on for T,
       P, E, Z, Y.

AUTHOR

       Written by Torbjorn Granlund and Richard M. Stallman.

REPORTING BUGS

       Report split bugs to bug-coreutils@gnu.org
       GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
       General help using GNU software: <http://www.gnu.org/gethelp/>

COPYRIGHT

       Copyright (C) 2009 Free Software Foundation, Inc.  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
       This is free software: you are free to change and redistribute it.  There is NO WARRANTY, to the extent permitted by law.

SEE ALSO

       The  full  documentation  for split is maintained as a Texinfo manual.  If the info and split programs are properly installed at your site,
       the command

	      info coreutils 'split invocation'

       should give you access to the complete manual.

GNU coreutils 7.1						     July 2010								  SPLIT(1)
All times are GMT -4. The time now is 01:54 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy