cat ${KUSY_LocalFile} | grep -v "^#" | grep -v "^$" | while read KUSYLINE
do
# Variablennamen aus Satzaufbbau
GP_PARTNER=`echo ${KUSYLINE} | nawk -F"|" '{print $1}'`
....a.s.f....until 11 declarations
GP_MC_NAME2_RG=`echo ${KUSYLINE} | nawk -F"|" '{print $11}'`
...
SQLStatement="INSERT INTO customdb..ZW_GPDaten VALUES( \"${GP_PARTNER}\", ....a.s.f.... )"
# Jetzt SQL-Statement ins File und Info
echo "${SQLStatement}" >> ${KUSY_SQLFile}
done
...isql < ${KUSY_SQLFile}
This works, but not fine. It tooks about 1h to generate the sql code.
I assume the nawk in every line of the 11 variable delarations is the source for this behaviour.
I'm now looking for faster working solution. In perl I would do something like
Code:
( $var1, ..$var12 ) = split( "|", $LINE );
I read that awk has equal to perl a split, too (I'm not so familar with awk except
simple usage like above).
So the idea was to pass the ${KUSYLINE} to awk and get
the results into a ksh array.
The idea works generally (see sample code in the file attached to this message).
But the set -A with spaces in the occurs into shifting of the vars.
I didn't get e.g. for GP_PARTNERX=${NAWKARRAY[y] -> Celal Dikici; I get GP_PARTNERX = Celal and GP_Number=Dikici.
What is the optimal way? Or can set -A used with different delimiters?
You do not need to use awk to parse each line. You can set the IFS to ';'
and simply read each line into an array (i.e. myarray) as shown in the following
code snippet.
Code:
#!/usr/bin/ksh93
IFS=';'
while read -A myarray
do
print ${myarray[*]}
done < file
To reduce run time you need to use shell builtins as much as possible and not
call external programs such as awk.
To reduce run time you need to use shell builtins as much as possible and not call external programs such as awk.
Even if ksh is *much* faster then bash for this type of parsing I often found that awk was faster than pure shell solutions in reading and processing lines. To make sure, I just ran a benchmark on a sample file with 20000 lines using Franklin52's awk snippet and the following ksh93 code:
Code:
IFS='|'
while read -A myarray
do
print "INSERT INTO customdb..ZW_GPDaten VALUES(${myarray[0]}, ${myarray[1]}, ${myarray[2]} )"
done < sample_file
Results:
awk: 0.146 sec.
ksh: 0.636 sec.
So, awk is five times faster albeit being an external program. And, just for the record, I also tested a bash version and, as expected, it's the slowest:
Hi
I'm trying to loop through an array that contains other arrays and these arrays consist of strings with spaces. The problem is that I can't seem to preserve the spacing in the string. The string with spaces are either divided into multiple items if I change IFS to \n or all the elements of... (4 Replies)
I'm having issue capturing a value from file.list with a multiple spaces in a variable $i, tried various options like using double quotes, no quotes, single quotes, curly braces but to no avail.
cat file.list
aaa test bbb
ccc test ddd
eee test fff
for i in `cat file.list`
do
echo "$i";... (2 Replies)
I have a file, names(i) where each entry is 'first last' name. 'cat names' is fine. But in a shell script
>for file in $(cat names)
> do
> echo $file
> done
the first and last name appear on 2 lines. I have tried escaping and quoting the space but to no avail. The names are to be... (4 Replies)
'String' file contains the following contents,
D11, D31, D92, D29, D24,
using ksh, I want to remove all white spaces between characters no matter how long the string is.
Would you please give me some help? (1 Reply)
Hi,
Well as the title says, I have an array of strings (delimited by null). The length of the array is variable and length of each string is variable as well. What I need is one huge string with the original strings in the array separated by spaces.
For example is an array is such that array... (12 Replies)
Hi,
Could someone has any suggestions on this? When read a line from a file, I need to check the first char in the line, it could be a space or any char. But the leading spaces are removed by read.
Thanks. (2 Replies)
Forgive me. I am very new to kornshell scripts. The simplest things stop me dead in my tracks.
Here are two such examples.
I want to save the first 19 characters of the following string to a variable.
"Operation Completed and blah blah blah"
I know this works (from another thread):
... (2 Replies)
Hi all!
i need your help.
I'm getting started with this...
in need to insert in an array string values. But the thing is that this strings have blank spaces...
for example if a want to put "filed1 = " and "field2 = " .... in the array , when i want to print all the fields, it only shows... (4 Replies)
dear pro-coders,
is there any command out there that takes out the front spaces from a string?
sample strings:
4 members
5 members
3 members
but it has to be like so:
4 members
5 members
3 members (3 Replies)
I have been working on some code for a while, that will parse a log file, look for a specified time discrepancy between entries, and then print that line +/- n other lines out to a file...
#!/bin/bash
file=$1 # The input log file
maxTime=$2 # The time discrepancy to look for
n=$3 ... (1 Reply)
