Sponsored Content
Top Forums Shell Programming and Scripting Merging lines - Tuning the script Post 302172832 by senthil_is on Tuesday 4th of March 2008 08:39:03 PM
Old 03-04-2008
Merging lines - Tuning the script

Hi, Anybody can tune the below script? Its taking 1hour to process 7000 lines.

addendaRecInd=7
entryDetailRecord=6
hdrDetailRecord=1
companyHdrRecord=5
companyCtrlRecord=8
trlRecord=9
recLength=94


FormatDone=false
trlCompleted=false

prevEntryCalc=false
batchCnt=0
hdrFound=false
while read line
do

# Find the Line #
grep -n "$line" $srcFile |cut -d : -f1 |read lineno
#find the Rec Type
if [ `echo $line|grep "^$companyHdrRecord"| wc -l` -ne 0 ]
then
hdrFound="true"
val=""
typeset -L94 line
hdr="$line"
typeset -R0 line
typeset -R0 batchCnt
fi
if [ `echo $line|grep "^$hdrDetailRecord"| wc -l` -ne 0 ]
then
typeset -L94 line
value=""
echo "$line" >> $destFile
line=""
typeset -R0 line
fi
if [ `echo $line|grep "^$trlRecord"| wc -l` -ne 0 -a $trlCompleted = false ]
then
echo $trlCompleted
value=""
typeset -L94 line
echo "$line" >> $destFile
line=""
typeset -R0 line
trlCompleted=true
echo $trlCompleted
fi
if [ `echo $line|grep "^$companyCtrlRecord"| wc -l` -ne 0 ]
then
if [ "$hdrFound" != "true" ]
then
echo "hdrFound" $hdrFound
echo "The last line is " $value
if [ "$value" != " " ]
then
typeset -L599 value
typeset -L6 batchCnt
echo "$value$batchCnt" >> $destFile
typeset -R0 value
typeset -R0 batchcnt
fi
typeset -L94 line
typeset -L6 batchcnt
echo "$line$batchCnt" >> $destFile
typeset -R0 line
typeset -R0 batchcnt
value=""
line=""
prevEntryCalc=false
fi
fi
if [ `echo $line|grep "^$entryDetailRecord"| wc -l` -ne 0 ]
then
# Print Header after the entry detail record has been read
if [ "$hdrFound" != "false" ]
then
batchCnt=`expr $batchCnt + 1 `
typeset -L94 hdr
typeset -L6 batchCnt
echo "$hdr$batchCnt" >> $destFile
typeset -R0 hdr
hdrFound="false"
hdr=""
fi
if [ "$prevEntryCalc" != "false" ]
then
typeset -L599 value
typeset -L6 batchCnt
echo "$value$batchCnt" >> $destFile
typeset -R0 value
typeset -R0 batchcnt
value=""
fi
line=`echo "$line"`
value=`echo "$line"`
prevEntryCalc="true"
fi
if [ `echo $line|grep "^$addendaRecInd" | wc -l` -ne 0 ]
then
typeset -L101 line
len=`echo $line|wc -c`
#echo "The Length of addenda record is "$len
value="$value$line"
typeset -R0 line
fi
done < $srcFile

Last edited by senthil_is; 03-04-2008 at 09:40 PM.. Reason: adding some more tips
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Merging lines into one

Hello. I would be very pleased if sb. help me to solve my problem. I've got a file with many non blank lines and I want to merge all lines into one not destroy the informations on them. I've tryed it with split and paste, tr, sed , but everything I've done has been wrong. I know about crazy... (8 Replies)
Discussion started by: Foxgard
8 Replies

2. Web Development

MySQL Tuning Tools with mysqltuner.pl and tuning-primer.sh

We have been tuning MySQL lately and I ran accoss two useful tools that you might be interested in: mysqltuner.pl tuning-primer.sh Both of these scripts are quite useful for MySQL tuning. Here is some sample output of mysqltuner.pl >> MySQLTuner 0.9.8 - Major Hayden... (3 Replies)
Discussion started by: Neo
3 Replies

3. Shell Programming and Scripting

Shell script for merging lines in a loop

Dear All, I need a script to merge lines of an input file in a loop, please guide me for the script or one liner(awk, sed, tr, shell, perl). I/P File------------------------- APaul,,,,SDH,,23,,,PPH,,2 ,,,,KKH,,19,,,MMH,,12, ,,,,CCH,,22,,,MNH,,19, ,,,,TCH,,55,,,NNH,,67,... (3 Replies)
Discussion started by: ashis.tewari
3 Replies

4. Shell Programming and Scripting

Merging lines

Hi folks. Could somebody help me write a script or command that will look through a file and for every line that doesn't contain a certain value, merge it with the one above? For example, the file contains: SCOTLAND|123|ABC|yes SCOTLAND|456|DEF|yes SCOTLAND|78 9|GHI|yes ... (3 Replies)
Discussion started by: MDM
3 Replies

5. Shell Programming and Scripting

merging two .txt files by alternating x lines from file 1 and y lines from file2

Hi everyone, I have two files (A and B) and want to combine them to one by always taking 10 rows from file A and subsequently 6 lines from file B. This process shall be repeated 40 times (file A = 400 lines; file B = 240 lines). Does anybody have an idea how to do that using perl, awk or sed?... (6 Replies)
Discussion started by: ink_LE
6 Replies

6. Shell Programming and Scripting

Merging lines

Thanks it worked for me. I have one more question on top of that. We had few records which were splitted in 2 lines instead of one. Now i identified those lines. The file is too big to open via vi and edit it. How can i do it without opening the file. Suppose, I want line number 1001 & 1002 to... (2 Replies)
Discussion started by: Gangadhar Reddy
2 Replies

7. Shell Programming and Scripting

Merging 2 lines together

I have a small problem, which due to my lack of knowledge, has left me unable to decipher some of the solutions that I looked at on these forums. So below is a piece of text, which I ran via cat -vet, which comes from within a program file. I have many such programs to process and repeatable,... (4 Replies)
Discussion started by: skarnm
4 Replies

8. Shell Programming and Scripting

Merging the lines of a file

Hello, I have a file with few lines starting with a digit (1-5 only ) followed by a dot (.). Remaining all the lines to be merged with its previous numbered lines. Merging must be done with a space. E.g., Source file: 3. abc def xyz 5. pqr mno def 4. jkl uvw 7. ghi 1. abc xyz 6. mno... (4 Replies)
Discussion started by: magnus29
4 Replies

9. Shell Programming and Scripting

Merging multiple lines to columns with awk, while inserting commas for missing lines

Hello all, I have a large csv file where there are four types of rows I need to merge into one row per person, where there is a column for each possible code / type of row, even if that code/row isn't there for that person. In the csv, a person may be listed from one to four times... (9 Replies)
Discussion started by: RalphNY
9 Replies

10. UNIX for Beginners Questions & Answers

Merging two lines into one (awk)

Hi, I am attempting to merge the following lines which run over two lines using awk. INITIAL OUTPUT 2019 Sep 28 10:47:24.695 hkaet9612 last message repeated 1 time 2019 Sep 28 10:47:24.695 hkaet9612 %ETHPORT-5-IF_DOWN_INTERFACE_REMOVED: Interfa ce Ethernet1/45 is down (Interface removed)... (10 Replies)
Discussion started by: sand1234
10 Replies
ECHO(3) 								 1								   ECHO(3)

echo - Output one or more strings

SYNOPSIS
void echo (string $arg1, [string $...]) DESCRIPTION
Outputs all parameters. echo is not actually a function (it is a language construct), so you are not required to use parentheses with it. echo (unlike some other language constructs) does not behave like a function, so it cannot always be used in the context of a function. Additionally, if you want to pass more than one parameter to echo, the parameters must not be enclosed within parentheses. echo also has a shortcut syntax, where you can immediately follow the opening tag with an equals sign. Prior to PHP 5.4.0, this short syn- tax only works with the short_open_tag configuration setting enabled. I have <?=$foo?> foo. PARAMETERS
o $arg1 - The parameter to output. o $... - RETURN VALUES
No value is returned. EXAMPLES
Example #1 echo examples <?php echo "Hello World"; echo "This spans multiple lines. The newlines will be output as well"; echo "This spans multiple lines. The newlines will be output as well."; echo "Escaping characters is done "Like this"."; // You can use variables inside of an echo statement $foo = "foobar"; $bar = "barbaz"; echo "foo is $foo"; // foo is foobar // You can also use arrays $baz = array("value" => "foo"); echo "this is {$baz['value']} !"; // this is foo ! // Using single quotes will print the variable name, not the value echo 'foo is $foo'; // foo is $foo // If you are not using any other characters, you can just echo variables echo $foo; // foobar echo $foo,$bar; // foobarbarbaz // Some people prefer passing multiple parameters to echo over concatenation. echo 'This ', 'string ', 'was ', 'made ', 'with multiple parameters.', chr(10); echo 'This ' . 'string ' . 'was ' . 'made ' . 'with concatenation.' . " "; echo <<<END This uses the "here document" syntax to output multiple lines with $variable interpolation. Note that the here document terminator must appear on a line with just a semicolon. no extra whitespace! END; // Because echo does not behave like a function, the following code is invalid. ($some_var) ? echo 'true' : echo 'false'; // However, the following examples will work: ($some_var) ? print 'true' : print 'false'; // print is also a construct, but // it behaves like a function, so // it may be used in this context. echo $some_var ? 'true': 'false'; // changing the statement around ?> NOTES
Note Because this is a language construct and not a function, it cannot be called using variable functions. SEE ALSO
print(3), printf(3), flush(3), Heredoc syntax. PHP Documentation Group ECHO(3)
All times are GMT -4. The time now is 09:34 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy