Improve script and get new output file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Improve script and get new output file
# 15  
Old 06-14-2015
Thanks a lot RudiC.
Is there the way to include in the script other output file merging all FILENAME".new" . . .
I am trying to do this now Smilie

Regards

---------- Post updated at 08:18 AM ---------- Previous update was at 07:38 AM ----------

Quote:
Originally Posted by jiam912
Thanks a lot RudiC.
Is there the way to include in the script other output file merging all FILENAME".new" . . .
I am trying to do this now Smilie

Regards
I got the merged file Smilie... thanks SmilieSmilie
# 16  
Old 06-14-2015
Please share your solution here for others to benefit...

---------- Post updated at 15:26 ---------- Previous update was at 15:24 ----------

Still need the commented/explained version?
This User Gave Thanks to RudiC For This Post:
# 17  
Old 06-14-2015
Dear Rudi C.

Maybe the way I got the file is not to good, but I got it using a loop for. cat every file and merge to single one . Smilie

Yes, please if you can, please provide some comments about the script procedures. This script is amazing Thanks.
# 18  
Old 06-14-2015
Why don't you use - in lieu of FILENAME".new", which will change with every input file - ONE constant file name which then would collect all output. Or just print to stdout and redirect that into your result file of choice?

---------- Post updated at 20:21 ---------- Previous update was at 19:47 ----------

Commented version of script:
Code:
awk '
FNR==1          {delete CHGARR                                          # 1. line in file: read in next change file; start with clean CHGARR
                 swath=FILENAME                                         # get sequence number from filename
                 sub (/^.*_/,  "", swath)
                 sub (/\..*$/, "", swath)
                 while (1 == getline LINE < (swath"offb1-Sx.sps"))      # read single line from .sps file,   
                        {split (LINE, TMP)                              # split into TMP array, and
                         CHGARR[TMP[5]]=TMP[1]                          # assign to CHGARR for later comparison
                        }
                }

                {IX = (int($4) int($5))}                                # create index value from 4th and 5th field

IX in CHGARR    {sub ($4, sprintf("%8.2f",  substr(CHGARR[IX], 2, 5)))  # if IX found in CHGARR, replace $4 and $5
                 sub ($5, sprintf("%8.2f3", substr(CHGARR[IX], 7, 5)))  # with resp. substr from CHGARR, FMT: 99999.99    
                }                                                       # add "3" to $5

                {print $0 > FILENAME".new"}                             # print to input filename extended by "new"
                                                                        # could be stdout!
' $(eval echo spread_{$fsw..$lsw}.x01)                                  # generate sequence of file names


Last edited by RudiC; 06-14-2015 at 05:15 PM..
This User Gave Thanks to RudiC For This Post:
# 19  
Old 06-14-2015
Thanks a lot for the explanation. AWK is amazing and you too.
# 20  
Old 06-17-2015
Dear RudiC,

Sorry to ask small question.

As the the off files are located in other folder, I have try to read it but I cant Smilie

I try to had this to your script.
Code:
offsetDbDir=/temp/
#------------------------------------------------------------------------------------------------------
awk '
FNR==1          {delete CHGARR 
                 swath=FILENAME
                 sub (/^.*_/,  "", swath)
                 sub (/\..*$/, "", swath)
                 while (1 == getline LINE < (find $offsetDbDir/ -type f -iname $swath"offb1-Sx.sps"))
                       {split (LINE, TMP)
                         CHGARR[TMP[5]]=TMP[1]
                        }
                }

                {IX = (int($4) int($5))}

IX in CHGARR    {sub ($4, sprintf("%8.2f",  substr(CHGARR[IX], 2, 5)))
                 sub ($5, sprintf("%8.2f3", substr(CHGARR[IX], 7, 5)))
                }

                {print $0 > FILENAME".new"}' $(eval echo spread_{$fsw..$lsw}.x01)

Code:
I got a error division by zero attempted

# 21  
Old 06-17-2015
In your latest script, change:
Code:
awk '

to:
Code:
awk -v DbDir="$offsetDbDir" '

and change:
Code:
                 while (1 == getline LINE < (find $offsetDbDir/ -type f -iname $swath"offb1-Sx.sps"))

to:
Code:
                 while (1 == get line LINE < (DbDir swath "offb1-Sx.sps"))

Your modification failed because:
  1. Shell variables aren't available inside awk scripts.
  2. Shell command substitution doesn't work inside awk scripts.
  3. In awk variable values are substituted by referencing their names; not by $ followed by their names as in the shell command language.
But this will only work if the filename you are reading matches the name specified exactly; it won't do a case-insensitive match unless the file resides on a filesystem that is case insensitive.
This User Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Improve script

Gents, Is there the possibility to improve this script to be able to have same output information. I did this script, but I believe there is a very short code to get same output here my script awk -F, '{if($10>0 && $10<=15) print $6}' tmp1 | sort -k1n | awk '{a++} END { for (n in a )... (23 Replies)
Discussion started by: jiam912
23 Replies

2. Shell Programming and Scripting

How to improve an script?

Gents. I have 2 different scripts for the same purpose: raw2csv_1 Script raw2csv_1 finish the process in less that 1 minute raw2csv_2 Script raw2csv_2 finish the process in more that 6 minutes. Can you please check if there is any option to improve the raw2csv_2. To finish the job... (4 Replies)
Discussion started by: jiam912
4 Replies

3. Shell Programming and Scripting

Improve sftp script

Dear all, I have written two scripts to transfer files to another server outside the company. One is a batch script , and the other script calls the batch script, send the files and archive the file sent. The problem is, that I want to get the list of files which have been uploaded the the... (10 Replies)
Discussion started by: arrals_vl
10 Replies

4. UNIX for Dummies Questions & Answers

How to improve the performance of this script?

Hi , i wrote a script to convert dates to the formate i want .it works fine but the conversion is tkaing lot of time . Can some one help me tweek this script #!/bin/bash file=$1 ofile=$2 cp $file $ofile mydates=$(grep -Po '+/+/+' $ofile) # gets 8/1/13 mydates=$(echo "$mydates" | sort |... (5 Replies)
Discussion started by: vikatakavi
5 Replies

5. Shell Programming and Scripting

Looking to improve the output of this awk one-liner

I have the following awk one-liner I came up with last night to gather some data. and it works pretty well (apologies, I'm quite new with awk, and don't know how to format this pretty-printed). You can see the output with it. awk '{if ($8 == 41015 && $21 == "requests") arr+=$20;if ($8 == 41015... (3 Replies)
Discussion started by: DeCoTwc
3 Replies

6. Shell Programming and Scripting

Var Check Script (Help improve if possible)

I am working on a script to check the var on all of my systems. Can someone help me fix it to work better or give me suggestions. #!/bin/ksh IN=/path/to/list_of_workstations.txt while read hostnames do if ping $hostnames 1 | grep alive > /dev/null then percent=`ssh -q... (3 Replies)
Discussion started by: whotippedmycow
3 Replies

7. Shell Programming and Scripting

Want to improve the performance of script

Hi All, I have written a script as follows which is taking lot of time in executing/searching only 3500 records taken as input from one file in log file of 12 GB Approximately. Working of script is read the csv file as an input having 2 arguments which are transaction_id,mobile_number and search... (6 Replies)
Discussion started by: poweroflinux
6 Replies

8. Shell Programming and Scripting

Improve the performance of a shell script

Hi Friends, I wrote the below shell script to generate a report on alert messages recieved on a day. But i for processing around 4500 lines (alerts) the script is taking aorund 30 minutes to process. Please help me to make it faster and improve the performace of the script. i would be very... (10 Replies)
Discussion started by: apsprabhu
10 Replies

9. Shell Programming and Scripting

Any way to improve performance of this script

I have a data file of 2 gig I need to do all these, but its taking hours, any where i can improve performance, thanks a lot #!/usr/bin/ksh echo TIMESTAMP="$(date +'_%y-%m-%d.%H-%M-%S')" function showHelp { cat << EOF >&2 syntax extreme.sh FILENAME Specify filename to parse EOF... (3 Replies)
Discussion started by: sirababu
3 Replies

10. Shell Programming and Scripting

Can I improve this script ???

Hi all, Still a newbie and learning as I go ... as you do :) Have created this script to report on disc usage and I've just included the ChkSpace function this morning. It's the first time I've read a file (line-by-bloody-line) and would like to know if I can improve this script ? FYI - I... (11 Replies)
Discussion started by: Cameron
11 Replies
Login or Register to Ask a Question