Sponsored Content
Top Forums Shell Programming and Scripting Need help improving my script. Post 302970869 by Don Cragun on Tuesday 12th of April 2016 10:08:26 PM
Old 04-12-2016
You have the comment:
Code:
# The script now reads the file ports2.txt and sorts the data into 4 files based on it finding "Inbound or Outbound" and "TCP or UDP" in the line.

but there is no ports2.txt anywhere in your script. Do you care about having the file GC_ports.txt, or do you really just need the four GC_(tcp|udp)(in|out)bound.txt files that are created from this file?

Assuming you have more than one compressed file that is less than a week old, using -exec zcat {} + will be faster than -exec zcat {} \; and you could replace all four awk scripts and the sed script with a single awk script (which would considerably reduce the time spent reading and writing data that should only need to be read once and written at most twice; instead of reading the data five times and writing it six or seven times).

But, I would guess (hard to make any sound judgements here with no samples of the data being processed) that the bulk of the time being spent in this script is in compressing and recompressing relatively large files for your archives. And if you just need the files that you are splitting out of GC_ports.txt, the time spent creating, compressing, and archiving that unneeded file could be significant.

Can you show us some sample uncompressed data that is being pushed through the pipeline by find ... -exec zcat {} \;? Figuring out exactly what that pipeline is doing without knowing where the slashes and colons are makes it hard to feel confident about suggesting ways to streamline your awk and sed scripts.

Even though echo is a shell built-in, invoking echo four times in a row instead of calling printf (another shell built-in) once is inefficient. (Depending on what operating system you're using, you could probably produce the same output with a single echo instead of a single printf, but I prefer print since its formatting options are more portable.) Do you really want/need that many empty lines in the output produced by this script?
This User Gave Thanks to Don Cragun For This Post:
 

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

improving my script (find & replace)

Hi all, I have a script that scan files, find old templet and replace it with new one. #!/bin/ksh file_name=$1 old_templet=$2 new_templet=$3 # Loop through every file like this for file in file_name do cat $file | sed "s/old_templet/new_templet/g" > $file.new #do a global searce and... (8 Replies)
Discussion started by: amir_yosha
8 Replies

2. Shell Programming and Scripting

improving my script

Hi; I want to access our customer database to retreive all clients that have as language index 2 or 3 and take their client number. My input is a file containing all client numbers. i access the data base using a function call "scpshow". The total number of clients i want to scan is 400 000... (6 Replies)
Discussion started by: bcheaib
6 Replies

3. UNIX for Dummies Questions & Answers

Improving Unix Skills

Kindly any advice to improve my unix skills as electronic books i can download or valuable sites as this one etc... (3 Replies)
Discussion started by: sak900354
3 Replies

4. Shell Programming and Scripting

Improving this validate function

Hi guys, I use this function which was provided to me by someone at this site. It works perfectly for validating a users input option against allowed options.. example: validateInput "1" "1 3 4 5" would return 0 (success) function validateInput { input=$1 allowedInput=$2 for... (4 Replies)
Discussion started by: pyscho
4 Replies

5. Shell Programming and Scripting

Improving code by using associative arrays

I have the following code, and I am changing it to #!/bin/bash hasArgumentCModInfile=0 hasArgumentSrcsInfile=0 hasArgumentRcvsInfile=0 OLDIFS="$IFS" IFS="|=" # IFS controls splitting. Split on "|" and "=", not whitespace. set -- $* # Set the positional... (3 Replies)
Discussion started by: kristinu
3 Replies

6. Shell Programming and Scripting

Basic help improving for in loop

I'm obviously very new to this. I'm trying to write a simple for loop that will read the directory names in /Users and then copy a file into the same subdir in each user directory. I have this, and it works but it isn't great. #!/bin/bash HOMEDIRS=/Users/* for dirs in $HOMEDIRS; do if ];... (5 Replies)
Discussion started by: Heath_T
5 Replies

7. Shell Programming and Scripting

Help with improving korn shell script

I am primarily a SQA/Tester and new to korn shell. How can I improve the following script? #/bin/ksh SourceLocation=~/Scripts/Test/Source TrackerLocation=~/Scripts/Test/Tracker TargetLocation=rdbusse@rdbmbp:/Users/rdbusse/Scripts/Test/Target for file in $(cd $SourceLocation; ls) do ... (7 Replies)
Discussion started by: bayouprophet
7 Replies

8. Shell Programming and Scripting

Improving code

Gents, I did the below code to get an output (report) ,.. the code works fine but I believe it can be more shorted using better method. Please if you can help, to generate same output improving the code , will be great. here my code. # get diff in time awk '{$9=$8-prev8;prev8=$8;print... (8 Replies)
Discussion started by: jiam912
8 Replies
All times are GMT -4. The time now is 08:24 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy