Thank you for taking the time to look at this and provide input.
To start, I am not a linux/unix expert but I muddle through the best I can.
I am also in no way shape or form a programmer. Please keep that in mind as you read this script.
This script is designed to find all files in a given directory that begin with "asalog", find lines containing a specific word and then process down those lines and output just the needed information. These files are currently zipped. The files are stored on remote ZFS storage. Copying all of the files down to the local system at once then unzipping is not feasible do to storage limitations. The script works as designed but it is very slow to do the task.
Please look over the code and suggest ways that I could improve its speed. The last run took 238 minutes to complete.
Due to access limitations I have to work within BASH, I do not have the option (nor the knowledge) to utilize perl, python, etc.
Any help is welcome as well as comments on the script as it sits. It has been cobbled together by remembering programming structure learned taking Turbo Pascal in high school (many years ago) and lots of google searches.
Code:
echo Search started at:
date +"%m/%d/%Y %T"
# Displays the start up information and the start time
find /var/network_logs/gc/archive/asalog* -mtime -7 -exec zcat {} \; | awk '/Built/&& !/10.10.120.145/{print $10, $11, $15, $18;}' | sed -e 's!/! !g' -e 's!:! !g' | awk '{if ($1 == "inbound") print $1, $2, $3, $4, $6, $7, $8; else if ($1 == "outbound") print $1, $2, $6, $7, $3, $4, $5;}' | awk '!seen[$0]++ {print}' >> /home/kenneth.cramer/asa/GC_ports.txt
# Finds all files with that begin with the name asalog that were written in the last 7 days. It then reads the files line by line looking
# for any lines containing the word Built but not the 10.10.120.145 IP address and prints out the 7th, 8th, 12th and 15th words in the line
# It then looks for any "/" slashes or ":" colons in the four words and replaaces them with spaces.
# The script now prints out the needed words from the line and then writes only unique lines to the output file.
echo
echo
echo
echo Sorting data into proper files.
# Displays that the script is now sorting the information
awk '{if ($1 == "inbound" && $2 == "TCP") print $2, $3, $4, $5, $6, $7 >> "/home/kenneth.cramer/asa/GC_tcpinbound.txt"; else if ($1 == "inbound" && $2 == "UDP") print $2, $3, $4, $5, $6, $7 >> "/home/kenneth.cramer/asa/GC_udpinbound.txt"; else if ($1 == "outbound" && $2 == "TCP") print $2, $3, $4, $5, $6, $7 >> "/home/kenneth.cramer/asa/GC_tcpoutbound.txt"; else if ($1 == "outbound" && $2 == "UDP") print $2, $3, $4, $5, $6, $7 >> "/home/kenneth.cramer/asa/GC_udpoutbound.txt";}' /home/kenneth.cramer/asa/GC_ports.txt
# The script now reads the file ports2.txt and sorts the data into 4 files based on it finding "Inbound or Outbound" and "TCP or UDP" in the line.
echo
echo
echo
echo Compressing files for transport
tar -czvf /home/kenneth.cramer/asa/GC_ports.tgz /home/kenneth.cramer/asa/GC_*.txt
# Compresses the output files into a single file for transport off the machine.
echo Process completed for Gold Camp at:
date +"%m/%d/%Y %T"
echo
echo
times
Last edited by garlandxj11; 04-12-2016 at 09:50 PM..
Hi all,
I have a script that scan files, find old templet and replace it with new one.
#!/bin/ksh
file_name=$1
old_templet=$2
new_templet=$3
# Loop through every file like this
for file in file_name
do
cat $file | sed "s/old_templet/new_templet/g" > $file.new
#do a global searce and... (8 Replies)
Hi;
I want to access our customer database to retreive all clients that have as language index 2 or 3 and take their client number.
My input is a file containing all client numbers.
i access the data base using a function call "scpshow". The total number of clients i want to scan is 400 000... (6 Replies)
Hi guys, I use this function which was provided to me by someone at this site. It works perfectly for validating a users input option against allowed options..
example:
validateInput "1" "1 3 4 5" would return 0 (success)
function validateInput {
input=$1
allowedInput=$2
for... (4 Replies)
I have the following code, and I am changing it to
#!/bin/bash
hasArgumentCModInfile=0
hasArgumentSrcsInfile=0
hasArgumentRcvsInfile=0
OLDIFS="$IFS"
IFS="|=" # IFS controls splitting. Split on "|" and "=", not whitespace.
set -- $* # Set the positional... (3 Replies)
I'm obviously very new to this. I'm trying to write a simple for loop that will read the directory names in /Users and then copy a file into the same subdir in each user directory.
I have this, and it works but it isn't great.
#!/bin/bash
HOMEDIRS=/Users/*
for dirs in $HOMEDIRS; do
if ];... (5 Replies)
I am primarily a SQA/Tester and new to korn shell. How can I improve the following script?
#/bin/ksh
SourceLocation=~/Scripts/Test/Source
TrackerLocation=~/Scripts/Test/Tracker
TargetLocation=rdbusse@rdbmbp:/Users/rdbusse/Scripts/Test/Target
for file in $(cd $SourceLocation; ls)
do
... (7 Replies)
Gents,
I did the below code to get an output (report) ,.. the code works fine but I believe it can be more shorted using better method.
Please if you can help, to generate same output improving the code , will be great.
here my code.
# get diff in time
awk '{$9=$8-prev8;prev8=$8;print... (8 Replies)
04-12-2016
