Sponsored Content
Top Forums Shell Programming and Scripting Need help improving my script. Post 302970870 by garlandxj11 on Tuesday 12th of April 2016 10:44:05 PM
Old 04-12-2016
Sorry, Ports2.txt is incorrect. I pull data from two different archives so I created one script to test functionality and then copied that into two separate script files using a third script to run the other two. I had the script itself running so I pulled the code from the original file instead and missed that it was still referencing the test file.

Ports.sh only contains lines to run
vacavilleports.sh and goldcampports.sh

the code I posted was from a file named testports.sh which was the test code copied into vacavilleports.sh and goldcampports.sh then each of those was modified to reference their proper archive locations and output files.

I did not know if opening the script file while it was running would impact it so I chose the safe route of opening the test version.

Here is sample output from
Code:
GC_tcpinbound.txt
TCP internal 10.20.114.190 intmgmt 10.20.100.175 258
TCP internal 10.20.114.190 intmgmt 10.20.100.175 6455
TCP internal 10.20.114.190 intmgmt 10.20.100.175 1678
TCP internal 10.20.114.190 intmgmt 10.20.100.162 33923

Here is some sample input
Code:
Apr  5 19:00:02 Apr 05 2016 19:01:02: %ASA-6-302015: Built outbound UDP connection 731199055 for internal:10.20.114.120/53 (10.20.114.120/53) to intmgmt:10.20.100.48/53099 (10.20.100.48/53099)
Apr  5 19:00:02 Apr 05 2016 19:01:02: %ASA-6-302015: Built outbound UDP connection 731199056 for internal:10.20.114.120/53 (10.20.114.120/53) to intmgmt:10.20.100.48/43185 (10.20.100.48/43185)
Apr  5 19:00:02 Apr 05 2016 19:01:02: %ASA-6-302015: Built outbound UDP connection 731199057 for internal:10.20.114.120/53 (10.20.114.120/53) to intmgmt:10.20.100.48/42319 (10.20.100.48/42319)
Apr  5 19:00:02 Apr 05 2016 19:01:02: %ASA-6-302016: Teardown UDP connection 731198699 for outside:158.96.0.254/53 to internal:10.20.114.124/58504 duration 0:00:00 bytes 179
Apr  5 19:00:02 Apr 05 2016 19:01:02: %ASA-6-302015: Built outbound UDP connection 731199059 for internal:10.20.114.120/53 (10.20.114.120/53) to intmgmt:10.20.100.48/54069 (10.20.100.48/54069)




Here is the exact code from goldcampports.sh

Code:
echo Search started at:
date +"%m/%d/%Y %T"
# Displays the start up information and the start time

find /var/network_logs/gc/archive/asalog*  -mtime -7 -exec zcat {} \;  |  awk '/Built/&& !/10.10.120.145/{print $10, $11, $15, $18;}' | sed -e 's!/! !g' -e  's!:! !g' | awk '{if ($1 == "inbound") print $1, $2, $3, $4, $6, $7, $8; else if ($1 == "outbound") print $1, $2, $6, $7, $3, $4, $5;}' | awk '!seen[$0]++ {print}' >> /home/kenneth.cramer/asa/GC_ports.txt

# Finds all files with that begin with the name asalog that were written in the last 7 days. It then reads the files line by line looking
# for any lines containing the word Built but not the 10.10.120.145 IP address and prints out the 7th, 8th, 12th and 15th words in the line
# It then looks for any "/" slashes or ":" colons in the four words and replaaces them with spaces.
# The script now prints out the needed words from the line and then writes only unique lines to the output file.

echo
echo
echo
echo Sorting data into proper files.
# Displays that the script is now sorting the information

awk '{if ($1 == "inbound" && $2 == "TCP") print $2, $3, $4, $5, $6, $7 >> "/home/kenneth.cramer/asa/GC_tcpinbound.txt"; else if ($1 == "inbound" && $2 == "UDP") print $2, $3, $4, $5,  $6, $7 >> "/home/kenneth.cramer/asa/GC_udpinbound.txt"; else if ($1 == "outbound" && $2 == "TCP") print $2, $3, $4, $5, $6, $7 >> "/home/kenneth.cramer/asa/GC_tcpoutbound.txt"; else if ($1 == "outbound" && $2 == "UDP") print $2, $3, $4, $5, $6, $7 >> "/home/kenneth.cramer/asa/GC_udpoutbound.txt";}' /home/kenneth.cramer/asa/GC_ports.txt
# Thee script now reads the file ports2.txt and sorts the data into 4 files based on it finding "Inbound or Outbound" and "TCP or UDP" in the line.
                             DOH!!! This  ^ should read    #The script now reads the file GC_ports.txt and sorts the data into 4 files based on it finding "Inbound or Outbound" and "TCP or UDP" in the line.

echo
echo
echo
echo Compressing files for transport

tar -czvf /home/kenneth.cramer/asa/GC_ports.tgz /home/kenneth.cramer/asa/GC_*.txt
# Compresses the output files into a single file for transport off the machine.

echo Process completed for Gold Camp at:
date +"%m/%d/%Y %T"
echo
echo
times

---------- Post updated at 09:44 PM ---------- Previous update was at 09:31 PM ----------

In answer to your other questions,

1. No I do not care about having the GC_ports.txt file.
2. My only goal is to reach the output in those 4 files.
3. The blank lines are just for spacing. I did not spend much time researching the best way to do blank lines as this script has minimal output, just enough to let the person who runs it know what it is doing. I am mainly the person who runs it, but I built that in just in case someone else had to run it and got confused by the system not returning immediately to the prompt.

I hope the script is not too hard to follow, I am a network engineer not a programmer or a unix admin. This all is to assist a client in redoing their firewall.
The input is from the firewall log. We are looking for lines with the word built in it and capturing the source IP, destination IP, destination port and protocol of the connections. The four output files are dumped into 4 sheets in excel for us to see what ip's are talking and what we need to build rules for. When a previous company setup the firewall they left any/any rules in place for internal traffic and only locked down the outside interface. So we have to figure out what rules we need to create before removing those any/any rules and causing massive connectivity issues.

Yes, there are many tools out there to do this for us, but this is all client owned hardware and they don't have those tools installed. So we are left with this.

The log files it pulls are in 1 hour intervals, so 24 files per day times 7 days = 168 compressed log files. I did try copying down the zipped files and then uncompressing them on the local machine but expanded they are almost 60 gig. (repetitive text compresses VERY VERY well) Smilie

Thank you again for your suggestions and assistance.

Last edited by Scrutinizer; 04-13-2016 at 04:44 PM.. Reason: code tags
 

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

improving my script (find & replace)

Hi all, I have a script that scan files, find old templet and replace it with new one. #!/bin/ksh file_name=$1 old_templet=$2 new_templet=$3 # Loop through every file like this for file in file_name do cat $file | sed "s/old_templet/new_templet/g" > $file.new #do a global searce and... (8 Replies)
Discussion started by: amir_yosha
8 Replies

2. Shell Programming and Scripting

improving my script

Hi; I want to access our customer database to retreive all clients that have as language index 2 or 3 and take their client number. My input is a file containing all client numbers. i access the data base using a function call "scpshow". The total number of clients i want to scan is 400 000... (6 Replies)
Discussion started by: bcheaib
6 Replies

3. UNIX for Dummies Questions & Answers

Improving Unix Skills

Kindly any advice to improve my unix skills as electronic books i can download or valuable sites as this one etc... (3 Replies)
Discussion started by: sak900354
3 Replies

4. Shell Programming and Scripting

Improving this validate function

Hi guys, I use this function which was provided to me by someone at this site. It works perfectly for validating a users input option against allowed options.. example: validateInput "1" "1 3 4 5" would return 0 (success) function validateInput { input=$1 allowedInput=$2 for... (4 Replies)
Discussion started by: pyscho
4 Replies

5. Shell Programming and Scripting

Improving code by using associative arrays

I have the following code, and I am changing it to #!/bin/bash hasArgumentCModInfile=0 hasArgumentSrcsInfile=0 hasArgumentRcvsInfile=0 OLDIFS="$IFS" IFS="|=" # IFS controls splitting. Split on "|" and "=", not whitespace. set -- $* # Set the positional... (3 Replies)
Discussion started by: kristinu
3 Replies

6. Shell Programming and Scripting

Basic help improving for in loop

I'm obviously very new to this. I'm trying to write a simple for loop that will read the directory names in /Users and then copy a file into the same subdir in each user directory. I have this, and it works but it isn't great. #!/bin/bash HOMEDIRS=/Users/* for dirs in $HOMEDIRS; do if ];... (5 Replies)
Discussion started by: Heath_T
5 Replies

7. Shell Programming and Scripting

Help with improving korn shell script

I am primarily a SQA/Tester and new to korn shell. How can I improve the following script? #/bin/ksh SourceLocation=~/Scripts/Test/Source TrackerLocation=~/Scripts/Test/Tracker TargetLocation=rdbusse@rdbmbp:/Users/rdbusse/Scripts/Test/Target for file in $(cd $SourceLocation; ls) do ... (7 Replies)
Discussion started by: bayouprophet
7 Replies

8. Shell Programming and Scripting

Improving code

Gents, I did the below code to get an output (report) ,.. the code works fine but I believe it can be more shorted using better method. Please if you can help, to generate same output improving the code , will be great. here my code. # get diff in time awk '{$9=$8-prev8;prev8=$8;print... (8 Replies)
Discussion started by: jiam912
8 Replies
All times are GMT -4. The time now is 11:27 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy