Using input from one file to define scope of other file in Linux
Hi,
I have two files A and B and would like to use A as a filter. Like this:
File A.txt: Contains a list of IP addresses, each one occurring only once and in order:
File B.txt: Contains the same IP addresses with a corresponding ping time each (in fact, file A.txt is generated with a simple sort -u command from file B.txt). B.txt could consist of thousands of rows but I have sorted all addresses and ping times in ascending order like this with another filter:
For each IP address in file A, I want to calculate the min/med/max ping time and the packet delay variation (difference in time between min/max) and store that in a separate file. So the result should look like this:
Have tried quite a few different ways to solve this. One obvious way would probably be to loop through file A, and for each IP address I do a grep in file B and direct the result into a new file which is analysed separately. Something like this:
I know the syntax won't do the trick (not any other similar syntax either, have tried them all), but if I could get one file per IP address I could figure out how to analyze the data.
After monitoring forums, I found out that a better way to do this on such big text files would be to use awk. Have found many examples in this forum, but none that takes entries from one file as input and uses that to define the scope for another file.
I have tried to combine 'awk' and 'read' but as the rookie I am I only manage to get some grey hair. Even this simple one (only trying to read A.txt and see if I can get the IP address out and match it with the left column in B.txt) fails:
Just wonder if someone could give some advise here? Have really scanned after solutions in other threads on many forums but just can't pull it together.
As side note: Would be very interesting to understand if all of this could be done in a single command in some way also (including my first steps where A.txt is created and B.txt is sorted). Generating new files all the time means writing to the flash over and over and also add code to clean up those files as I get new files all the time. Would like to avoid that if possible.
Thanks!
Z
Last edited by Zooma; 03-12-2015 at 11:13 PM..
Reason: Fixed two typos.
The sample data you have shown has averages that happen to work out to whole numbers. Do you always want the averages displayed as values rounded to whole numbers? If not, how many decimal places do you want in the output?
What formula are you using to calculate the packet delay variation? How many decimal places do you want to display in the output in the PDV column?
Does the output need to be sorted by IP address? (If not, the sort step can probably be eliminated.)
This User Gave Thanks to Don Cragun For This Post:
Hi Don,
Thanks for reaching out. Made the sample data like that for a simple printout, but I realize it wasn't smart . One decimal would be sufficient for all entries in the result file.
The packet delay variation (jitter) is obtained by first finding the max and min ping times for an IP address and then substract the min from the max. Ex: max=10, min=2, PDV=10-2=8.
The output doesn't have to be sorted. If I need that later I can fix it. If more than one row is required that's also fine. What I mainly want is to avoid writing/deleting files if possible.
If I'm reading Chubbier_XL's code correctly, if the input file is not sorted, the max value (X[$1]) may not be saved correctly. The following slightly modified code (adjusted for 1 decimal point in the average output and to save the correct maximum value for each IP address) should work:
which, with your sample input, produces the output:
If someone else wants to try this on a Solaris system, change awk to /usr/xpg4/bin/awk.
These 2 Users Gave Thanks to Don Cragun For This Post:
Cope sample1: test.sh
i=0
echo " Outside loop i = $i "
while
do
i=$(( $i + 1))
echo "Inside loop i = $i "
done
echo " Out of loop i is : $i "
When run output :
Outside loop i = 0
Inside loop i = 1
Inside loop i = 2
Inside loop i = 3
Inside loop i = 4
Inside loop i = 5
Inside... (8 Replies)
Hello,
I would like to modify an existing script of mine that uses a manually defined "MCVERSION" variable and make it define that variable instead based on this JSON file stored online:
https://s3.amazonaws.com/Minecraft.Download/versions/versions.json
Within that JSON, I 'm looking for... (4 Replies)
Hi all ,
The FTP user defind in my passwd file has ! in the hash password field and i want to know way is that its usually either
MD5(Unix) hash or * can anyone explain to me i'm new for unix and want to learn
this how my passwd file looks :
... (2 Replies)
HI
I have file A.txt
_1A
_2A
_3A
_4A
I want define all as different variable.
$1A=_1A
$2B=_2A
$3C=_3A
$4D=_4A
Now i can use any variable in my script. (3 Replies)
The scope of the shell/perl script is to read the input text file. Validate the expiry date of each certificate and send the mail to the user. The user takes action to add the new certificate to the storage file and user owns the responsibility to update the input text file with the new certificate... (5 Replies)
awk -v nfiles="10"
{
.........................
............
}' $1 $2 $3 $4 $5 $6 $7 $8 $9 $10
up to9 inputs script is running fine but after that it is throwing error file not found though there is a file.
I used {$10} but no use. (1 Reply)
All,
I am trying to figure out a script to run in windows that will allow me to match on First column in file1 to 8th Column in File2 then
Insert file1 column2 to file2 column4 then create a new file.
File1:
12345 Sam
12346 Bob
12347 Bill
File2:... (1 Reply)
i try to install solaris OS in M4000 using OK prompts by boot net .
in my rules file i define the cpus like below
probe cpus
probe networks
networks 2 && cpus 8-10 && disksize c0t0d0 100000-292000 && disksize c0t1d0 100000-292000 && memsize 16384 && model SUNW,SPARC-Enterprise... (0 Replies)
howdy experts,
i am using 2 server- Solaris 5.9
i have tape device attached with 1 of my solaris server. But others not.
# modinfo|grep tape
152 13d43e4 1333c 33 1 st (SCSI tape Driver 1.231)
now i want to Backup DATA file and System File in Tape Drive.
How do I take data and... (3 Replies)