Sponsored Content
Top Forums Shell Programming and Scripting Field matching between 2 files Post 302444665 by Spunkerspawn on Thursday 12th of August 2010 11:46:58 AM
Old 08-12-2010
Field matching between 2 files

Okay so I'm pretty new to scripting therefore this problem seems pretty tough. I have a main file that has a column of IP addresses and I have to compare it with 3 separate files that also have IP address columns.

These 3 files are automatically generated from 3 different servers.

Each time an IP address from the main file matches an IP address from one of the 3 server files. It has to add a new field and name it Server1 (if it matched the first file), Server2 etc...

Here's a visual output:

The main file:
Code:
Kesik,1,172.17.214.93,4005
Erisim var,2,172.17.214.93,4006
Erisim var,3,172.17.214.93,4007
Erisim var,4,192.168.13.87,4033
Kesik,5,172.28.251.21,4014
Kesik,6,192.168.13.87,4034

The files generated from the server all have the same type of output:
Code:
AVAE34-IS1472-2P01.1,In Service,AMM 2PB,somsrv01,12713,AVAE34-IS1472-2P01,MTN12713,1,172.17.213.145 
AVE34-KOTT13-6P-01.1,In Service,AMM 6PC,Undefined,13008,AVE34-KOTT13-6P-01,MTN13008,1,172.17.253.121
AVE4A54-SA4642-6P-01.1,Comms Fail,AMM 6PD,somsrv01,12818,AVE4A54-SA4642-6P-01,MTN12818,1,172.17.146.90

The final output should look something like this (Depending on which server's file the IP address is matched):
Code:
Kesik,1,172.17.214.93,4005, Srv1
Erisim var,2,172.17.214.93,4006, Srv1
Erisim var,3,172.17.214.93,4007, Srv1
Erisim var,4,192.168.13.87,4033, Srv2
Kesik,5,172.28.251.21,4014, Srv3
Kesik,6,192.168.13.87,4034, Srv1

There's also another dilemma. The main file may have the same IP address but a different port number (as you can see from the first 3 rows).

I hope I was clear in my explanation, if anything looks confusing please say so Smilie

Thank you guys!

Last edited by Franklin52; 08-12-2010 at 01:12 PM.. Reason: Please use code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Print matching field using awk

Hi All, I have a string like below: str="Hold=True Map=False 'This will map the data' Run=Yes Modify=False" I want to print the field Run=Yes and retrive the value "Yes". I cannot use simple awk command because the position of the "Run" will be different at different times. Is there a way... (6 Replies)
Discussion started by: deepakgang
6 Replies

2. Shell Programming and Scripting

Need some help matching a field in one file with a field in another

One file (file1) is as such: Abc.txt 1.1 1.3 Abc_v2.txt 1.1 1.4 Tree.txt 1.3 1.4 Grass.txt 1.3 4.5 The other (file2): Horse.txt 1.1 ref23232 Abc.txt 1.1 1.2 ref 3232-3232 1.3 ref 3232-3232 Plane.txt 1.1 1.2 ref 3232-3232 1.3 ref 3232-3232 1.4 Tree.txt 1.3 ref 3232-3232 1.4... (5 Replies)
Discussion started by: linuxkid
5 Replies

3. Shell Programming and Scripting

Matching lines across multiple csv files and merging a particular field

I have about 20 CSV's that all look like this: "","","","","","","","","","","","","","","",""What I've been told I need to produce is the exact same thing, but with each file now containing the start_code from every other file where the email matches. It doesn't matter if any of the other... (1 Reply)
Discussion started by: Demosthenes
1 Replies

4. Shell Programming and Scripting

Adding matching field

I have a flat file test.log red,5,,,,, green,7,,,,, blue,4,,,,, red,8,,,,, green,9,,,,, How i get a a result: blue,4,,,,, green,16,,,,, red,13,,,,, Thanks Video tutorial on how to use code tags in The UNIX and Linux Forums. (2 Replies)
Discussion started by: sabercats
2 Replies

5. Shell Programming and Scripting

Matching two files with special field separator

Hello, I have a file with such structure: >ENSGALG00000000011|ENSGALT00000000012|57|1123|1125 AACTGTGTGTTTTTT >ENSGALG00000000012|ENSGALT00000000013|57|1145|1155 AAAAAAGGTCCTGTGTGC >ENSGALG00000000015|ENSGALT00000000014|57|1144|1155 AAAATGTGTGTGTGTGTGTGTG I want to use another file... (8 Replies)
Discussion started by: Homa
8 Replies

6. Shell Programming and Scripting

Sum column values matching other field

this is part of a KT i am going thru. i am writing a script in bash shell, linux where i have 2 columns where 1st signifies the nth hour like 00, 01, 02...23 and 2nd the file size. sample data attached. Desired output is 3 columns which will give the nth hour, number of entries in nth hour and... (3 Replies)
Discussion started by: alpha_1
3 Replies

7. Shell Programming and Scripting

Field matching in two data files

Hello, I am looking to output all of the lines from file2 whose 11th field is present in the first field in file1. Then the second field from file1 should be appended as such: file1: 2222 0.35 4444 0.25 5555 0.75 file2: col1 col2 col3 col4 col5 col6 col7 col8 col9 col10 1111 col1 col2... (4 Replies)
Discussion started by: palex
4 Replies

8. Shell Programming and Scripting

awk to update field using matching value in file1 and substring in field in file2

In the awk below I am trying to set/update the value of $14 in file2 in bold, using the matching NM_ in $12 or $9 in file2 with the NM_ in $2 of file1. The lengths of $9 and $12 can be variable but what is consistent is the start pattern will always be NM_ and the end pattern is always ;... (2 Replies)
Discussion started by: cmccabe
2 Replies

9. UNIX for Beginners Questions & Answers

Continued trouble matching fields in different files and selective field printing ([g]awk)

I apologize in advance, but I continue to have trouble searching for matches between two files and then printing portions of each to output in awk and would very much appreciate some help. I have data as follows: File1 PS012,002 PRQ 0 1 1 17 1 0 -1 3 2 1 2 -1 ... (7 Replies)
Discussion started by: jvoot
7 Replies

10. Shell Programming and Scripting

awk to average field if matching string in another

In the awk below I am trying to get the average of the sum of $7 if the string in $4 matches in the line below it. The --- in the desired out is not needed, it is just to illustrate the calculation. The awk executes and produces the current out. I am not sure why the middle line is skipped and the... (2 Replies)
Discussion started by: cmccabe
2 Replies
IPSEC_EROUTE(5) 						  [FIXME: manual]						   IPSEC_EROUTE(5)

NAME
ipsec_eroute - list of existing eroutes SYNOPSIS
ipsec eroute cat/proc/net/ipsec_eroute OBSOLETE
Note that eroute is only supported on the classic KLIPS stack. It is not supported on any other stack and will be completely removed in future versions. On the mast stack, use ipsec policy, on the netkey stack, use ip xfrm DESCRIPTION
/proc/net/ipsec_eroute lists the IPSEC extended routing tables, which control what (if any) processing is applied to non-encrypted packets arriving for IPSEC processing and forwarding. At this point it is a read-only file. A table entry consists of: + packet count, + source address with mask and source port (0 if all ports or not applicable) + a '->' separator for visual and automated parsing between src and dst + destination address with mask and destination port (0 if all ports or not applicable) + a '=>' separator for visual and automated parsing between selection criteria and SAID to use + SAID (Security Association IDentifier), comprised of: + protocol (proto), + address family (af), where '.' stands for IPv4 and ':' for IPv6 + Security Parameters Index (SPI), + effective destination (edst), where the packet should be forwarded after processing (normally the other security gateway) together indicate which Security Association should be used to process the packet, + a ':' separating the SAID from the transport protocol (0 if all protocols) + source identity text string with no whitespace, in parens, + destination identity text string with no whitespace, in parens Addresses are written as IPv4 dotted quads or IPv6 coloned hex, protocol is one of "ah", "esp", "comp" or "tun" and SPIs are prefixed hexadecimal numbers where the prefix '.' is for IPv4 and the prefix ':' is for IPv6 SAIDs are written as "protoafSPI@edst". There are also 5 "magic" SAIDs which have special meaning: + %drop means that matches are to be dropped + %reject means that matches are to be dropped and an ICMP returned, if possible to inform + %trap means that matches are to trigger an ACQUIRE message to the Key Management daemon(s) and a hold eroute will be put in place to prevent subsequent packets also triggering ACQUIRE messages. + %hold means that matches are to stored until the eroute is replaced or until that eroute gets reaped + %pass means that matches are to allowed to pass without IPSEC processing EXAMPLES
1867 172.31.252.0/24:0 -> 0.0.0.0/0:0 => tun0x130@192.168.43.1:0 () () means that 1,867 packets have been sent to an eroute that has been set up to protect traffic between the subnet 172.31.252.0 with a subnet mask of 24 bits and the default address/mask represented by an address of 0.0.0.0 with a subnet mask of 0 bits using the local machine as a security gateway on this end of the tunnel and the machine 192.168.43.1 on the other end of the tunnel with a Security Association IDentifier of tun0x130@192.168.43.1 which means that it is a tunnel mode connection (4, IPPROTO_IPIP) with a Security Parameters Index of 130 in hexadecimal with no identies defined for either end. 746 192.168.2.110/32:0 -> 192.168.2.120/32:25 => esp0x130@192.168.2.120:6 () () means that 746 packets have been sent to an eroute that has been set up to protect traffic sent from any port on the host 192.168.2.110 to the SMTP (TCP, port 25) port on the host 192.168.2.120 with a Security Association IDentifier of tun0x130@192.168.2.120 which means that it is a transport mode connection with a Security Parameters Index of 130 in hexadecimal with no identies defined for either end. 125 3049:1::/64 -> 0:0/0 => tun:130@3058:4::5 () () means that 125 packets have been sent to an eroute that has been set up to protect traffic between the subnet 3049:1:: with a subnet mask of 64 bits and the default address/mask represented by an address of 0:0 with a subnet mask of 0 bits using the local machine as a security gateway on this end of the tunnel and the machine 3058:4::5 on the other end of the tunnel with a Security Association IDentifier of tun:130@3058:4::5 which means that it is a tunnel mode connection with a Security Parameters Index of 130 in hexadecimal with no identies defined for either end. 42 192.168.6.0/24:0 -> 192.168.7.0/24:0 => %passthrough means that 42 packets have been sent to an eroute that has been set up to pass the traffic from the subnet 192.168.6.0 with a subnet mask of 24 bits and to subnet 192.168.7.0 with a subnet mask of 24 bits without any IPSEC processing with no identies defined for either end. 2112 192.168.8.55/32:0 -> 192.168.9.47/24:0 => %hold (east) () means that 2112 packets have been sent to an eroute that has been set up to hold the traffic from the host 192.168.8.55 and to host 192.168.9.47 until a key exchange from a Key Management daemon succeeds and puts in an SA or fails and puts in a pass or drop eroute depending on the default configuration with the local client defined as "east" and no identy defined for the remote end. 2001 192.168.2.110/32:0 -> 192.168.2.120/32:0 => esp0xe6de@192.168.2.120:0 () () means that 2001 packets have been sent to an eroute that has been set up to protect traffic between the host 192.168.2.110 and the host 192.168.2.120 using 192.168.2.110 as a security gateway on this end of the connection and the machine 192.168.2.120 on the other end of the connection with a Security Association IDentifier of esp0xe6de@192.168.2.120 which means that it is a transport mode connection with a Security Parameters Index of e6de in hexadecimal using Encapsuation Security Payload protocol (50, IPPROTO_ESP) with no identies defined for either end. 1984 3049:1::110/128 -> 3049:1::120/128 => ah:f5ed@3049:1::120 () () means that 1984 packets have been sent to an eroute that has been set up to authenticate traffic between the host 3049:1::110 and the host 3049:1::120 using 3049:1::110 as a security gateway on this end of the connection and the machine 3049:1::120 on the other end of the connection with a Security Association IDentifier of ah:f5ed@3049:1::120 which means that it is a transport mode connection with a Security Parameters Index of f5ed in hexadecimal using Authentication Header protocol (51, IPPROTO_AH) with no identies defined for either end. FILES
/proc/net/ipsec_eroute, /usr/local/bin/ipsec SEE ALSO
ipsec(8), ipsec_manual(8), ipsec_tncfg(5), ipsec_spi(5), ipsec_spigrp(5), ipsec_klipsdebug(5), ipsec_eroute(8), ipsec_version(5), ipsec_pf_key(5) HISTORY
Written for the Linux FreeS/WAN project <http://www.freeswan.org/> by Richard Guy Briggs. [FIXME: source] 10/06/2010 IPSEC_EROUTE(5)
All times are GMT -4. The time now is 02:21 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy