Sponsored Content
Top Forums Shell Programming and Scripting Need To Delete Lines Based On Search Criteria Post 302770582 by angshuman on Sunday 17th of February 2013 08:51:43 AM
Old 02-17-2013
Need To Delete Lines Based On Search Criteria

Hi All,

I have following input file. I wish to retain those lines which match multiple search criteria. The search criteria is stored in a variable seperated from each other by comma(,).

HTML Code:
SEARCH_CRITERIA = "REJECT, DUPLICATE" 

Input File:

ERROR,MYFILE_20130214_11387,9,37.75
REJECT,MYFILE_20130214_12486,1,20
DUPLICATE,MYFILE_20130214_12486,8,17.75
REJECT,MYFILE_20130214_14903,9,37.75
ERROR,MYFILE_20130214_17108,9,37.75
DUPLICATE,MYFILE_20130214_1811,0,0
My ouput file should look like below:

HTML Code:
REJECT,MYFILE_20130214_12486,1,20
DUPLICATE,MYFILE_20130214_12486,8,17.75
REJECT,MYFILE_20130214_14903,9,37.75
DUPLICATE,MYFILE_20130214_1811,0,0
Thanks & Regards
Angshuman
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Select records based on search criteria on first column

Hi All, I need to select only those records having a non zero record in the first column of a comma delimited file. Suppose my input file is having data like: "0","01/08/2005 07:11:15",1,1,"Created",,"01/08/2005" "0","01/08/2005 07:12:40",1,1,"Created",,"01/08/2005"... (2 Replies)
Discussion started by: shashi_kiran_v
2 Replies

2. Shell Programming and Scripting

remove lines based on score criteria

Hi guys, Please guide for Solution. PART-I INPUT FILE (has 2 columns ID and score) TC5584_1 93.9 DV161411_2 79.5 BP132435_5 46.8 EB682112_1 34.7 BP132435_4 29.5 TC13860_2 10.1 OUTPUT FILE (It shudn't contain the line ' BP132435_4 29.5 ' as BP132435 is repeated... (2 Replies)
Discussion started by: smriti_shridhar
2 Replies

3. Shell Programming and Scripting

Append specific lines to a previous line based on sequential search criteria

I'll try explain this as best I can. Let me know if it is not clear. I have large text files that contain data as such: 143593502 09-08-20 09:02:13 xxxxxxxxxxx xxxxxxxxxxx 09-08-20 09:02:11 N line 1 test line 2 test line 3 test 143593503 09-08-20 09:02:13... (3 Replies)
Discussion started by: jesse
3 Replies

4. Shell Programming and Scripting

Delete new lines based on search criteria

Hi all! A bit of background: I am trying to create a script that formats SQL statements. I have gotten so far as to add new lines based on certain match criteria like commas, keywords etc. In the process, I end up adding newlines where I don't want. For example: substr(colName, 1, 10)... (3 Replies)
Discussion started by: jayarkay
3 Replies

5. Shell Programming and Scripting

awk search & delete located criteria

Guys, I manages to get awk to search and print the files that I want to delete. However I am stuck on the delete portion. Here is the command that I am using to fins these files. find /usr/local/apache/conf/vhosts/ -type f | awk '/e$/' The output is perfect. The files look like so: ... (4 Replies)
Discussion started by: jaysunn
4 Replies

6. Shell Programming and Scripting

Extract data based on specific search criteria

I have a huge file (about 2 millions records) contains data separated by “,” (comma). As part of the requirement, I can't change the format. The objective is to remove some of the records with the following condition. If the 23rd field on each line start with 302 , I need to remove that from the... (4 Replies)
Discussion started by: jaygamini
4 Replies

7. Shell Programming and Scripting

Merging Lines based on criteria

Hello, Need help with following scenario. A file contains following text: {beginning of file} New: This is a new record and it is not on same line. Since I have lost touch with script take this challenge and bring all this in one line. New: Hello losttouch. You seem to be struggling... (4 Replies)
Discussion started by: losttouch
4 Replies

8. Shell Programming and Scripting

Select lines from a file based on a criteria

Hi I need to select lines from a txt file, I have got a line starting with ZMIO:MSISDN= and after a few line I have another line starting with 'MOBILE STATION ISDN NUMBER' and another one starting with 'VLR-ADDRESS' I need to copy these three lines as three different columns in a separate... (3 Replies)
Discussion started by: Tlcm sam
3 Replies

9. Shell Programming and Scripting

Copying section of file based on search criteria

Hi Guru's, I am new to unix scripting. I have a huge file with user details in it(file2) and I have another file with a list of users(file1). Script has to search a user from file1 and get all the associated lines from file2. Example: fiel1: cn=abc cn=DEF cn=xyx File 2: dn:... (10 Replies)
Discussion started by: Samingla
10 Replies

10. Shell Programming and Scripting

Delete duplicate row based on criteria

Hi, I have an input file as shown below: 20140102;13:30;FR-AUD-LIBOR-1W;2.495 20140103;13:30;FR-AUD-LIBOR-1W;2.475 20140106;13:30;FR-AUD-LIBOR-1W;2.495 20140107;13:30;FR-AUD-LIBOR-1W;2.475 20140108;13:30;FR-AUD-LIBOR-1W;2.475 20140109;13:30;FR-AUD-LIBOR-1W;2.475... (2 Replies)
Discussion started by: shash
2 Replies
SHOREWALL6-PROVIDER(5)						  [FIXME: manual]					    SHOREWALL6-PROVIDER(5)

NAME
providers - Shorewall6 Providers file SYNOPSIS
/etc/shorewall6/providers DESCRIPTION
This file is used to define additional routing tables. You will want to define an additional table if: o You have connections to more than one ISP or multiple connections to the same ISP o You run Squid as a transparent proxy on a host other than the firewall. o You have other requirements for policy routing. Each entry in the file defines a single routing table. If you wish to omit a column entry but want to include an entry in the next column, use "-" for the omitted entry. The columns in the file are as follows. NAME - name The provider name. Must be a valid shell variable name. The names 'local', 'main', 'default' and 'unspec' are reserved and may not be used as provider names. NUMBER - number The provider number -- a number between 1 and 15. Each provider must be assigned a unique value. MARK (Optional) - value A FWMARK value used in your shorewall6-tcrules(5)[1] file to direct packets to this provider. If HIGH_ROUTE_MARKS=Yes in shorewall6.conf(5)[2], then the value must be a multiple of 256 between 256 and 65280 or their hexadecimal equivalents (0x0100 and 0xff00 with the low-order byte of the value being zero). Otherwise, the value must be between 1 and 255. Each provider must be assigned a unique mark value. This column may be omitted if you don't use packet marking to direct connections to a particular provider. DUPLICATE - routing-table-name The name of an existing table to duplicate to create this routing table. May be main or the name of a previously listed provider. You may select only certain entries from the table to copy by using the COPY column below. INTERFACE - interface The name of the network interface to the provider. Must be listed in shorewall6-interfaces(5)[3]. GATEWAY - {-|address|detect} The IP address of the provider's gateway router. You can enter "detect" here and Shorewall6 will attempt to detect the gateway automatically. For PPP devices, you may omit this column. OPTIONS (Optional) - [-|option[,option]...] A comma-separated list selected from the following. The order of the options is not significant but the list may contain no embedded whitespace. balance Added in Shorewall 4.4.25. Causes a default route to this provider's gateway to be added to the main routing table (USE_DEFAULT_RT=No) or to the balance routing table (USE_DEFAULT_RT=Yes). At most one provider can specify this option. fallback Added in Shorewall 4.4.25. Causes a default route to this provider's gateway to be added to the default routing table.At most one provider can specify this option. track If specified, inbound connections on this interface are to be tracked so that responses may be routed back out this same interface. You want to specify track if internet hosts will be connecting to local servers through this provider. Beginning with Shorewall 4.4.3, track defaults to the setting of the TRACK_PROVIDERS option in shorewall6.conf[4] (5). If you set TRACK_PROVIDERS=Yes and want to override that setting for an individual provider, then specify notrack (see below). loose Shorewall6 normally adds a routing rule for each IP address on an interface which forces traffic whose source is that IP address to be sent using the routing table for that interface. Setting loose prevents creation of such rules on this interface. notrack Added in Shorewall 4.4.3. When specified, turns off track. optional (deprecated for use with providers that do not share an interface) If the interface named in the INTERFACE column is not up and configured with an IPv4 address then ignore this provider. If not specified, the value of the optional option for the INTERFACE in shorewall6-interfaces(5)[3] is assumed. Use of that option is preferred to this one, unless an address is provider in the INTERFACE column. src=source-address Specifies the source address to use when routing to this provider and none is known (the local client has bound to the 0 address). May not be specified when an address is given in the INTERFACE column. If this option is not used, Shorewall6 substitutes the primary IP address on the interface named in the INTERFACE column. mtu=number Specifies the MTU when forwarding through this provider. If not given, the MTU of the interface named in the INTERFACE column is assumed. tproxy Added in Shorewall 4.5.4. Used for supporting the TPROXY action in shorewall-tcrules(5). See http://www.shorewall.net/Shorewall_Squid_Usage.html. When specified, the MARK, DUPLICATE and GATEWAY columns should be empty, INTERFACE should be set to 'lo' and tproxy should be the only OPTION. Only one tproxy provider is allowed. COPY - [{none|interface[,interface]...}] A comma-separated list of other interfaces on your firewall. Wildcards specified using an asterisk ("*") are permitted (e.g., tun* ). Usually used only when DUPLICATE is main. Only copy routes through INTERFACE and through interfaces listed here. If you only wish to copy routes through INTERFACE, enter none in this column. EXAMPLES
Example 1: You run squid in your DMZ on IP address 2002:ce7c:92b4:1::2. Your DMZ interface is eth2 #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS Squid 1 1 - eth2 2002:ce7c:92b4:1::2 - Example 2: eth0 connects to ISP 1. The ISP's gateway router has IP address 2001:ce7c:92b4:1::2. eth1 connects to ISP 2. The ISP's gateway router has IP address 2001:d64c:83c9:12::8b. eth2 connects to a local network. #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY ISP1 1 1 main eth0 2001:ce7c:92b4:1::2 track eth2 ISP2 2 2 main eth1 2001:d64c:83c9:12::8b track eth2 FILES
/etc/shorewall6/providers SEE ALSO
http://shorewall.net/MultiISP.html http://shorewall.net/configuration_file_basics.htm#Pairs shorewall6(8), shorewall6-accounting(5), shorewall6-actions(5), shorewall6-blacklist(5), shorewall6-hosts(5), shorewall6-interfaces(5), shorewall6-maclist(5), shoewall6-netmap(5),shorewall6-params(5), shorewall6-policy(5), shorewall6-rtrules(5), shorewall6-routestopped(5), shorewall6-rules(5), shorewall6.conf(5), shorewall6-secmarks(5), shorewall6-tcclasses(5), shorewall6-tcdevices(5), shorewall6-tcrules(5), shorewall6-tos(5), shorewall6-tunnels(5), shorewall6-zones(5) NOTES
1. shorewall6-tcrules(5) http://www.shorewall.net/manpages6/shorewall6-tcrules.html 2. shorewall6.conf(5) http://www.shorewall.net/manpages6/shorewall6.conf.html 3. shorewall6-interfaces(5) http://www.shorewall.net/manpages6/shorewall6-interfaces.html 4. shorewall6.conf http://www.shorewall.net/manpages6/shorwewall6.conf.html [FIXME: source] 06/28/2012 SHOREWALL6-PROVIDER(5)
All times are GMT -4. The time now is 12:40 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy