Sponsored Content
Top Forums Shell Programming and Scripting Search multiple patterns in multiple files Post 302490039 by Chubler_XL on Sunday 23rd of January 2011 04:14:30 PM
Old 01-23-2011
With some tweaking it may still be possible to get this brute force solution to work fast enough, but it's not looking good. I suspect your are running out of physical memory and the system is swapping, how big is the file1.txt and how much physical memory do you have on your system?

You could consider retaining some of the work from previous scans. This really depends on your dataset and leads to the following questions about your data.

How static is it?
I'd assume the zip file contents don't change much, but perhaps you remove old zips and add new ones?

How about the contents of file1.txt is this completely different each night? Are any items searched for searched for again at later dates? (For example if we know the XYZ wasn't in the zips lastnight and it's searched for again, all we need to scan are files added since last night's scan).

Last edited by Chubler_XL; 01-23-2011 at 06:26 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to parameterize multiple search patterns and generate a new file

I have one file: 123*100*abcd*10 123*101*abcd*-29*def 123*100*abcd*-10 123*102*abcd*-105*asd I would like to parameterize the search patterns in the following way so that the user could dynamically change the search pattern. *100* and *- (ie *minus) *102* and *- The output that is... (6 Replies)
Discussion started by: augustinep
6 Replies

2. Shell Programming and Scripting

Perl: Match a line with multiple search patterns

Hi I'm not very good with the serach patterns and I'd need a sample how to find a line that has multiple patterns. Say I want to find a line that has "abd", "123" and "QWERTY" and there can be any characters or numbers between the serach patterns, I have a file that has thousands of lines and... (10 Replies)
Discussion started by: Juha
10 Replies

3. Shell Programming and Scripting

Perl - How to search a text file with multiple patterns?

Good day, great gurus, I'm new to Perl, and programming in general. I'm trying to retrieve a column of data from my text file which spans a non-specific number of lines. So I did a regexp that will pick out the columns. However,my pattern would vary. I tried using a foreach loop unsuccessfully.... (2 Replies)
Discussion started by: Sp3ck
2 Replies

4. Shell Programming and Scripting

search multiple patterns

I have two lists in a file that look like a b b a e f c d f e d c I would like a final list a b c d e f I've tried multiple grep and awk but can't get it to work (8 Replies)
Discussion started by: godzilla07
8 Replies

5. Shell Programming and Scripting

How to search Multiple patterns in unix

Hi, I tried to search multiple pattern using awk trans=1234 reason=LN MISMATCH rec=`awk '/$trans/ && /'"$reason"'/' file` whenevr i tried to run on command promt it is executing but when i tried to implment same logic in shell script,it is failing i.e $rec is empty ... (6 Replies)
Discussion started by: ns64110
6 Replies

6. Shell Programming and Scripting

Search & Replace: Multiple Strings / Multiple Files

I have a list of files all over a file system e.g. /home/1/foo/bar.x /www/sites/moose/foo.txtI'm looking for strings in these files and want to replace each occurrence with a replacement string, e.g. if I find: '#@!^\&@ in any of the files I want to replace it with: 655#@11, etc. There... (2 Replies)
Discussion started by: spacegoose
2 Replies

7. Shell Programming and Scripting

Grep from multiple patterns multiple file multiple output

Hi, I want to grep multiple patterns from multiple files and save to multiple outputs. As of now its outputting all to the same file when I use this command. Input : 108 files to check for 390 patterns to check for. output I need to 108 files with the searched patterns. Xargs -I {} grep... (3 Replies)
Discussion started by: Diya123
3 Replies

8. Shell Programming and Scripting

Search patterns in multiple logs parallelly.

Hi All, I am starting a service which will redirect its out put into 2 logs say A and B. Now for succesful startup of the service i need to search pattern1 in log A and pattern2 in log B which are writen continuosly. Now my requirement is to find the patterns in the increasing logs A and B... (19 Replies)
Discussion started by: Girish19
19 Replies

9. Shell Programming and Scripting

Search Multiple patterns and display

Hi, I have scenario like below and need to search for multiple patterns Eg: Test Time Started= secs Time Ended = secc Green test Test Time Started= secs Time Ended = secc Green test Output: I need to display the text starting with Test and starting with Time... (2 Replies)
Discussion started by: weknowd
2 Replies

10. UNIX for Beginners Questions & Answers

Issue with search and replacing multiple items in multiple files

Im having an issue when trying to replace the first column with a new set of values in multiple files. The results from the following code only replaces the files with the last set of values in val.txt. I want to replace all the files with all the values. for date in {1..31} do for val in... (1 Reply)
Discussion started by: ncwxpanther
1 Replies
GET-OUI(1)						      General Commands Manual							GET-OUI(1)

NAME
get-oui - Fetch the arp-scan OUI file from the IEEE website SYNOPSIS
get-oui [options] DESCRIPTION
get-oui fetches the Ethernet OUI file from the IEEE website, and saves it in the format used by arp-scan. The OUI file contains all of the OUIs (Organizationally Unique Identifiers) that have been registered with IEEE. Each OUI entry in the file specifies the first 24-bits of the 48-bit Ethernet hardware address, leaving the remaining 24-bits for use by the registering organi- sation. For example the OUI entry "080020", registered to Sun Microsystems, applies to any Ethernet hardware address from 08:00:20:00:00:00 to 08:00:20:ff:ff:ff inclusive. Each OUI assignment represents a total of 2^24 (16,777,216) Ethernet addresses. Every major Ethernet hardware vendor registers an OUI for their equipment, and larger vendors will need to register more than one. For example, 3Com have a total of 37 OUI entries. Organisations that only produce a small number of Ethernet devices will often obtain an IAB registration instead. See get-iab(1) for details. This script can be used to update the arp-scan OUI file from the latest data on the IEEE website. Most of the Ethernet addresses in use belong to an OUI registration, so this is the most important of the files that arp-scan uses to decode Ethernet hardware addresses. You should therefore run get-oui occasionally to keep the arp-scan OUI file up to date. The OUI data is fetched from the URL http://standards.ieee.org/regauth/oui/oui.txt and the output file is saved to the file ieee-oui.txt in the current directory. The URL to fetch the data from can be changed with the -u option, and the output file name can be changed with the -f option. The ieee-oui.txt file that is produced by this script is used by arp-scan to determine the Ethernet card vendor from its hardware address. The directory that arp-scan will look for the ieee-oui.txt file depends on the options used when it was built. If it was built using the default options, then it will look in /usr/local/share/arp-scan. OPTIONS
-h Display a brief usage message and exit. -f <fn> Write the output to the specified file instead of the default ieee-oui.txt. -u <URL> Use the specified URL to fetch the raw OUI data from instead of the default http://standards.ieee.org/regauth/oui/oui.txt. -v Display verbose progress messages. FILES
ieee-oui.txt The default output file. EXAMPLES
$ get-oui -v Renaming ieee-oui.txt to ieee-oui.txt.bak Fetching OUI data from http://standards.ieee.org/regauth/oui/oui.txt Fetched 1467278 bytes Opening output file ieee-oui.txt 9274 OUI entries written to file ieee-oui.txt NOTES
get-oui is implemented in Perl, so you need to have the Perl interpreter installed on your system to use it. get-oui uses the LWP::Simple Perl module to fetch the data from the IEEE website. You must have this module installed on your system for it to work. This module is available on most distributions, often called libwww-perl. It is also available in source form from CPAN. You can use a proxy server by defining the http_proxy environment variable. AUTHOR
Roy Hills <Roy.Hills@nta-monitor.com> SEE ALSO
arp-scan(1) get-iab(1) arp-fingerprint(1) http://www.nta-monitor.com/wiki/ The arp-scan wiki page. March 30, 2007 GET-OUI(1)
All times are GMT -4. The time now is 06:43 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy