Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Search multiple patterns in multiple files
Top Forums Shell Programming and Scripting Search multiple patterns in multiple files Post 302490046 by methyl on Sunday 23rd of January 2011 05:10:42 PM
Old 01-23-2011
What Operating System and version are you running? This is very important. I haven't seen the "fork" error in many years.
We gather that you have ksh.

Quote:
#!/usr/bin/ksh
for num in `cat file1.txt`
do
find . -name "*processed" -print | xargs gunzip -c | grep -q $num || echo "$num not found" >> outputfile.txt &
done
The script posted makes no sense because it does not search for zipped files.


If "file1.txt" contains 5,000,000 numbers this level of blunt processing is absurd in unix Shell when searching 5Gb of data.
You appear to want to search a specific field at a specific position within a record but have provided sample data in "file1.txt" which does not match the exact length of the highlighted field.


Do you have use of a professional Systems Analyst?
Do you have a database engine (e.g. Oracle) and use of professional Database Programmers?

IMHO you are way out of you depth. Hire a professional.



The background "&" within a 5,000,000 iteration loop is why you are getting "fork" errors. It would take a seriously special kernel build to create a unix which could cope with 5,000,000 concurrent processes (hmm. temptied to try it). I am surprised that you did not crash the computer with this irresponsible, uninformed and ignorant code.


Ps: Given a decent commercial database engine and some top-class Database Programmers this problem is solveable.
Last edited by methyl; 01-23-2011 at 06:46 PM.. Reason: Footnote:, layout and more layout
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to parameterize multiple search patterns and generate a new file

I have one file: 123*100*abcd*10 123*101*abcd*-29*def 123*100*abcd*-10 123*102*abcd*-105*asd I would like to parameterize the search patterns in the following way so that the user could dynamically change the search pattern. *100* and *- (ie *minus) *102* and *- The output that is... (6 Replies)
Discussion started by: augustinep
6 Replies

2. Shell Programming and Scripting

Perl: Match a line with multiple search patterns

Hi I'm not very good with the serach patterns and I'd need a sample how to find a line that has multiple patterns. Say I want to find a line that has "abd", "123" and "QWERTY" and there can be any characters or numbers between the serach patterns, I have a file that has thousands of lines and... (10 Replies)
Discussion started by: Juha
10 Replies

3. Shell Programming and Scripting

Perl - How to search a text file with multiple patterns?

Good day, great gurus, I'm new to Perl, and programming in general. I'm trying to retrieve a column of data from my text file which spans a non-specific number of lines. So I did a regexp that will pick out the columns. However,my pattern would vary. I tried using a foreach loop unsuccessfully.... (2 Replies)
Discussion started by: Sp3ck
2 Replies

4. Shell Programming and Scripting

search multiple patterns

I have two lists in a file that look like a b b a e f c d f e d c I would like a final list a b c d e f I've tried multiple grep and awk but can't get it to work (8 Replies)
Discussion started by: godzilla07
8 Replies

5. Shell Programming and Scripting

How to search Multiple patterns in unix

Hi, I tried to search multiple pattern using awk trans=1234 reason=LN MISMATCH rec=`awk '/$trans/ && /'"$reason"'/' file` whenevr i tried to run on command promt it is executing but when i tried to implment same logic in shell script,it is failing i.e $rec is empty ... (6 Replies)
Discussion started by: ns64110
6 Replies

6. Shell Programming and Scripting

Search & Replace: Multiple Strings / Multiple Files

I have a list of files all over a file system e.g. /home/1/foo/bar.x /www/sites/moose/foo.txtI'm looking for strings in these files and want to replace each occurrence with a replacement string, e.g. if I find: '#@!^\&@ in any of the files I want to replace it with: 655#@11, etc. There... (2 Replies)
Discussion started by: spacegoose
2 Replies

7. Shell Programming and Scripting

Grep from multiple patterns multiple file multiple output

Hi, I want to grep multiple patterns from multiple files and save to multiple outputs. As of now its outputting all to the same file when I use this command. Input : 108 files to check for 390 patterns to check for. output I need to 108 files with the searched patterns. Xargs -I {} grep... (3 Replies)
Discussion started by: Diya123
3 Replies

8. Shell Programming and Scripting

Search patterns in multiple logs parallelly.

Hi All, I am starting a service which will redirect its out put into 2 logs say A and B. Now for succesful startup of the service i need to search pattern1 in log A and pattern2 in log B which are writen continuosly. Now my requirement is to find the patterns in the increasing logs A and B... (19 Replies)
Discussion started by: Girish19
19 Replies

9. Shell Programming and Scripting

Search Multiple patterns and display

Hi, I have scenario like below and need to search for multiple patterns Eg: Test Time Started= secs Time Ended = secc Green test Test Time Started= secs Time Ended = secc Green test Output: I need to display the text starting with Test and starting with Time... (2 Replies)
Discussion started by: weknowd
2 Replies

10. UNIX for Beginners Questions & Answers

Issue with search and replacing multiple items in multiple files

Im having an issue when trying to replace the first column with a new set of values in multiple files. The results from the following code only replaces the files with the last set of values in val.txt. I want to replace all the files with all the values. for date in {1..31} do for val in... (1 Reply)
Discussion started by: ncwxpanther
1 Replies

LEARN ABOUT DEBIAN

oakdecode

oakdecode(1)						      General Commands Manual						      oakdecode(1)

NAME

       oakdecode - Decode an OAKT printer stream into human readable form.

SYNOPSIS

       oakdecode [options] <OAKT-file

DESCRIPTION

       oakdecode decodes an OAKT printer stream into human readable form.

       An OAKT printer stream is the printer langauge used by the HP Color LaserJet 1500 and other printers.

COMMAND LINE OPTIONS

       These are the options that can appear on the command line.

       -d basename
	      Basename of .pbm file for saving decompressed planes.

       -r basename
	      Basename of .jbg file for saving raw planes

       -i     Suppress display of image records.

       -o     Print file offsets.

       -D level
	      Set Debug level [0].

EXAMPLES

       Decode an OAKT file created by foo2oak.

       $ oakdecode < testpage.oak
       0d(80) 1 OTHER
       0c(64) Wed Nov 05 16:30:50 2003        a07d3   100005  32001e
       0a(80) testpage.pdf
       14(16) (no args)
       28(16) Source=Tray1
       29(80) PaperType=0 UNK8=2,0,0,0, blanks(63)
       2a(32) Copies=1        UNK=0
       2b(32) papercode=25    xwid=4648       ywid=9000       UNK=0
       33(64)
	       u0      u1      w       h       resx    resy    nBits
	       x0      x0      2128    4300    600     600     x1
       15(16) (no args)
	    bih0     w	   h	l0     bih5   dlen  plen unk yOff P subP
       3c(64) 00010000 2176  256  256 58030020  1050  1056 000   64 3 0
		       DL = 0, D = 0, P = 1, - = 0, XY = 2176 x 256
		       L0 = 256, MX = 32, MY = 0
		       Order   = 3  ILEAVE SMID
		       Options = 88  LRLTWO TPDON TPBON
		       1 stripes, 0 layers, 1 planes
       3c(64) 00010000 2176  256  256 58030020  3668  3680 000  320 3 0
       3c(64) 00010000 2176  256  256 58030020  1463  1472 000  640 3 0
       3c(64) 00010000 2176  256  256 58030020  1975  1984 000  896 3 0
       3c(64) 00010000 2176  224  224 58030020  2744  2752 000 1152 3 0
       3c(64) 00010000 2176  256  256 58030020   988	992 000 1440 3 0
       3c(64) 00010000 2176  256  256 58030020  2892  2896 000 1696 3 0
       3c(64) 00010000 2176  256  256 58030020  3634  3648 000 1952 3 0
       3c(64) 00010000 2176  256  256 58030020  3236  3248 000 2208 3 0
       3c(64) 00010000 2176  256  256 58030020  2279  2288 000 2464 3 0
       3c(64) 00010000 2176  256  256 58030020  3746  3760 000 2720 3 0
       3c(64) 00010000 2176  200  200 58030020  2404  2416 000 2976 3 0
       3c(64) 00010000 2176  256  256 58030020  3114  3120 000 3240 3 0
       3c(64) 00010000 2176   96   96 58030020  1142  1152 000 3496 3 0
       3c(64) 00010000 2176  256  256 58030020  2094  2112 000 3752 3 0
       3c(64) 00010000 2176  256  256 58030020  1319  1328 000 4008 3 0
       3c(64) 00010000 2176   36   36 58030020   208	224 000 4264 3 0
       17(16) (no args)
       18(16) UNK=0
       0b(16) (no args)

FILES

       /usr/bin/oakdecode

SEE ALSO

       foo2oak-wrapper(1), foo2oak(1), jbg2pbm(1)

AUTHOR

       Rick Richardson <rick.richardson@comcast.net>
       http://foo2oak.rkkda.com/
All times are GMT -4. The time now is 05:44 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy