01-23-2011
What Operating System and version are you running? This is very important. I haven't seen the "fork" error in many years.
We gather that you have ksh.
Quote:
#!/usr/bin/ksh
for num in `cat file1.txt`
do
find . -name "*processed" -print | xargs gunzip -c | grep -q $num || echo "$num not found" >> outputfile.txt &
done
The script posted makes no sense because it does not search for zipped files.
If "file1.txt" contains 5,000,000 numbers this level of blunt processing is absurd in unix Shell when searching 5Gb of data.
You appear to want to search a specific field at a specific position within a record but have provided sample data in "file1.txt" which does not match the exact length of the highlighted field.
Do you have use of a professional Systems Analyst?
Do you have a database engine (e.g. Oracle) and use of professional Database Programmers?
IMHO you are way out of you depth. Hire a professional.
The background "&" within a 5,000,000 iteration loop is why you are getting "fork" errors. It would take a seriously special kernel build to create a unix which could cope with 5,000,000 concurrent processes (hmm. temptied to try it). I am surprised that you did not crash the computer with this irresponsible, uninformed and ignorant code.
Ps: Given a decent commercial database engine and some top-class Database Programmers this problem is solveable.
Last edited by methyl; 01-23-2011 at 06:46 PM..
Reason: Footnote:, layout and more layout
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I have one file:
123*100*abcd*10
123*101*abcd*-29*def
123*100*abcd*-10
123*102*abcd*-105*asd
I would like to parameterize the search patterns in the following way so that the user could dynamically change the search pattern.
*100* and *- (ie *minus)
*102* and *-
The output that is... (6 Replies)
Discussion started by: augustinep
6 Replies
2. Shell Programming and Scripting
Hi
I'm not very good with the serach patterns and I'd need a sample how to find a line that has multiple patterns.
Say I want to find a line that has "abd", "123" and "QWERTY" and there can be any characters or numbers between the serach patterns, I have a file that has thousands of lines and... (10 Replies)
Discussion started by: Juha
10 Replies
3. Shell Programming and Scripting
Good day, great gurus,
I'm new to Perl, and programming in general. I'm trying to retrieve a column of data from my text file which spans a non-specific number of lines. So I did a regexp that will pick out the columns. However,my pattern would vary. I tried using a foreach loop unsuccessfully.... (2 Replies)
Discussion started by: Sp3ck
2 Replies
4. Shell Programming and Scripting
I have two lists in a file that look like
a b
b a
e f
c d
f e
d c
I would like a final list
a b
c d
e f
I've tried multiple grep and awk but can't get it to work (8 Replies)
Discussion started by: godzilla07
8 Replies
5. Shell Programming and Scripting
Hi,
I tried to search multiple pattern using awk
trans=1234
reason=LN MISMATCH
rec=`awk '/$trans/ && /'"$reason"'/' file`
whenevr i tried to run on command promt it is executing but when i tried to implment same logic in shell script,it is failing i.e $rec is empty
... (6 Replies)
Discussion started by: ns64110
6 Replies
6. Shell Programming and Scripting
I have a list of files all over a file system e.g.
/home/1/foo/bar.x
/www/sites/moose/foo.txtI'm looking for strings in these files and want to replace each occurrence with a replacement string, e.g.
if I find: '#@!^\&@ in any of the files I want to replace it with: 655#@11, etc.
There... (2 Replies)
Discussion started by: spacegoose
2 Replies
7. Shell Programming and Scripting
Hi,
I want to grep multiple patterns from multiple files and save to multiple outputs. As of now its outputting all to the same file when I use this command.
Input : 108 files to check for 390 patterns to check for. output I need to 108 files with the searched patterns.
Xargs -I {} grep... (3 Replies)
Discussion started by: Diya123
3 Replies
8. Shell Programming and Scripting
Hi All,
I am starting a service which will redirect its out put into 2 logs say A and B.
Now for succesful startup of the service i need to search pattern1 in log A and pattern2 in log B which are writen continuosly.
Now my requirement is to find the patterns in the increasing logs A and B... (19 Replies)
Discussion started by: Girish19
19 Replies
9. Shell Programming and Scripting
Hi,
I have scenario like below and need to search for multiple patterns
Eg:
Test
Time Started= secs
Time Ended = secc
Green test
Test
Time Started= secs
Time Ended = secc
Green test
Output:
I need to display the text starting with Test and starting with Time... (2 Replies)
Discussion started by: weknowd
2 Replies
10. UNIX for Beginners Questions & Answers
Im having an issue when trying to replace the first column with a new set of values in multiple files. The results from the following code only replaces the files with the last set of values in val.txt. I want to replace all the files with all the values.
for date in {1..31}
do
for val in... (1 Reply)
Discussion started by: ncwxpanther
1 Replies
LEARN ABOUT DEBIAN
ogmsplit
OGMSPLIT(1) User Commands OGMSPLIT(1)
NAME
ogmsplit - Split OGG/OGM files into several smaller OGG/OGM files
SYNOPSIS
ogmsplit [options] inname
DESCRIPTION
ogmsplit can be used to easily split an OGM file after a given size. Several OGM files will be created that each start with a keyframe.
inname Use 'inname' as the source.
-o, --output out
Use 'out' as the base name. Ascending part numbers will be appended to it. Default is 'inname'. Examples:
1) If -o output.ogg is given on the command line then ogmsplit will create output-000001.ogg, output-000002.ogg and so on.
2) If no -o option is given and the input's name is movie.ogm then ogmsplit will create movie-000001.ogm and so on.
The operation mode can be set with exactly one of -s, -t, -c or -p. The default mode is to split by size (-s).
-s, --size size
Size in MiB ( = 1024 * 1024 bytes) after which a new file will be opened (approximately). Default is 700MiB. Size can end in 'B' to
indicate 'bytes' instead of 'MiB'.
-t, --time time
Split after the given elapsed time (approximately). 'time' takes the form HH:MM:SS.sss or simply SS(.sss), e.g. 00:05:00.000 or
300.000 or simply 300.
-c, --cuts cuts
Produce output files as specified by cuts, a list of slices of the form "start-end" or "start+length", separated by commas. If start
is omitted, it defaults to the end of the previous cut. start and end take the same format as the arguments to -t.
-n, --num num
Don't create more than num separate files. The last one may be bigger than the desired size. Default is an unlimited number of
files. Can only be used with -s or -t.
--frontend
Frontend mode. Progress output will be terminated by
instead of
.
-p, --print-splitpoints
Only print the key frames and the number of bytes encountered before each. Useful to find the exact splitting point.
-v, --verbose
Be verbose and show each OGG packet. Can be used twice to increase verbosity.
-h, --help
Show this help.
-V, --version
Show version information.
CHAPTER INFORMATION
ogmsplit correctly handles chapter information. During the first pass the chapter information, if any is present, will be adjusted to match
the output files generated. Chapters that are not contained in the current output file are removed entirely. The other chapters are renum-
bered to start at 1, and their timestamps will be recalculated.
Example: If your source file contains these four chapters:
CHAPTER01=00:00:00.000
CHAPTER01NAME=Chapter 01
CHAPTER02=00:10:00.000
CHAPTER02NAME=Chapter 02
CHAPTER03=00:20:00.000
CHAPTER03NAME=Chapter 03
CHAPTER04=00:25:00.000
CHAPTER04NAME=Chapter 04
and you split after 15 minutes, then the first output file will only contain the first two chapters as shown above, and the second output
file will contain the following two chapters and the remaining part of the first:
CHAPTER01=00:00:00.000
CHAPTER01NAME=Chapter 02 (continued)
CHAPTER02=00:05:00.000
CHAPTER02NAME=Chapter 03
CHAPTER03=00:10:00.000
CHAPTER03NAME=Chapter 04
Note that only variable names are changed, not the chapter names themselves. The exception is the first chapter of the second and follow-
ing files where "(continued)" is appended in order to indicate that this is not the start of this chapter. If you want to change them as
well you'll have to remerge the resulting file with a new chapter file.
AUTHOR
ogmsplit was written by Moritz Bunkus <moritz@bunkus.org>.
SEE ALSO
ogmmerge(1), ogminfo(1), ogmdemux(1), ogmcat(1), dvdxchap(1)
WWW
The newest version can always be found at <http://www.bunkus.org/videotools/ogmtools/> <http://www.bunkus.org/videotools/ogmtools/>
ogmsplit v1.5 November 2004 OGMSPLIT(1)