01-23-2011
With some tweaking it may still be possible to get this brute force solution to work fast enough, but it's not looking good. I suspect your are running out of physical memory and the system is swapping, how big is the file1.txt and how much physical memory do you have on your system?
You could consider retaining some of the work from previous scans. This really depends on your dataset and leads to the following questions about your data.
How static is it?
I'd assume the zip file contents don't change much, but perhaps you remove old zips and add new ones?
How about the contents of file1.txt is this completely different each night? Are any items searched for searched for again at later dates? (For example if we know the XYZ wasn't in the zips lastnight and it's searched for again, all we need to scan are files added since last night's scan).
Last edited by Chubler_XL; 01-23-2011 at 06:26 PM..
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I have one file:
123*100*abcd*10
123*101*abcd*-29*def
123*100*abcd*-10
123*102*abcd*-105*asd
I would like to parameterize the search patterns in the following way so that the user could dynamically change the search pattern.
*100* and *- (ie *minus)
*102* and *-
The output that is... (6 Replies)
Discussion started by: augustinep
6 Replies
2. Shell Programming and Scripting
Hi
I'm not very good with the serach patterns and I'd need a sample how to find a line that has multiple patterns.
Say I want to find a line that has "abd", "123" and "QWERTY" and there can be any characters or numbers between the serach patterns, I have a file that has thousands of lines and... (10 Replies)
Discussion started by: Juha
10 Replies
3. Shell Programming and Scripting
Good day, great gurus,
I'm new to Perl, and programming in general. I'm trying to retrieve a column of data from my text file which spans a non-specific number of lines. So I did a regexp that will pick out the columns. However,my pattern would vary. I tried using a foreach loop unsuccessfully.... (2 Replies)
Discussion started by: Sp3ck
2 Replies
4. Shell Programming and Scripting
I have two lists in a file that look like
a b
b a
e f
c d
f e
d c
I would like a final list
a b
c d
e f
I've tried multiple grep and awk but can't get it to work (8 Replies)
Discussion started by: godzilla07
8 Replies
5. Shell Programming and Scripting
Hi,
I tried to search multiple pattern using awk
trans=1234
reason=LN MISMATCH
rec=`awk '/$trans/ && /'"$reason"'/' file`
whenevr i tried to run on command promt it is executing but when i tried to implment same logic in shell script,it is failing i.e $rec is empty
... (6 Replies)
Discussion started by: ns64110
6 Replies
6. Shell Programming and Scripting
I have a list of files all over a file system e.g.
/home/1/foo/bar.x
/www/sites/moose/foo.txtI'm looking for strings in these files and want to replace each occurrence with a replacement string, e.g.
if I find: '#@!^\&@ in any of the files I want to replace it with: 655#@11, etc.
There... (2 Replies)
Discussion started by: spacegoose
2 Replies
7. Shell Programming and Scripting
Hi,
I want to grep multiple patterns from multiple files and save to multiple outputs. As of now its outputting all to the same file when I use this command.
Input : 108 files to check for 390 patterns to check for. output I need to 108 files with the searched patterns.
Xargs -I {} grep... (3 Replies)
Discussion started by: Diya123
3 Replies
8. Shell Programming and Scripting
Hi All,
I am starting a service which will redirect its out put into 2 logs say A and B.
Now for succesful startup of the service i need to search pattern1 in log A and pattern2 in log B which are writen continuosly.
Now my requirement is to find the patterns in the increasing logs A and B... (19 Replies)
Discussion started by: Girish19
19 Replies
9. Shell Programming and Scripting
Hi,
I have scenario like below and need to search for multiple patterns
Eg:
Test
Time Started= secs
Time Ended = secc
Green test
Test
Time Started= secs
Time Ended = secc
Green test
Output:
I need to display the text starting with Test and starting with Time... (2 Replies)
Discussion started by: weknowd
2 Replies
10. UNIX for Beginners Questions & Answers
Im having an issue when trying to replace the first column with a new set of values in multiple files. The results from the following code only replaces the files with the last set of values in val.txt. I want to replace all the files with all the values.
for date in {1..31}
do
for val in... (1 Reply)
Discussion started by: ncwxpanther
1 Replies
LEARN ABOUT REDHAT
unix2dos
unix2dos(1) General Commands Manual unix2dos(1)
NAME
unix2dos - UNIX to DOS text file format converter
SYNOPSYS
unix2dos [options] [-c convmode] [-o file ...] [-n infile outfile ...]
Options:
[-hkqV] [--help] [--keepdate] [--quiet] [--version]
DESCRIPTION
This manual page documents unix2dos, the program that converts text files in UNIX format to DOS format.
OPTIONS
The following options are available:
-h --help
Print online help.
-k --keepdate
Keep the date stamp of output file same as input file.
-q --quiet
Quiet mode. Suppress all warning and messages.
-V --version
Prints version information.
-c --convmode convmode
Sets conversion mode. Simulates unix2dos under SunOS.
-o --oldfile file ...
Old file mode. Convert the file and write output to it. The program default to run in this mode. Wildcard names may be used.
-n --newfile infile outfile ...
New file mode. Convert the infile and write output to outfile. File names must be given in pairs and wildcard names should NOT be
used or you WILL lost your files.
EXAMPLES
Get input from stdin and write output to stdout.
unix2dos
Convert and replace a.txt. Convert and replace b.txt.
unix2dos a.txt b.txt
unix2dos -o a.txt b.txt
Convert and replace a.txt in ASCII conversion mode. Convert and replace b.txt in ISO conversion mode.
unix2dos a.txt -c iso b.txt
unix2dos -c ascii a.txt -c iso b.txt
Convert and replace a.txt while keeping original date stamp.
unix2dos -k a.txt
unix2dos -k -o a.txt
Convert a.txt and write to e.txt.
unix2dos -n a.txt e.txt
Convert a.txt and write to e.txt, keep date stamp of e.txt same as a.txt.
unix2dos -k -n a.txt e.txt
Convert and replace a.txt. Convert b.txt and write to e.txt.
unix2dos a.txt -n b.txt e.txt
unix2dos -o a.txt -n b.txt e.txt
Convert c.txt and write to e.txt. Convert and replace a.txt. Convert and replace b.txt. Convert d.txt and write to f.txt.
unix2dos -n c.txt e.txt -o a.txt b.txt -n d.txt f.txt
DIAGNOSTICS
BUGS
The program does not work properly under MSDOS in stdio processing mode. If you know why is that so, please tell me.
AUTHOR
Benjamin Lin - ( blin@socs.uts.edu.au )
MISCELLANY
Tested environment:
Linux 1.2.0 with GNU C 2.5.8
SunOS 4.1.3 with GNU C 2.6.3
MS-DOS 6.20 with Borland C++ 4.02
Suggestions and bug reports are welcome.
SEE ALSO
dos2unix(1)
1995.03.31 unix2dos v2.2 unix2dos(1)