11-26-2008
grep -f CPU performances
Hi
I would like to thank you all for this excellent forum.
Today i tried to compare two files and i get some problem with it.
I have two files and i want to get all the data that match the first file like this
File1 (pattern file)
___________________________
9007
9126
9918
9127
9977
___________________________
File 2
_______________________________
9124 2008-12-11 16:00:00
4963 2007-12-16 17:00:00
9126 2006-11-11 16:00:00
9127 2007-12-10 17:00:00
3912 2008-10-11 18:00:00
______________________________
This is how the output file should be
________________________________
9127 2007-12-10 17:00:00
9126 2006-11-11 16:00:00
________________________________
The first file has more than 50000 line and the second file has more than 600000 lines.
I used " grep -f file1 file2 > output.file "
but this take to long I let it running at my Intel@2x1.8GHz(processor load 100% by grep) for 3 hour but i don't get any results.
I also tried to split the first file (pattern file) into smaller parts, but again no results after 3 hours waiting.
this is the script that i used to split the file and to "grep -f"
_________________________________________________
split -l 100 file1 file1.split.
for CHUNK in file1.split.* ; do
grep -f "$CHUNK" file2
done
rm file1.split.*
_________________________________________________
Does someone know how i can do that faster or does anyone has an idea how it can be done faster?
Thanks in advance.
10 More Discussions You Might Find Interesting
1. AIX
Helo:
We updated form AIX 4.3.3 to AIX 5.1-7 and after this we spent more than double time in read from external disks. Aparently the ssa cards microcode is at last level and all the ptf and apars are instaled.
Out backups expent more than double time, but curiously in read only, if we write in... (0 Replies)
Discussion started by: Javier Gutierre
0 Replies
2. UNIX for Dummies Questions & Answers
Hi,
I have one basic doubt, that using grep command frequently , will it have direct impact on the CPU load, pls clarify
for eg, if i run a non stop script containing while loop to grep some parameters, what will be the load in CPU..
thanks (3 Replies)
Discussion started by: vasikaran
3 Replies
3. UNIX for Dummies Questions & Answers
hi,
i want to know cpu utilizatiion per process per cpu..for single processor also if multicore in linux ..to use these values in shell script to kill processes exceeding cpu utilization.ps (pcpu) command does not give exact values..top does not give persistant values..psstat,vmstat..does njot... (3 Replies)
Discussion started by: pankajd
3 Replies
4. Solaris
Hello Friends,
On one of my Solaris 10 box, CPU usage shows 100% using "sar", "vmstat". However, it has 4 CPUs and prstat and glance are not showing enough processes to justify high CPU utilization.
=========================================================================
$ prstat -a
... (4 Replies)
Discussion started by: mahive
4 Replies
5. Solaris
Hello every body,
Maybe someone could help me.
I have a SUN Server with 6 disks, each of 150 Gb.
I have mounted the first two disk in mirror (RAID1) for the system files.
I have mounted 3 disks in RAID5 for users file systems.
I kept the last one as spare and I have mounted it standalone... (6 Replies)
Discussion started by: aribault
6 Replies
6. UNIX for Advanced & Expert Users
Hello !
I will use my best english possible to explain my objective. I'm french so pardon for the lack of precision...
So, what i would like to do in shell script (but you will possibly answer ''not possible in script'' have to use low level langage or something like that) is described below.... (3 Replies)
Discussion started by: Gnaag
3 Replies
7. UNIX for Dummies Questions & Answers
guys i need suggestion about how to grep cpu usage and then compare it
example :
if cpu usage <= 40% then print normal and how much cpu usage is
or cpu usage between 40%-65% print normal and much cpu usage is
i've tried like this one but got error
DOMAIN=`uname -n`... (9 Replies)
Discussion started by: ashary
9 Replies
8. UNIX and Linux Applications
How can grep hog your entire cpu? I am absolutely shocked by this. (11 Replies)
Discussion started by: cokedude
11 Replies
9. UNIX for Dummies Questions & Answers
We have a single threaded application which is restricted by CPU usage even though there are multiple CPUs on the server, hence leading to significant performance issues. Is it possible to merge / combine multiple CPUs at OS level so it appear as a single CPU for the application? (6 Replies)
Discussion started by: Dissa
6 Replies
10. UNIX for Beginners Questions & Answers
I read that Entitlement CPU should be set to max 75% compare to Virtual CPU. May I know the reason.
I have set the Entitlement CPU = Virtual CPU on AIX . It works fine .
Can you help to understand. (1 Reply)
Discussion started by: gabhanes
1 Replies
ZGREP(1) General Commands Manual ZGREP(1)
NAME
zgrep - search possibly compressed files for a regular expression
SYNOPSIS
zgrep [ grep_options ] [ -e ] pattern filename...
DESCRIPTION
Zgrep invokes grep on compressed or gzipped files. These grep options will cause zgrep to terminate with an error code:
(-[drRzZ]|--di*|--exc*|--inc*|--rec*|--nu*). All other options specified are passed directly to grep. If no file is specified, then the
standard input is decompressed if necessary and fed to grep. Otherwise the given files are uncompressed if necessary and fed to grep.
If the GREP environment variable is set, zgrep uses it as the grep program to be invoked.
EXIT CODE
2 - An option that is not supported was specified.
AUTHOR
Charles Levert (charles@comm.polymtl.ca)
SEE ALSO
grep(1), gzexe(1), gzip(1), zdiff(1), zforce(1), zmore(1), znew(1)
ZGREP(1)