Are you sure of that?
Shell expands "/etc/*" to any file and folder beneath /etc directory. "grep -r" recursively checks in those directories. So if you do not have permission to get into the directory, how else would you access files/folders beneath it?
Code:
So what? That doesn't hang the system. Linux is a pre-emptive kernel. If a process wastes too much time, it will be stopped so something else can go. At worst grep could've lagged the system from using 100% CPU. It wouldn't freeze it.
You are right. Linux indeed gives priority to interactive processes when "preemptive"-ness is compiled into the kernel. On server platform, as you are less likely to use interactive programs, preemption is not set on some distro's kernel. Take a look at my RHEL 6.2 kernel config:
Code:
# uname -a
Linux blue 2.6.32-220.el6.x86_64 #1 SMP Wed Nov 9 08:03:13 EST 2011 x86_64 x86_64 x86_64 GNU/Linux
# grep -i preempt /boot/config-2.6.32-220.el6.x86_64
# CONFIG_TREE_PREEMPT_RCU is not set
CONFIG_PREEMPT_NOTIFIERS=y
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
So unless you are compiling a vanilla kernel or using a desktop platform, you are highly likely using non-preemptive kernel
Server distributions often lower the tick rate so pre-emption happens less often, but it'd be ridiculous to turn it off. Anyone who's ever used windows 3.1 knows the consequences of that, and allowing one faulty usermode process to bomb your system is beyond stupid. I don't accept that option means what you think it does. I'll investigate in more detail myself.
There's an enormous problem with your explanation, anyway. His system did, in fact, pre-empt grep. How else could he have regained control by ctrl-alt-f1? Why would it pre-empt for ctrl-alt-f1, and not for the GUI? His system wasn't hardlocked. The GUI had malfunctioned.
---------- Post updated at 10:19 AM ---------- Previous update was at 08:51 AM ----------
I'm pretty sure you're talking about kernel-mode pre-emption, which is something different. It allows the kernel to pre-empt itself in some circumstances so it can respond more quickly and smoothly to realtime requests. This is often left disabled for servers, because it may trade some performance for response time.
This has nothing to do with the pre-emption of user-mode programs. A linux system that lacks user-mode pre-emption would not be stable or sane.
So I stand by my earlier point: grep should not have been able to freeze a system by consuming 100% CPU. Something else is going wrong with this system. If this is the system I think it is, COKEDUDE has had strange problems with this system before, like random segmentation faults in perfectly good programs, so I don't trust that grep itself is the issue.
The "grep" is searching everything under /etc (including pipes or whatever). The scope of the search needs fixing such that "grep" only searches ascii text files.
Whether 100% cpu means a lot depends on how many cpus you have fitted. To achieve 100% cpu on a single cpu and run a program to report the value is pretty unlikely.
This will search every inode in directory /etc whether or not that inode is suitable for "grep". You can even find pipes under /etc .
On my system it would try to search over 8000 inodes.
As a minimum, consider limiting the list by using "find -xdev -type f" and then further checking for text files with the "file" command.
Can you please explain what -xdev -and -type f do in in find?
I was reading this and don't understand.
Aren't all files considered regular files or directories? Can you explain the
different filetypes?
Code:
-type c
File is of type c:
b block (buffered) special
c character (unbuffered) special
d directory
p named pipe (FIFO)
f regular file
l symbolic link; this is never true if the -L option or the
-follow option is in effect, unless the symbolic link is
broken. If you want to search for symbolic links when -L
is in effect, use -xtype.
s socket
D door (Solaris)
Wouldn't giving it the directory /etc prevent it from descending into another filesystem?
Code:
-xdev Don't descend directories on other filesystems.
Is there any particular reason you were running it as root? Nearly nothing in /etc/ needs root to be read. Using root needlessly is dangerous.
grep has no power to freeze your GUI. It only gets as much CPU time as it's given, it can't hog it. This was an OS or GUI crash of some sort.
Is this the same system you were having severe system instability with before? Did you ever really track that down?
Yes, I created the file as root and didn't give anyone else read permissions. root is the only way to create files in the /etc directory. Also many files in fedora are readonly by root only.
Grep did freeze my gui. I tested 3 times on a fresh startup. I also tested 3 times after leaving my computer running for an hour. The same thing happened all 6 times.
I reformatted. I also took it to 2 different computer stores and neither could find a problem.
Quote:
Originally Posted by admin_xor
You better use find as suggested. When I forget (that happens more often) about the location of any config file or any such thing, but somehow if I remember any unique word in that file I do this:
Code:
find /etc -xdev -type f | xargs grep -i "unique_word"
This is lot faster.
@Corona688
Some directories in /etc are indeed restricted for root. This depends on what UNIX/Linux variant you are using and the installed software/packages. See below:
Code:
##for a RHEL 6.1
$ ls -ltr /etc/* 2>err
$ cat err
ls: cannot open directory /etc/sudoers.d: Permission denied
ls: cannot open directory /etc/dhcp: Permission denied
ls: cannot open directory /etc/named: Permission denied
ls: cannot open directory /etc/sssd: Permission denied
ls: cannot open directory /etc/audisp: Permission denied
ls: cannot open directory /etc/audit: Permission denied
##for a Solaris11
$ ls -ltr /etc/* 2>err
$ cat err
/etc/sudoers.d: Permission denied
##for a FreeBSD
$ ls -ltr /etc/* 2>err
$ cat err
ls: /etc/ntp: Permission denied
What if the file you are searching are in one of those directories.
Also, my RHEL 6.1 VM took 100% of CPU time of 4 allocated cores (with only 953 inodes in /etc directory):
So there's a pretty good reason why your GUI got stuck as grep has to run a loop for each inode (that includes directories as well which is absolutely meaning less in this case).
Use find!!!
The find trick seemed way faster when it works but it doesn't seem reliable. The second time I saw it was searching binary files and there is no point of the that so I added the -I option to grep. The third time it looks like it skipped /etc directory.
So what? That doesn't hang the system. Linux is a pre-emptive kernel. If a process wastes too much time, it will be stopped so something else can go. At worst grep could've lagged the system from using 100% CPU. It wouldn't freeze it.
Back in my college days, a fellow student accidentally left behind a process which due to a bug did nothing but eat 100% CPU. Several weeks later, it was still there. Nobody noticed it until I ran 'top'. It hadn't interfered with people's use of the system at all.
Yes I am. The file is somewhere in the /etc directory.
I only have 2 cores.
Quote:
Originally Posted by admin_xor
Are you sure of that?
Shell expands "/etc/*" to any file and folder beneath /etc directory. "grep -r" recursively checks in those directories. So if you do not have permission to get into the directory, how else would you access files/folders beneath it?
Quote:
So what? That doesn't hang the system. Linux is a pre-emptive kernel. If a process wastes too much time, it will be stopped so something else can go. At worst grep could've lagged the system from using 100% CPU. It wouldn't freeze it.
You are right. Linux indeed gives priority to interactive processes when "preemptive"-ness is compiled into the kernel. On server platform, as you are less likely to use interactive programs, preemption is not set on some distro's kernel. Take a look at my RHEL 6.2 kernel config:
Code:
# uname -a
Linux blue 2.6.32-220.el6.x86_64 #1 SMP Wed Nov 9 08:03:13 EST 2011 x86_64 x86_64 x86_64 GNU/Linux
# grep -i preempt /boot/config-2.6.32-220.el6.x86_64
# CONFIG_TREE_PREEMPT_RCU is not set
CONFIG_PREEMPT_NOTIFIERS=y
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
So unless you are compiling a vanilla kernel or using a desktop platform, you are highly likely using non-preemptive kernel
Your right. Its not set for me either.
Code:
$ uname -a
Linux hi 2.6.41.9-1.fc15.i686 #1 SMP Fri Jan 13 16:43:37 UTC 2012 i686 i686 i386 GNU/Linux
$ grep -i preempt /boot/config-2.6.41.9-1.fc15.i686
# CONFIG_PREEMPT_RCU is not set
CONFIG_PREEMPT_NOTIFIERS=y
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
Quote:
Originally Posted by Corona688
Server distributions often lower the tick rate so pre-emption happens less often, but it'd be ridiculous to turn it off. Anyone who's ever used windows 3.1 knows the consequences of that, and allowing one faulty usermode process to bomb your system is beyond stupid. I don't accept that option means what you think it does. I'll investigate in more detail myself.
There's an enormous problem with your explanation, anyway. His system did, in fact, pre-empt grep. How else could he have regained control by ctrl-alt-f1? Why would it pre-empt for ctrl-alt-f1, and not for the GUI? His system wasn't hardlocked. The GUI had malfunctioned.
---------- Post updated at 10:19 AM ---------- Previous update was at 08:51 AM ----------
I'm pretty sure you're talking about kernel-mode pre-emption, which is something different. It allows the kernel to pre-empt itself in some circumstances so it can respond more quickly and smoothly to realtime requests. This is often left disabled for servers, because it may trade some performance for response time.
This has nothing to do with the pre-emption of user-mode programs. A linux system that lacks user-mode pre-emption would not be stable or sane.
So I stand by my earlier point: grep should not have been able to freeze a system by consuming 100% CPU. Something else is going wrong with this system. If this is the system I think it is, COKEDUDE has had strange problems with this system before, like random segmentation faults in perfectly good programs, so I don't trust that grep itself is the issue.
I have several computers at work and school I can use.
Quote:
Originally Posted by methyl
The "grep" is searching everything under /etc (including pipes or whatever). The scope of the search needs fixing such that "grep" only searches ascii text files.
Whether 100% cpu means a lot depends on how many cpus you have fitted. To achieve 100% cpu on a single cpu and run a program to report the value is pretty unlikely.
What command do you recommend? The find trick doesn't seem reliable.
I have a dual core on the computer I am talking about.
@COKEDUDE
It's time to read up and learn how to search only inodes on the current filesystem which are ascii text files.
When you come up with your original script, please post it on this forum for the benefit of others.
Quote:
I have several computers at work and school I can use.
Don't forget to post in the Homework Forum if your post is nothing to do with your daytime work.
Let me give you a complete example what I am trying to achieve.
1. Below is the log file structure where I need 2,5 and 14th column of the logs after grepping through the linkId=1ddoic.
Log file structure:-
abc.com 20120829001415 127.0.0.1 app none11111 sas 0 0 N clk Mozilla/5.0... (3 Replies)
As per my understanding below mentioned line of code finding a word 'boy' in $ACULOG...
num_errors=`grep -i -e fail -e illegal -e exception -e "<E" -e boy $ACULOG | wc -l`
if I'm not corerct, please correct me. How I can find entire line like "This is a boy" with something similar as above... (1 Reply)
As per my understanding below mentioned line of code finding a word 'boy' in $ACULOG...
num_errors=`grep -i -e fail -e illegal -e exception -e "<E" -e boy $ACULOG | wc -l`
if I'm not corerct, please correct me. How I can find entire line like "This is a boy" with something similar as above... (1 Reply)
Okay, I am trying to come up with a multi-platform script to report top ten CPU and memory hog processes, which will be run by our enterprise monitoring application as an auto-action item when the CPU and Memory utilization gets reported as higher than a certain threshold
I use top on other... (5 Replies)
Hi friends,
This is my very first post on forum, so kindly excuse if my doubts are found too silly.
I am trying to automate a piece of routine work and this is where I am stuck at the moment-I need to grep a particular ID through a file containing many records(which start with <LRECORD> and end... (6 Replies)
Sorry for the title, I really don't know how to word this question or what to even search for. I tried "grep one match", "grep 1 match", "stop grep" on both google and here and haven't found something that helps, so here I go:
I have a file that's about 1.5 million lines long, every line looks... (3 Replies)
I'm having problems since few days ago, and i'm not able to make it works with a simple awk+grep script (or other way to do this).
For example, i have a input file1.txt:
cat inputfile1.txt
218299910417
1172051195
1172070231
1172073514
1183135117
1183135118
1183135119
1281440202
... (3 Replies)
Hi
I would like to thank you all for this excellent forum.
Today i tried to compare two files and i get some problem with it.
I have two files and i want to get all the data that match the first file like this
File1 (pattern file)
___________________________
9007
9126
9918
9127 ... (6 Replies)
(extract from SQL binlog file...)
# at 4960
#080801 14:35:31 server id 4 end_log_pos 195 Query thread_id=63121426 exec_time=0 error_code=0
use d_jds;
SET TIMESTAMP=1217581531;
UPDATE bid_details set bidding = 3170.37 ,deduction=if((3170.37 < 37.43),0,deduction)
where... (3 Replies)
Hi,
I have one basic doubt, that using grep command frequently , will it have direct impact on the CPU load, pls clarify
for eg, if i run a non stop script containing while loop to grep some parameters, what will be the load in CPU..
thanks (3 Replies)
01-17-2012
