|
|
Sponsored Content
Top Forums
UNIX for Advanced & Expert Users
System call failed with 127 .. after 500 times when called in loop
Post 302591584 by binnyjeshan on Friday 20th of January 2012 01:58:24 AM
01-20-2012
Hello all in group,
I have Fixed
the actual issue that was causing a pain to the poor 'system() api' call 
that i had been doubting on for the past few weeks. The problem wasn't with unavailable memory (that i earlier suspected 
due to uncleaned child processes) etc etc. this magic number of failure "after around 500" gave me clues that either "uncleaned child processes" or its "other limits reached". If it was the first one, then even if the command was executed directly on the shell console, it would fail, but in my case it din't happen that way. So, i concentrated on the second case. Google-d 10's of links about resource limits and learned some facts about NR_TASKS, file descriptor max etc and found numbers like 512, 1024 etc.
The actual problem was like this.
after system("script.sh > file.txt") had been called, there were two other functions that were later executed. In those functions, there was one operation like this...
DIR* dir = opendir("/data/cores");
what has actually happened is, my kernel allowed only 1024 open file descriptors per process (ulimit -n) as default. once the testing started, each loop will open 2 descriptors, but didn't close them. As a result, after approx 500 times, approx of 1024 descriptors was still open (known to the kernel) by the process itself, and after that when system call system(..) API tried to open file.txt using the 'cat' ">" file.txt, it did not get a free descriptor from the kernel. and hence the execl inside the system(.) call API failed, returning 127 error code.
How did i find?
One possible way is to re-compile kernel enabling all debug logs around the glibc areas.
i dint go by the above. I tried to open a file next to system() call api directly using 'open' system call, and that returned NULL. What if you use 'perror" after that failure. The whole thing is there. It coveyed the whole information of what was going wrong.
also later i learnt that in /proc/pid/fd if you see will list all file descriptors that are in use. Ideally there will be 3 or 4 default. in my case, there were 1024 :P
GAME OVER
Thanks
I have Fixed
the actual issue that was causing a pain to the poor 'system() api' call that i had been doubting on for the past few weeks. The problem wasn't with unavailable memory (that i earlier suspected due to uncleaned child processes) etc etc. this magic number of failure "after around 500" gave me clues that either "uncleaned child processes" or its "other limits reached". If it was the first one, then even if the command was executed directly on the shell console, it would fail, but in my case it din't happen that way. So, i concentrated on the second case. Google-d 10's of links about resource limits and learned some facts about NR_TASKS, file descriptor max etc and found numbers like 512, 1024 etc. The actual problem was like this.
after system("script.sh > file.txt") had been called, there were two other functions that were later executed. In those functions, there was one operation like this...
DIR* dir = opendir("/data/cores");
what has actually happened is, my kernel allowed only 1024 open file descriptors per process (ulimit -n) as default. once the testing started, each loop will open 2 descriptors, but didn't close them. As a result, after approx 500 times, approx of 1024 descriptors was still open (known to the kernel) by the process itself, and after that when system call system(..) API tried to open file.txt using the 'cat' ">" file.txt, it did not get a free descriptor from the kernel. and hence the execl inside the system(.) call API failed, returning 127 error code.
How did i find?
One possible way is to re-compile kernel enabling all debug logs around the glibc areas.
i dint go by the above. I tried to open a file next to system() call api directly using 'open' system call, and that returned NULL. What if you use 'perror" after that failure. The whole thing is there. It coveyed the whole information of what was going wrong.
also later i learnt that in /proc/pid/fd if you see will list all file descriptors that are in use. Ideally there will be 3 or 4 default. in my case, there were 1024 :P
GAME OVER
Thanks
|
|
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I have the following shell script to map a drive to a windows machine:
echo Enter password:
stty -echo
read passwd
stty echo
mount -t smbfs -ousername=myusername,password=$passwd //192.168.2.5/sharename /mnt/mountname
It works fine and the share mounts, but for some reason I get this... (4 Replies)
Discussion started by: Spetnik
4 Replies
2. BSD
Hello,
I have a small inquiry.
Sometimes, my good friend, Charlie Root, sends me security notifications that a possible breakin attempt has occured. It looks like this:
Oct 29 06:58:17 cigva sshd: reverse mapping checking getaddrinfo for 180.144.164.220.broad.sm.yn.dynamic.163data.com.cn ... (2 Replies)
Discussion started by: brightstorm
2 Replies
3. Shell Programming and Scripting
Hello everybody,
I have a requirement in my script..
When i'am executing a script, it'll ask a passwd of some service account.. I need to pass it to the script through a zipped file when it asks for it.
The script can be executed by more people many number times. So for securty purpose, it... (1 Reply)
Discussion started by: raghu.iv85
1 Replies
4. Shell Programming and Scripting
Hi Guys,
Even though, i have called db2 sql file earlier, this is my first time to call a oracle sql file.
I need to make a database(oracle) connection and then call the sql file in a loop.
Can you please help me out.
Thanks for your help and time.
Regards,
Magesh (4 Replies)
Discussion started by: mac4rfree
4 Replies
5. Programming
Hi gurus can you explain following lines of code ?
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
int main(void)
{
pid_t pid;
int rv;
switch(pid = fork()) {
case -1:
... (7 Replies)
Discussion started by: wakatana
7 Replies
6. UNIX Desktop Questions & Answers
hey,
how do i create a for loop that runs for i from 001 to 500 ?
i need that the zero prefix will remain so when i print "i" it will
look like so:
001
002
.
.
008
009
.
.
058
059
.
.
500
please advise. (2 Replies)
Discussion started by: boaz733
2 Replies
7. UNIX for Dummies Questions & Answers
We have a program source C and is required to indicate how many times each function is called from the C program. also print the line number where there is a call.
I've tried something like this:
#!/bin/sh
for i in $*;do
if !
then
echo $i is not a C file.
else echo $i... (0 Replies)
Discussion started by: oana06
0 Replies
8. Shell Programming and Scripting
Hi,
I want to read file multiple times. Right now i am using while loop but that is not working.
ex.
While read line
do
while read line2
do
echo stmt1
#processing some data based on data.,
done < file2.txt
done < file1.txt # This will have 10... (4 Replies)
Discussion started by: tmalik79
4 Replies
9. Shell Programming and Scripting
I've created a tag in the makefile:
mytag: $(shell ${PWD}/script.sh)
When i do: make clean - the script is executed
When i perform make or make mytag the script is again executed with the output:
make: Nothing to be done for mytag
What i want ?
I want script.sh to be executed only... (0 Replies)
Discussion started by: Pufo
0 Replies
10. Shell Programming and Scripting
Hello,
I have this awk code in a bash script to perform a find and replace task. This finds one unique line in a file and substitutes the found line with a replacement.
#! /bin/bash
# value determined elsewhere
total_outputs_p1=100
# file being modified... (5 Replies)
Discussion started by: LMHmedchem
5 Replies
LEARN ABOUT LINUX
opendir
OPENDIR(3) Linux Programmer's Manual OPENDIR(3) NAME
opendir, fdopendir - open a directory SYNOPSIS
#include <sys/types.h> #include <dirent.h> DIR *opendir(const char *name); DIR *fdopendir(int fd); Feature Test Macro Requirements for glibc (see feature_test_macros(7)): fdopendir(): Since glibc 2.10: _XOPEN_SOURCE >= 700 || _POSIX_C_SOURCE >= 200809L Before glibc 2.10: _GNU_SOURCE DESCRIPTION
The opendir() function opens a directory stream corresponding to the directory name, and returns a pointer to the directory stream. The stream is positioned at the first entry in the directory. The fdopendir() function is like opendir(), but returns a directory stream for the directory referred to by the open file descriptor fd. After a successful call to fdopendir(), fd is used internally by the implementation, and should not otherwise be used by the application. RETURN VALUE
The opendir() and fdopendir() functions return a pointer to the directory stream. On error, NULL is returned, and errno is set appropri- ately. ERRORS
EACCES Permission denied. EBADF fd is not a valid file descriptor opened for reading. EMFILE Too many file descriptors in use by process. ENFILE Too many files are currently open in the system. ENOENT Directory does not exist, or name is an empty string. ENOMEM Insufficient memory to complete the operation. ENOTDIR name is not a directory. VERSIONS
fdopendir() is available in glibc since version 2.4. CONFORMING TO
opendir() is present on SVr4, 4.3BSD, and specified in POSIX.1-2001. fdopendir() is specified in POSIX.1-2008. NOTES
The underlying file descriptor of the directory stream can be obtained using dirfd(3). The opendir() function sets the close-on-exec flag for the file descriptor underlying the DIR *. The fdopendir() function leaves the set- ting of the close-on-exec flag unchanged for the file descriptor, fd. POSIX.1-200x leaves it unspecified whether a successful call to fdopendir() will set the close-on-exec flag for the file descriptor, fd. SEE ALSO
open(2), closedir(3), dirfd(3), readdir(3), rewinddir(3), scandir(3), seekdir(3), telldir(3) COLOPHON
This page is part of release 3.27 of the Linux man-pages project. A description of the project, and information about reporting bugs, can be found at http://www.kernel.org/doc/man-pages/. GNU
2010-06-20 OPENDIR(3)