Hello all in group,
I have Fixed
the actual issue that was causing a pain to the poor 'system() api' call
that i had been doubting on for the past few weeks. The problem wasn't with unavailable memory (that i earlier suspected
due to uncleaned child processes) etc etc. this magic number of failure "after around
500" gave me clues that either "uncleaned child processes" or its "
other limits reached". If it was the first one, then even if the command was executed directly on the shell console, it would fail, but in my case it din't happen that way. So, i concentrated on the second case. Google-d 10's of links about resource limits and learned some facts about NR_TASKS, file descriptor max etc and found numbers like 512, 1024 etc.
The actual problem was like this.
after system("script.sh > file.txt") had been called, there were two other functions that were later executed. In those functions, there was one operation like this...
DIR* dir = opendir("/data/cores");
what has actually happened is, my kernel allowed only 1024 open file descriptors per process (ulimit -n) as default. once the testing started, each loop will open 2 descriptors, but didn't close them. As a result, after approx 500 times, approx of 1024 descriptors was still open (known to the kernel) by the process itself, and after that when system call system(..) API tried to open file.txt using the 'cat' ">" file.txt, it did not get a free descriptor from the kernel. and hence the execl inside the system(.) call API failed, returning 127 error code.
How did i find?
One possible way is to re-compile kernel enabling all debug logs around the glibc areas.
i dint go by the above. I tried to open a file next to system() call api directly using 'open' system call, and that returned NULL. What if you use 'perror" after that failure. The whole thing is there. It coveyed the whole information of what was going wrong.
also later i learnt that in /proc/pid/fd if you see will list all file descriptors that are in use. Ideally there will be 3 or 4 default. in my case, there were 1024 :P
GAME OVER
Thanks