Assistance pls - pipe error: Too many open files in system


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Assistance pls - pipe error: Too many open files in system
# 1  
Old 03-01-2010
Assistance pls - pipe error: Too many open files in system

When I run a bash script in the customer system, it throws the warning and script exits

Exec '/root/sample.sh' @ hostname-- OK
(warn) /root/sample.sh: pipe error: Too many open files in system
/root/sample.sh: n + : syntax error: operand expected (error token is " ")

Exec '/root/sample.sh' @ hostname -- OK
(warn) /root/sample.sh: fork: Cannot allocate memory
/root/sample.sh: n + : syntax error: operand expected (error token is " ")

In the script - variable n is used in this below statement.Where as variable n is not initialised to zero.

Code:
 n=$(( n + $(find $j -type f | wc -l) ))

whether the variables need to be explicity initialized to zero beforing using them.

does this cause below warnings Too many open files in system / cannot allocate memory Smilie

Please anyone clarify me
# 2  
Old 03-01-2010
I thought you must exceed your limits allocated by the systems ,
check using ulimit command.
# 3  
Old 03-01-2010
Not clear about waht exactly you are trying to do.

Anyway, Check the ulimit settings
# 4  
Old 03-01-2010
It isn't giving the error because of not initializing the variable.It is related to some pipe errors.You may open lot of files in your system.Check it out.In bash script,you can use the variables without initializing it.
# 5  
Old 03-01-2010
script scans all the files in a specific directory and checks whether they are unzippable. all the monitoring information like how many files are scanned /corrupted are logged in a log file. I dint get any problem when I tested in my lab.

---------- Post updated at 02:59 PM ---------- Previous update was at 02:54 PM ----------

this is my code

Code:
for i in /store/archive/TP_* ; do
# Checks for the errors in compressed bitfiles.
      for j in `find $i -name "L2*" -type d`; do
           n=$(( n + $(find $j -type f | wc -l) ))
            unzip -qT $j/\* 2>>/dev/null;
                if [ $? -ne 0 ]; then
                     for k in `find $j -type f`; do
                               unzip -qT $k 2>>/dev/null;
                               if [ $? -ne 0 ]; then
                                       echo `date -r $k` $k >> $bfile;
                                     corruptedBitfiles=`expr $corruptedBitfiles + 1`;
                               fi
                                # Counting bitfiles in each subdirectory
                                bitfileCounter=`expr $bitfileCounter + 1`;
     # Keeping track of the progress of every 5k files in each sub-directory
    let batchCount=$bitfileCounter%$increment
    if [ $bitfileCounter -ge $increment -a $batchCount -eq 0 ]; then
    fi
   done
   bitfileCounter=0;
   corruptedTPDirCount=`expr $corruptedTPDirCount + $corruptedBitfiles`;
   corruptedBitfiles=0;
   fi
#Summing up the count of all sub-directories under main TP directory
  tpDirCount=`expr $tpDirCount + $n`;
  n=0;
  done
  tpDirCount=0;
  corruptedTPDirCount=0;

# 6  
Old 03-01-2010
The maximum number of open files per userid has been exceeded. This is a kernel configuration parameter.
Check if the same login is used concurrently.
Check if files are not being closed when the process is finished with them
Re-link the kernel with a higher value.
ulimit refers to the maximum size of a single file.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk assistance - Comparing 2 csv files

Hello all, I have searched high and low for a solution to this, many have come really close but not quite what I'm after. I have 2 files. One contains GUID's, for example: 8121E002-96FE-4C9C-BC5A-6AFF20DACECD 84468F30-F3B7-418B-81F0-0908E80792BF A second file, contains a path to the... (8 Replies)
Discussion started by: tirmUK
8 Replies

2. Shell Programming and Scripting

How can we overcome Broken pipe error during scp,SFTP,Rsync while transferring big files.?

Hello All, Hope all are doing well. We use scp (some times sftp and rsync also) for transferring big files (around 2GB each ) from 1 Network to another Network. The Issues which we face :- During transfer some times( Once in 1 week (or twice)) , the speed of transfer gets down to 30 kb/s,... (2 Replies)
Discussion started by: Upendra Bhushan
2 Replies

3. UNIX for Dummies Questions & Answers

Error when using vim to open files

Hi all, 1st post here. I'm on Solaris and currently, I get the following error if I try to use vim to edit anything: Error detected while processing function <SNR>6_initialize: line 2: E697: Missing end of List ']': E116: Invalid arguments for function <SNR>6_defineOption line 3: E10:... (6 Replies)
Discussion started by: fred2028
6 Replies

4. Shell Programming and Scripting

Capture output of open pipe to a file in perl

Hi, I am trying to capture the output of the an open pipe in perl. but I am not sure how to do this. can some one please help me do that? Below is the script I am using (2 Replies)
Discussion started by: ahmedwaseem2000
2 Replies

5. Shell Programming and Scripting

Cannot make pipe for process substitution: Too many open files

Hi, I've came across an issue with a script I've been writing to check DHCP addresses on an Solaris system, the script has been running reasonably well, until it hit the following problem: ./sub_mon_v2: redirection error: cannot duplicate fd: Too many open files ./sub_mon_v2: cannot make... (3 Replies)
Discussion started by: CiCa
3 Replies

6. Shell Programming and Scripting

Assistance for sorting files

Hi there, I have tried using the "find" command to do this but to no avail as the "find -mtime" command I used descend to the directories from my current working directory. Say in "directoryA" has multiple files and those files are created on a daily basis. Under "directoryB", there are... (4 Replies)
Discussion started by: chewku
4 Replies

7. Shell Programming and Scripting

Need Assistance with gzipping files with same names

Hello, I have an issue on a webserver where I have a perl script rotating and moving the logs to an archive directory where they are gzipped. This is working fine. The problem is when there is tracing enabled on my webserver where the tracelogs are rotated whenenver they reach 100Meg and they... (2 Replies)
Discussion started by: techwiz45
2 Replies

8. Shell Programming and Scripting

C, sh, perl, system(): Can't transfer a return code appropriately: help, pls

I am using a perl-script from C-code, executing it by the 'system(..)' comand. The problem is to return the perl-return code to the C correctly. Default the 'system()' shell is Bourne: sh My try: (perl_src.c_pl - the perl script; t_sys - C-program with system() call (I will show it... (7 Replies)
Discussion started by: alex_5161
7 Replies

9. Programming

Is there a system call other than 'open' for opening very large files?

Dear all, Inside a C program, I want to open a very big file (about 12 GB) in order to read its content. Here is the code: /* argv contains the path to the file. */ inputFileDescriptor = open(argv, O_RDONLY); if (inputFileDescriptor < 0) { ... (6 Replies)
Discussion started by: dariyoosh
6 Replies

10. Shell Programming and Scripting

Assistance with regex and config files

I am trying to write a shell script that will such in data from a config file. The script should mount device nodes that are contained in a config file in the following format: # filesystem type # read/write #device # Mount Point xfs w ... (1 Reply)
Discussion started by: pryker
1 Replies
Login or Register to Ask a Question