Cannot make pipe for process substitution: Too many open files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Cannot make pipe for process substitution: Too many open files
# 1  
Old 02-25-2013
Cannot make pipe for process substitution: Too many open files

Hi,

I've came across an issue with a script I've been writing to check DHCP addresses on an Solaris system, the script has been running reasonably well, until it hit the following problem:

Code:
./sub_mon_v2: redirection error: cannot duplicate fd: Too many open files
./sub_mon_v2: cannot make pipe for process substitution: Too many open files
./sub_mon_v2: cannot make pipe for process substitution: Too many open files
./sub_mon_v2: line 45: <(echo $result | awk -F"/" '{print $1,$2,$3}'): ambiguous redirect

./sub_mon_v2: cannot make pipe for process substitution: Too many open files
./sub_mon_v2: cannot make pipe for process substitution: Too many open files
./sub_mon_v2: line 45: <(echo $result | awk -F"/" '{print $1,$2,$3}'): ambiguous redirect

./sub_mon_v2: cannot make pipe for process substitution: Too many open files
./sub_mon_v2: cannot make pipe for process substitution: Too many open files
./sub_mon_v2: line 45: <(echo $result | awk -F"/" '{print $1,$2,$3}'): ambiguous redirect

./sub_mon_v2: cannot make pipe for process substitution: Too many open files
./sub_mon_v2: cannot make pipe for process substitution: Too many open files
./sub_mon_v2: line 45: <(echo $result | awk -F"/" '{print $1,$2,$3}'): ambiguous redirect

I'm pretty sure this is down to scrappy coding (with me being the culprit - I'm still learning Smilie).

Is there anyone here that could maybe provide a few pointers as to how I can clean up the following function to try and fix the problem displayed above (I've higlighed what is almost certainly the problem line in bold):
Code:
check ()
{
        > $low_subs

        # Following line creates the list of subnets to check (minus the exempt subnets).
        /usr/xpg4/bin/grep -vxFf <(egrep -v "#|^$" $exempt) $check_list > $final_list

        while read line
        do
                while read result
                do
                        #Read each the date for each IP address in Day Month Year variables for checking
                        read month day year < <(echo $result | awk -F"/" '{print $1,$2,$3}')

                        # if it expired in a previous year, bin it
                        if [ $year -lt $curr_year ]
                        then
                                (( total_ip = total_ip +1 ))
                        fi

                        # if it expired in a previous month, bin it.
                        if [ $year -eq $curr_year ] && [ $month -lt $curr_month ]
                        then
                                (( total_ip = total_ip +1 ))
                        fi

                        # if it expired earlier this month
                        if [ $year -eq $curr_year ] && [ $month -eq $curr_month ] && [ $day -lt $curr_day ]
                        then
                                (( total_ip = total_ip +1 ))
                        fi

                        if [ $total_ip -le 20 ]
                        then
                                echo "Subnet "$line" running low on available IP's -"$total_ip" remaining." >> $low_subs;
                        fi

        # Pass in date field of each line returned from pntadm -P (minus Free/Reserved addresses)
                done < <(pntadm -P $line | egrep -v "Client|$^|Zero|Forever" | awk '{print $5}')
    
    # Pass in list of problem subnets
        done < $final_list
}

Thanks in advance
CiCa
# 2  
Old 02-25-2013
Well, this seems to be VERY many open files - use getconf OPEN_MAX to find out how many open files you're allowed for.
Why don't you read the date variables immediately from the pntadm output? Or, even better, create one single number in the form yyyymmdd (20130225 for today, run date +%Y%m%d) that you can compare in one go? Guessing about the output format from your code above, I'd propose sth. like (untested):
Code:
awk '{split ($5, dt, "\/"); print dt[3]dt[1]dt[2]}'

Plus, I'd guess, pntadm might as well give you an overall result list for all subnets, that you then can analyse against check list / final list.
# 3  
Old 02-25-2013
Thanks RudiC...

The problem I have with the dates, is that pntadm -P <subnet> outputs the date for each IP address on that subnet in the mm/dd/yyyy format, this is why I took the current date and broke it into seperate numbers for easy comparison - again probably not the slickest way of doing it, considering each run of pntadm -Poutputs around 257 lines - but it's the best I've got.

Reading the data into an array was something I thought about, although I'm not very familiar with array manipulation i.e. writing/reading data to and from an array - so this idea didn't progress either.

Maybe my error has something to do with a while loop within a while loop...
# 4  
Old 02-25-2013
I was not talking arrays, but files. Have pntadm output its info on ALL subnets into one file, and then you have your check file / final file. testing one against the other, you can select / reject the single info lines of interest having exactly two files open, not in excess of 1024 (which is OPEN_MAX on my linux box)
This User Gave Thanks to RudiC For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Red Hat

Process to use open files

I would like to test open files usage in my system, if I want to create a process ( or script ) that can use a certain amount ( eg. 1000 ) of open files , is it possible to create such process ( or script ) ? (3 Replies)
Discussion started by: ust
3 Replies

2. UNIX for Dummies Questions & Answers

Environment variable substitution in Linux make

I know that I can do this in bash ver=${VERSION:-$DEFVERSION} so ver is $VERSION if it's set but $DEFVERSION if $VERSION isn't set I want to do the same thing as a macro in a Makefile and can't get it to work - maybe something like... VER=$(shell ${$(VERSION):-$(DEFVERSION)}) Any help... (1 Reply)
Discussion started by: JerryHone
1 Replies

3. Shell Programming and Scripting

sed returns different results while substitution on a pipe delimited file

Hi, Need help with a sed command that I am using to substitute 3 positions of a pipe delimited file. i am getting different results while substituting the same position of two different files with the same value. Please see details below: $ cat chk2 ... (3 Replies)
Discussion started by: vmenon
3 Replies

4. Programming

Make: Bad Substitution

Hi, I have a make file which I try to execute, but it failed when it arrived to the line: for r in ${PIPESTATUS }; do if (($r != 0)); then exit $r; fi;done; With the Error: ""make:/bin/sh: Bad substitution"" Or the Error: "make:${PIPESTATUS[...}: Bad substitution" (Depend on the operating... (3 Replies)
Discussion started by: nadne
3 Replies

5. Shell Programming and Scripting

Using Make to batch process files

Hello all, I have a make question, and I was hoping somebody here might be able to point me in the right direction. Here is my issue; I have a command-line tool that I use to run a conversion on an input XML file that results in an output binary file. However, this particular tool needs to... (1 Reply)
Discussion started by: jujumbura
1 Replies

6. Red Hat

cannot set user id: Resource temporarily unavailable (not open file/open process related)

First post, sorry to be a bother but this one has been dogging me. I have a process user (java application server) that trips a resource limit every couple weeks and need help finding what limit we're hitting. First, this is what's running: This is the error when jobs are run or the... (0 Replies)
Discussion started by: Katahdin
0 Replies

7. Shell Programming and Scripting

Assistance pls - pipe error: Too many open files in system

When I run a bash script in the customer system, it throws the warning and script exits Exec '/root/sample.sh' @ hostname-- OK (warn) /root/sample.sh: pipe error: Too many open files in system /root/sample.sh: n + : syntax error: operand expected (error token is " ") Exec... (5 Replies)
Discussion started by: vidhyamirra
5 Replies

8. Shell Programming and Scripting

How to make the parent process to wait for the child process

Hi All, I have two ksh script. 1st script calls the 2nd script and the second script calls an 'C' program. I want 1st script to wait until the 'C' program completes. I cant able to get the process id for the 'C' program (child process) to make the 1st script to wait for the second... (7 Replies)
Discussion started by: sennidurai
7 Replies

9. OS X (Apple)

Open Files for a process

I am having a client/server application which will download files from server. If server has directories, it will create directories/sub directories and then download files. In this process, I observed that number of open files are more than 400 (which is approxmately same as number of dir/subdir... (1 Reply)
Discussion started by: satyam90
1 Replies

10. Shell Programming and Scripting

how to start a process and make it sleep for 5 mins and then kill that process

how to start a process and make it sleep for 5 mins and then kill that process (6 Replies)
Discussion started by: shrao
6 Replies
Login or Register to Ask a Question