Sponsored Content
Full Discussion: awk (Too many open files)
Top Forums Shell Programming and Scripting awk (Too many open files) Post 303012090 by Chubler_XL on Monday 29th of January 2018 04:46:03 PM
Old 01-29-2018
Nice solution disedorgue.

However, I did have some issues running under bash 4.4.12(3) in a cygwin environment. It appears that the pipe write sometimes blocks waiting for a reader, causing things to lock up after a sha1 is calculated for the first line.

After a little bit of playing with it I came up with this modification, that appears to be a little more stable and reduces the number of FIFOs to 3.

Here I use file descriptor 7 and 9 for the read and write ends of std0. Avoiding using the external awk program, should also have it run faster:

Code:
func_openssl ()
{ 
  exec 7< std0
  openssl > /dev/null < <(
    while read -u 7 mdp
    do
      printf "%s\n" "dgst -sha256 -out out0 in0"
      printf "%s\n" "$mdp" >in0
    done
  )
}

func_mngopenssl ()
{
    exec 9> std0
    while read a1 b1 c1 d1 e1 f1; do
        printf "%s\n" "$e1" >&9
        IFS=" " read discard e1sha < out0
        printf "%s\n" "$a1 $b1 $c1 $d1 $e1sha $f1"
    done
    exec 9>&-
    exec 7<&-
}

trap 'rm std0 in0 out0' EXIT
mkfifo std0 in0 out0

func_openssl & func_mngopenssl <fichier

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk: cannot open inputf

I'd like some help with awk too. Somewhere in my bash script i have written something like this: awk -v outputfile=$finalFile -v inputf=$file 'BEGIN { commands } { more commands } ΕND{ }' inputf where the file is a .input file with data. I also tried this: awk -v... (2 Replies)
Discussion started by: little_fairy
2 Replies

2. Shell Programming and Scripting

awk: can't open

Hi Guys, I was just trying to run a bunch of test scripts which ran perfectly before i changed servers to a different solaris machine. But i just changed it to run on a different solaris version and my awk statement in the script shows an error. awk BEGIN{s=ARGV; r=ARGV; \ ... (2 Replies)
Discussion started by: wick3dsunny
2 Replies

3. Shell Programming and Scripting

awk Can't open file

Hello all, I am creating a small bash script to cycle through some files that are the results of an analysis I have run (I am a statistician). There are many folders called job001, job002,...,job135. Each folder has a text file called results.txt. I want to create two summary files of the... (3 Replies)
Discussion started by: jem43
3 Replies

4. Shell Programming and Scripting

awk: can't open the file

I have a shell script with the following awk command. awk '$1 == "GEOG_SRV"{split($2,a,":");print a}' /etc/qm when I run it manually command line its fine and giving the output. bash-3.00$ awk '$1 == "GEOG_SRV"{split($2,a,":");print a}' /etc/qm 9010 But in the script its... (5 Replies)
Discussion started by: Tuxidow
5 Replies

5. Shell Programming and Scripting

How to open large datafie in awk?

I just tried awk '{print}' all.plo awk: cannot open all.plo (Value too large for defined data type)awk '{print $8"-"$7"-"$6,$9,$4,$5,$12,$15}' all.plo awk: cannot open all.plo (Value too large for defined data type) datafile size is 4.8GB any other provision ? only cat works FS is... (4 Replies)
Discussion started by: Akshay Hegde
4 Replies

6. Shell Programming and Scripting

Awk: can't open file!!

To whom it may concern, Please help! I am trying to use awk to open all the files in a folder and remove one row and append everything into an output file. for X in `ls /Users/misssmith/Desktop/birdseed/`; do awk '(NR==546412)' /Users/misssmith/Desktop/birdseed/$X >>... (6 Replies)
Discussion started by: salpie
6 Replies

7. Shell Programming and Scripting

Awk: can't open error

Hi , In a directory i've the files in the following format pay:year:mon:11789604 pay:year:mon:17675644 --- and i need to get 4th part of the above file name so i used awk command in the below code #!/bin/ksh for test_data in pay* do txt_awk = awk -F':' '{print $4;}' $test_data ... (7 Replies)
Discussion started by: smile689
7 Replies

8. Shell Programming and Scripting

awk throws makes too many open files

Hi, I have a below awk script. BEGIN { FS=","; } { system("curl -v -H \"Authorization: SSWS test" -H \"Accept: application/json\" -H \"Content-Type: application/json\" -X POST \"https://tes.test.com/api/v1/users?activate=false\" -d \'{ \"profile\": { \"firstName\": \"" $1 "... (7 Replies)
Discussion started by: Krrishv
7 Replies

9. Shell Programming and Scripting

awk output yields error: awk:can't open job_name (Autosys)

Good evening, Im newbie at unix specially with awk From an scheduler program called Autosys i want to extract some data reading an inputfile that comprises jobs names, then formating the output to columns for example 1. This is the inputfile: $ more MapaRep.txt ds_extra_nikira_usuarios... (18 Replies)
Discussion started by: alexcol
18 Replies

10. Shell Programming and Scripting

awk date too many open files

Dear Community; I have a csv file with msb and lsb in $3 and $5 fields which provides the epochtime (factor 65536). Further I need to convert epochtime to readable datetime. But am getting an error. File Sample: 5000a,1000,20671,0,16421,0,1,NULL,0 5000b,1000,20974,0,-16284,0,1,NULL,0... (4 Replies)
Discussion started by: mystition
4 Replies
pthread_rwlock_wrlock(3T)												 pthread_rwlock_wrlock(3T)

NAME
pthread_rwlock_wrlock(), pthread_rwlock_trywrlock() - lock or attempt to lock a read-write lock for writing SYNOPSIS
PARAMETERS
rwlock Pointer to the read-write lock to be locked for writing. DESCRIPTION
The function applies a write lock to the read-write lock object referenced by rwlock. The calling thread acquires the write lock if no other thread (reader or writer) holds the read-write lock rwlock. Otherwise, the thread blocks (that is, it does not return from the call) until it can acquire the lock. Results are undefined if the calling thread holds the read-write lock (whether a read or a write lock) at the time the call is made. The function applies a write lock as in the function with the exception that the function fails if any thread currently holds rwlock (for reading or writing). Results are undefined if any of these functions are called with an uninitialized read-write lock. If a signal is delivered to a thread waiting for a read-write lock, upon return from the signal handler, the thread shall resume waiting for the read-write lock as if it was not interrupted. RETURN VALUE
Upon successful completion, and return zero. Otherwise, an error number is returned to indicate the error (the variable is not set). ERRORS
If any of the following occur, the function returns the corresponding error number: [EBUSY] The read-write lock rwlock could not be acquired for writing because it was already locked for reading or writing. For each of the following conditions, if the condition is detected, the and functions return the corresponding error number: [EINVAL] The value specified by rwlock does not refer to an initialized read-write lock. [EDEADLK] The current thread already owns the read-write lock for reading or writing. AUTHOR
and were developed by X/Open. SEE ALSO
pthread_rwlock_init(3T), pthread_rwlock_destroy(3T), pthread_rwlock_tryrdlock(3T), pthread_rwlock_rdlock(3T), pthread_rwlock_unlock(3T). STANDARDS CONFORMANCE
Pthread Library pthread_rwlock_wrlock(3T)
All times are GMT -4. The time now is 06:40 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy