Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk (Too many open files)
Top Forums Shell Programming and Scripting awk (Too many open files) Post 303011684 by disedorgue on Monday 22nd of January 2018 11:51:20 AM
Old 01-22-2018
In your case, you must create 2 functions and 5 fifo for work fine:
First function:
Code:
func_openssl ()
{
    openssl < <(
 j=1
 while read mdp
 do
  x=$(( j++ % 2 ))
  echo "dgst -sha256 -out out$x in$x"
  echo "$mdp" >in$x
done <std0
)
}

second function:
Code:
func_mngopenssl ()
{
    read a1 b1 c1 d1 e1 f1;
    echo "$e1" > std0;
    o=1;
    while read a2 b2 c2 d2 e2 f2; do
        x=$(( o++ %2 ));
        echo "$e2" > std0;
        e3=$(awk '{print $NF}' out$x);
        echo "$a1;$b1;$c1;$d1;$e3;$f1";
        a1="$a2";
        b1="$b2";
        c1="$c2";
        d1="$d2";
        e1="$e2";
        f1="$f2";
    done;
    x=$(( o++ %2 ));
    e3=$(awk '{print $NF}' out$x);
    echo "$a1;$b1;$c1;$d1;$e3;$f1"
}

create fifo std0 in0 in1 out0 out1 in your curent directory:
Code:
mkfifo std0 in0 in1 out0 out1

And you can use as:
Code:
func_openssl >/dev/null &  func_mngopenssl <fichier

You can change your word separator as (here, I choose ';' as separator) :
Code:
func_openssl >/dev/null &  IFS=\; func_mngopenssl <fichier

File example:
Code:
$ cat /tmp/words.csv
word_01;word_02;word_03;word_04;word_05;word_06;word_07
word_11;word_12;word_13;word_14;word_15;word_16;word_17
word_01;word_02;word_03;word_04;word_05;word_06;word_07
word_11;word_12;word_13;word_14;word_15;word_16;word_17
word_01;word_02;word_03;word_04;word_05;word_06;word_07
word_11;word_12;word_13;word_14;word_15;word_16;word_17
word_01;word_02;word_03;word_04;word_05;word_06;word_07
word_11;word_12;word_13;word_14;word_15;word_16;word_17
word_01;word_02;word_03;word_04;word_05;word_06;word_07
word_11;word_12;word_13;word_14;word_15;word_16;word_17
word_01;word_02;word_03;word_04;word_05;word_06;word_07
word_11;word_12;word_13;word_14;word_15;word_16;word_17

execution:
Code:
$ func_openssl >/dev/null & IFS=\; func_mngopenssl </tmp/words.csv
[1] 31328
word_01;word_02;word_03;word_04;f0a73d8cb2a8fb7b724132be9853f9cf72ac00ca42abeaf7bce1bbcfd299d594;word_06;word_07
word_11;word_12;word_13;word_14;467cb4d2ed408093ce85766df9836a759e7f3dcd710308f1a2bba10fe0b921c6;word_16;word_17
word_01;word_02;word_03;word_04;f0a73d8cb2a8fb7b724132be9853f9cf72ac00ca42abeaf7bce1bbcfd299d594;word_06;word_07
word_11;word_12;word_13;word_14;467cb4d2ed408093ce85766df9836a759e7f3dcd710308f1a2bba10fe0b921c6;word_16;word_17
word_01;word_02;word_03;word_04;f0a73d8cb2a8fb7b724132be9853f9cf72ac00ca42abeaf7bce1bbcfd299d594;word_06;word_07
word_11;word_12;word_13;word_14;467cb4d2ed408093ce85766df9836a759e7f3dcd710308f1a2bba10fe0b921c6;word_16;word_17
word_01;word_02;word_03;word_04;f0a73d8cb2a8fb7b724132be9853f9cf72ac00ca42abeaf7bce1bbcfd299d594;word_06;word_07
word_11;word_12;word_13;word_14;467cb4d2ed408093ce85766df9836a759e7f3dcd710308f1a2bba10fe0b921c6;word_16;word_17
word_01;word_02;word_03;word_04;f0a73d8cb2a8fb7b724132be9853f9cf72ac00ca42abeaf7bce1bbcfd299d594;word_06;word_07
word_11;word_12;word_13;word_14;467cb4d2ed408093ce85766df9836a759e7f3dcd710308f1a2bba10fe0b921c6;word_16;word_17
word_01;word_02;word_03;word_04;f0a73d8cb2a8fb7b724132be9853f9cf72ac00ca42abeaf7bce1bbcfd299d594;word_06;word_07
word_11;word_12;word_13;word_14;467cb4d2ed408093ce85766df9836a759e7f3dcd710308f1a2bba10fe0b921c6;word_16;word_17
[1]+  Done                    func_openssl > /dev/null

Here, output separator is forced to ';' , but you can modified code in function.
This User Gave Thanks to disedorgue For This Post:
tealc
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk: cannot open inputf

I'd like some help with awk too. Somewhere in my bash script i have written something like this: awk -v outputfile=$finalFile -v inputf=$file 'BEGIN { commands } { more commands } ΕND{ }' inputf where the file is a .input file with data. I also tried this: awk -v... (2 Replies)
Discussion started by: little_fairy
2 Replies

2. Shell Programming and Scripting

awk: can't open

Hi Guys, I was just trying to run a bunch of test scripts which ran perfectly before i changed servers to a different solaris machine. But i just changed it to run on a different solaris version and my awk statement in the script shows an error. awk BEGIN{s=ARGV; r=ARGV; \ ... (2 Replies)
Discussion started by: wick3dsunny
2 Replies

3. Shell Programming and Scripting

awk Can't open file

Hello all, I am creating a small bash script to cycle through some files that are the results of an analysis I have run (I am a statistician). There are many folders called job001, job002,...,job135. Each folder has a text file called results.txt. I want to create two summary files of the... (3 Replies)
Discussion started by: jem43
3 Replies

4. Shell Programming and Scripting

awk: can't open the file

I have a shell script with the following awk command. awk '$1 == "GEOG_SRV"{split($2,a,":");print a}' /etc/qm when I run it manually command line its fine and giving the output. bash-3.00$ awk '$1 == "GEOG_SRV"{split($2,a,":");print a}' /etc/qm 9010 But in the script its... (5 Replies)
Discussion started by: Tuxidow
5 Replies

5. Shell Programming and Scripting

How to open large datafie in awk?

I just tried awk '{print}' all.plo awk: cannot open all.plo (Value too large for defined data type)awk '{print $8"-"$7"-"$6,$9,$4,$5,$12,$15}' all.plo awk: cannot open all.plo (Value too large for defined data type) datafile size is 4.8GB any other provision ? only cat works FS is... (4 Replies)
Discussion started by: Akshay Hegde
4 Replies

6. Shell Programming and Scripting

Awk: can't open file!!

To whom it may concern, Please help! I am trying to use awk to open all the files in a folder and remove one row and append everything into an output file. for X in `ls /Users/misssmith/Desktop/birdseed/`; do awk '(NR==546412)' /Users/misssmith/Desktop/birdseed/$X >>... (6 Replies)
Discussion started by: salpie
6 Replies

7. Shell Programming and Scripting

Awk: can't open error

Hi , In a directory i've the files in the following format pay:year:mon:11789604 pay:year:mon:17675644 --- and i need to get 4th part of the above file name so i used awk command in the below code #!/bin/ksh for test_data in pay* do txt_awk = awk -F':' '{print $4;}' $test_data ... (7 Replies)
Discussion started by: smile689
7 Replies

8. Shell Programming and Scripting

awk throws makes too many open files

Hi, I have a below awk script. BEGIN { FS=","; } { system("curl -v -H \"Authorization: SSWS test" -H \"Accept: application/json\" -H \"Content-Type: application/json\" -X POST \"https://tes.test.com/api/v1/users?activate=false\" -d \'{ \"profile\": { \"firstName\": \"" $1 "... (7 Replies)
Discussion started by: Krrishv
7 Replies

9. Shell Programming and Scripting

awk output yields error: awk:can't open job_name (Autosys)

Good evening, Im newbie at unix specially with awk From an scheduler program called Autosys i want to extract some data reading an inputfile that comprises jobs names, then formating the output to columns for example 1. This is the inputfile: $ more MapaRep.txt ds_extra_nikira_usuarios... (18 Replies)
Discussion started by: alexcol
18 Replies

10. Shell Programming and Scripting

awk date too many open files

Dear Community; I have a csv file with msb and lsb in $3 and $5 fields which provides the epochtime (factor 65536). Further I need to convert epochtime to readable datetime. But am getting an error. File Sample: 5000a,1000,20671,0,16421,0,1,NULL,0 5000b,1000,20974,0,-16284,0,1,NULL,0... (4 Replies)
Discussion started by: mystition
4 Replies
All times are GMT -4. The time now is 04:29 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy