Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk (Too many open files)
Top Forums Shell Programming and Scripting awk (Too many open files) Post 303012683 by disedorgue on Wednesday 7th of February 2018 02:02:48 PM
Old 02-07-2018
Hi,
In another french forum for similar problem, I give a solution like as:
Code:
#!/bin/bash
mkfifo in0 in1 out0 out1
openssl < <(awk 'BEGIN{while(1){print "dgst -out out0 -sha256 in0\ndgst -out out1 -sha256 in1"}}') > /dev/null &
while true ;  do IFS=" " read T X <out$((h++%2)) ; echo -n "$X" ; done &

while read a1 a2 a3 a4 a5 a6  ; do echo -n "$a1 $a2 $a3 $a4 " ; echo -n $a5 >in$((h++%2)) ; echo " $a6" ; done <file

exec 2>/dev/null
jobs -p | xargs kill
rm -f in0 in1 out0 out1

Duration of this method on my computer for 100000 iterations:
real 0m42,974s
user 0m33,780s
sys 0m29,528s

Duration of your previous method on my computer for 100000 iterations:
real 1m3,779s
user 0m32,084s
sys 0m27,428s
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk: cannot open inputf

I'd like some help with awk too. Somewhere in my bash script i have written something like this: awk -v outputfile=$finalFile -v inputf=$file 'BEGIN { commands } { more commands } ΕND{ }' inputf where the file is a .input file with data. I also tried this: awk -v... (2 Replies)
Discussion started by: little_fairy
2 Replies

2. Shell Programming and Scripting

awk: can't open

Hi Guys, I was just trying to run a bunch of test scripts which ran perfectly before i changed servers to a different solaris machine. But i just changed it to run on a different solaris version and my awk statement in the script shows an error. awk BEGIN{s=ARGV; r=ARGV; \ ... (2 Replies)
Discussion started by: wick3dsunny
2 Replies

3. Shell Programming and Scripting

awk Can't open file

Hello all, I am creating a small bash script to cycle through some files that are the results of an analysis I have run (I am a statistician). There are many folders called job001, job002,...,job135. Each folder has a text file called results.txt. I want to create two summary files of the... (3 Replies)
Discussion started by: jem43
3 Replies

4. Shell Programming and Scripting

awk: can't open the file

I have a shell script with the following awk command. awk '$1 == "GEOG_SRV"{split($2,a,":");print a}' /etc/qm when I run it manually command line its fine and giving the output. bash-3.00$ awk '$1 == "GEOG_SRV"{split($2,a,":");print a}' /etc/qm 9010 But in the script its... (5 Replies)
Discussion started by: Tuxidow
5 Replies

5. Shell Programming and Scripting

How to open large datafie in awk?

I just tried awk '{print}' all.plo awk: cannot open all.plo (Value too large for defined data type)awk '{print $8"-"$7"-"$6,$9,$4,$5,$12,$15}' all.plo awk: cannot open all.plo (Value too large for defined data type) datafile size is 4.8GB any other provision ? only cat works FS is... (4 Replies)
Discussion started by: Akshay Hegde
4 Replies

6. Shell Programming and Scripting

Awk: can't open file!!

To whom it may concern, Please help! I am trying to use awk to open all the files in a folder and remove one row and append everything into an output file. for X in `ls /Users/misssmith/Desktop/birdseed/`; do awk '(NR==546412)' /Users/misssmith/Desktop/birdseed/$X >>... (6 Replies)
Discussion started by: salpie
6 Replies

7. Shell Programming and Scripting

Awk: can't open error

Hi , In a directory i've the files in the following format pay:year:mon:11789604 pay:year:mon:17675644 --- and i need to get 4th part of the above file name so i used awk command in the below code #!/bin/ksh for test_data in pay* do txt_awk = awk -F':' '{print $4;}' $test_data ... (7 Replies)
Discussion started by: smile689
7 Replies

8. Shell Programming and Scripting

awk throws makes too many open files

Hi, I have a below awk script. BEGIN { FS=","; } { system("curl -v -H \"Authorization: SSWS test" -H \"Accept: application/json\" -H \"Content-Type: application/json\" -X POST \"https://tes.test.com/api/v1/users?activate=false\" -d \'{ \"profile\": { \"firstName\": \"" $1 "... (7 Replies)
Discussion started by: Krrishv
7 Replies

9. Shell Programming and Scripting

awk output yields error: awk:can't open job_name (Autosys)

Good evening, Im newbie at unix specially with awk From an scheduler program called Autosys i want to extract some data reading an inputfile that comprises jobs names, then formating the output to columns for example 1. This is the inputfile: $ more MapaRep.txt ds_extra_nikira_usuarios... (18 Replies)
Discussion started by: alexcol
18 Replies

10. Shell Programming and Scripting

awk date too many open files

Dear Community; I have a csv file with msb and lsb in $3 and $5 fields which provides the epochtime (factor 65536). Further I need to convert epochtime to readable datetime. But am getting an error. File Sample: 5000a,1000,20671,0,16421,0,1,NULL,0 5000b,1000,20974,0,-16284,0,1,NULL,0... (4 Replies)
Discussion started by: mystition
4 Replies

LEARN ABOUT POSIX

dialups

dialups(4)							   File Formats 							dialups(4)

NAME

       dialups - list of terminal devices requiring a dial-up password

SYNOPSIS

       /etc/dialups

DESCRIPTION

       dialups	is  an	ASCII file which contains a list of terminal devices  that require a dial-up password. A dial-up password is an additional
       password required of users who access the computer through a modem or dial-up port.  The correct password must be entered before  the  user
       is granted access to the computer. The set of ports that require a dial-up password are	listed in the dialups file.

       Each entry in the dialups file is a single line of the form:

	      terminal-device

       where

       terminal-device	       The  full  path	name  of the terminal device that will require a dial-up password for users accessing the computer
			       through a modem or dial-up port.

       The dialups file should be owned by the root user and the root group.  The file should have read and write permissions for the owner (root)
       only.

EXAMPLES

       Example 1: A sample dialups file.

       Here is a sample dialups file:

       /dev/term/a
       /dev/term/b
       /dev/term/c

FILES

       /etc/d_passwd	       dial-up password file

       /etc/dialups	       list of dial-up ports requiring dial-up passwords

SEE ALSO

       d_passwd(4)

SunOS 5.10							    4 May 1994								dialups(4)
All times are GMT -4. The time now is 06:27 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy