Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk (Too many open files)
Top Forums Shell Programming and Scripting awk (Too many open files) Post 303011551 by tealc on Friday 19th of January 2018 01:37:38 PM
Old 01-19-2018
awk (Too many open files)
Hi Team,

This is my first post, hope I am doing it right.
I have a large file, like 6 GB. Its a proxy file so vendor requested to change username from logs for saving the confidentiality of the user.

This is the script I created (With the help of Google):

Code:
awk '{
    tmp="echo " $5 " | sha256sum | cut -f5 "
tmp | getline cksum
$5=cksum
print
}' < file.txt

So the solution will be to encrypt the username (which is the 5th word per each line), that way vendor will keep track of user navigation but will not know who they are.

Problem:
This is the error I am getting:

awk: cmd. line:2: (FILENAME=- FNR=5433) fatal: cannot open pipe `echo ZS317AP | sha256sum | cut -f5 ' (Too many open files)

Looks like I need to close the file somewhere, butI donīt know where.
Could you please help me to prevent that error.

Thank you very much!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk: cannot open inputf

I'd like some help with awk too. Somewhere in my bash script i have written something like this: awk -v outputfile=$finalFile -v inputf=$file 'BEGIN { commands } { more commands } ΕND{ }' inputf where the file is a .input file with data. I also tried this: awk -v... (2 Replies)
Discussion started by: little_fairy
2 Replies

2. Shell Programming and Scripting

awk: can't open

Hi Guys, I was just trying to run a bunch of test scripts which ran perfectly before i changed servers to a different solaris machine. But i just changed it to run on a different solaris version and my awk statement in the script shows an error. awk BEGIN{s=ARGV; r=ARGV; \ ... (2 Replies)
Discussion started by: wick3dsunny
2 Replies

3. Shell Programming and Scripting

awk Can't open file

Hello all, I am creating a small bash script to cycle through some files that are the results of an analysis I have run (I am a statistician). There are many folders called job001, job002,...,job135. Each folder has a text file called results.txt. I want to create two summary files of the... (3 Replies)
Discussion started by: jem43
3 Replies

4. Shell Programming and Scripting

awk: can't open the file

I have a shell script with the following awk command. awk '$1 == "GEOG_SRV"{split($2,a,":");print a}' /etc/qm when I run it manually command line its fine and giving the output. bash-3.00$ awk '$1 == "GEOG_SRV"{split($2,a,":");print a}' /etc/qm 9010 But in the script its... (5 Replies)
Discussion started by: Tuxidow
5 Replies

5. Shell Programming and Scripting

How to open large datafie in awk?

I just tried awk '{print}' all.plo awk: cannot open all.plo (Value too large for defined data type)awk '{print $8"-"$7"-"$6,$9,$4,$5,$12,$15}' all.plo awk: cannot open all.plo (Value too large for defined data type) datafile size is 4.8GB any other provision ? only cat works FS is... (4 Replies)
Discussion started by: Akshay Hegde
4 Replies

6. Shell Programming and Scripting

Awk: can't open file!!

To whom it may concern, Please help! I am trying to use awk to open all the files in a folder and remove one row and append everything into an output file. for X in `ls /Users/misssmith/Desktop/birdseed/`; do awk '(NR==546412)' /Users/misssmith/Desktop/birdseed/$X >>... (6 Replies)
Discussion started by: salpie
6 Replies

7. Shell Programming and Scripting

Awk: can't open error

Hi , In a directory i've the files in the following format pay:year:mon:11789604 pay:year:mon:17675644 --- and i need to get 4th part of the above file name so i used awk command in the below code #!/bin/ksh for test_data in pay* do txt_awk = awk -F':' '{print $4;}' $test_data ... (7 Replies)
Discussion started by: smile689
7 Replies

8. Shell Programming and Scripting

awk throws makes too many open files

Hi, I have a below awk script. BEGIN { FS=","; } { system("curl -v -H \"Authorization: SSWS test" -H \"Accept: application/json\" -H \"Content-Type: application/json\" -X POST \"https://tes.test.com/api/v1/users?activate=false\" -d \'{ \"profile\": { \"firstName\": \"" $1 "... (7 Replies)
Discussion started by: Krrishv
7 Replies

9. Shell Programming and Scripting

awk output yields error: awk:can't open job_name (Autosys)

Good evening, Im newbie at unix specially with awk From an scheduler program called Autosys i want to extract some data reading an inputfile that comprises jobs names, then formating the output to columns for example 1. This is the inputfile: $ more MapaRep.txt ds_extra_nikira_usuarios... (18 Replies)
Discussion started by: alexcol
18 Replies

10. Shell Programming and Scripting

awk date too many open files

Dear Community; I have a csv file with msb and lsb in $3 and $5 fields which provides the epochtime (factor 65536). Further I need to convert epochtime to readable datetime. But am getting an error. File Sample: 5000a,1000,20671,0,16421,0,1,NULL,0 5000b,1000,20974,0,-16284,0,1,NULL,0... (4 Replies)
Discussion started by: mystition
4 Replies

LEARN ABOUT OSF1

sha256sum

SHA256SUM(1)							   User Commands						      SHA256SUM(1)

NAME

       sha256sum - compute and check SHA256 message digest

SYNOPSIS

       sha256sum [OPTION]... [FILE]...

DESCRIPTION

       Print or check SHA256 (256-bit) checksums.

       With no FILE, or when FILE is -, read standard input.

       -b, --binary
	      read in binary mode

       -c, --check
	      read SHA256 sums from the FILEs and check them

       --tag  create a BSD-style checksum

       -t, --text
	      read in text mode (default)

   The following five options are useful only when verifying checksums:
       --ignore-missing
	      don't fail or report status for missing files

       --quiet
	      don't print OK for each successfully verified file

       --status
	      don't output anything, status code shows success

       --strict
	      exit non-zero for improperly formatted checksum lines

       -w, --warn
	      warn about improperly formatted checksum lines

       --help display this help and exit

       --version
	      output version information and exit

       The sums are computed as described in FIPS-180-2.  When checking, the input should be a former output of this program.  The default mode is
       to print a line with checksum, a space, a character indicating input mode ('*' for binary, ' ' for text or where binary is  insignificant),
       and name for each FILE.

AUTHOR

       Written by Ulrich Drepper, Scott Miller, and David Madore.

REPORTING BUGS

       GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
       Report sha256sum translation bugs to <http://translationproject.org/team/>

COPYRIGHT

       Copyright (C) 2017 Free Software Foundation, Inc.  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
       This is free software: you are free to change and redistribute it.  There is NO WARRANTY, to the extent permitted by law.

SEE ALSO

       Full documentation at: <http://www.gnu.org/software/coreutils/sha256sum>
       or available locally via: info '(coreutils) sha2 utilities'

GNU coreutils 8.28						   January 2018 						      SHA256SUM(1)
All times are GMT -4. The time now is 08:01 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy