Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk throws makes too many open files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk throws makes too many open files
# 1  
Old 08-19-2014
awk throws makes too many open files
Hi,

I have a below awk script.
Code:
BEGIN {
        FS=",";
}
{

system("curl -v -H \"Authorization: SSWS test"  -H \"Accept: application/json\"  -H \"Content-Type: application/json\"  -X POST \"https://tes.test.com/api/v1/users?activate=false\"  -d  \'{  \"profile\": { \"firstName\": \"" $1 " \",  \"lastName\": \"" $2" \",  \"email\": \"" $3 "\", \"login\": \"" $4 "\" } }\'"| getline response);

  #Printing the create user response
  system("echo " response);

#get the UID from the response for adding to the group
system("echo " response "| cut -d' ' -f1 | cut -d':' -f2" | getline id);


#Add the created user to group from csv file
        system("curl -H \"Authorization: SSWS test" -H \"Accept: application/json\" -H \"Content-Type: application/json\" -X PUT \"https://test.test.com/api/v1/groups/00g2msc01fEYNMEAXNMZ/users/" id "\"");

}

when i execute the command in my mac os
Code:
awk -f Finaluser.awk test1.csv

it creates 8 users fine in the 9th user i get a below error.
Code:
awk: echo {"id":"00u2mut1cf","status":"STAGED","created":"2014-08-19T06:03:52.000Z","activated":null,"statusChanged":null,"lastLogin":null,"lastUpdated":"2014-08-19T06:03:52.000Z","passwordChanged":null,"profile":{"firstName":"Adam ","lastName":"test ","email":"test.adam@2l.test","login":"test.adam@2l.test","mobilePhone":null},"credentials":{"provider":{"type":"TEST","name":"TEST"}},"_links":{"activate":{"href":"https://test.test.com/api/v1/users/00u2mut1cf/lifecycle/activate","method":"POST"},"deactivate":{"href":"https://test.test.com/api/v1/users/00u2mut1cf/lifecycle/deactivate","method":"POST"}}}| cut -d' ' -f1 | cut -d':' -f2 makes too many open files
 input record number 9, file test1.csv

Please throw some light on this.
Last edited by Franklin52; 08-19-2014 at 03:28 AM.. Reason: Please use code tags
# 2  
Old 08-19-2014
Using a system() function will only render an exit code within awk, not a response, the getline statements will not work like that and the quoting is not properly done. I would suggest using a shell script instead which really is better suited for this kind of application..
This User Gave Thanks to Scrutinizer For This Post:
Kibou
# 3  
Old 08-19-2014
@Scrutinizer
The above script is working fine. It does the job of executing my input file, I only have the error "makes too many open files"

---------- Post updated 08-20-14 at 06:48 AM ---------- Previous update was 08-19-14 at 12:50 PM ----------

Hi,

Appreciate some response. Come on help me Guru's..
# 4  
Old 08-19-2014
Try this function

Code:
awk '

function exe_cmd(cmde,result){
                              while ((cmde | getline line) > 0)
                                  {
                                      result = result (result=="" ? "" : "\n")line
                                  }
                              close(cmde)
                              return result
                         }

BEGIN{
	command = "whoami"
	me  = exe_cmd(command)
	print me
}'


or else try to write shell script as Scrutinizer suggested

Code:
IFS=,; while read firstname lastname email login etc; do
  # your curl commands goes here...
  echo $firstname $lastname $email $login
done < "your_csv_file"

# 5  
Old 08-20-2014
@Akshay,

Thank you so much . I am trying to do this in Shell. When i use the curl command and keeping firstname:"$f1" i am not getting the variable assigned properly. But when i do echo $f1 it prints be proper value. Any clue why curl is not taking the below below is my script.

Code:
!/bin/bash
IFS=","
while read f1 f2 f3 f4
do
echo "line is" :  "$f4"
        #Create user with credentials



(curl -v -H "Authorization: SSWS 00PM9livMo9jIpji"  -H "Accept: application/json"  -H "Content-Type: application/json"  -X POST "https://test.test.com/api/v1/users?activate=false"  -d  '{  "profile": { "firstName": "$f1" ,  "lastName": "$f2" ,  "email": "$f3", "login": "$f4" } }')

echo $f1 $f2 $f3 $f4

done< /Users/kn/Downloads/test1.csv
~

# 6  
Old 08-20-2014
Quote:
Originally Posted by Krrishv
@Akshay,

Thank you so much . I am trying to do this in Shell. When i use the curl command and keeping firstname:"$f1" i am not getting the variable assigned properly. But when i do echo $f1 it prints be proper value. Any clue why curl is not taking the below below is my script.

Code:
!/bin/bash
IFS=","
while read f1 f2 f3 f4
do
echo "line is" :  "$f4"
        #Create user with credentials



(curl -v -H "Authorization: SSWS 00PM9livMo9jIpji"  -H "Accept: application/json"  -H "Content-Type: application/json"  -X POST "https://test.test.com/api/v1/users?activate=false"  -d  '{  "profile": { "firstName": "$f1" ,  "lastName": "$f2" ,  "email": "$f3", "login": "$f4" } }')

echo $f1 $f2 $f3 $f4

done< /Users/kn/Downloads/test1.csv
~

Problem is with single quote replace it with double quotes..

Code:
-d  "{  'profile': { 'firstName': '$f1' ,  'lastName': '$f2' ,  'email': '$f3', 'login': '$f4' } }"

Last edited by Akshay Hegde; 08-20-2014 at 02:04 AM..
# 7  
Old 08-20-2014
If i do double quotes. I get a error message that the server is expecting a single quote...Smilie

Server is looking for the right format.
Login or Register to Ask a Question

Previous Thread | Next Thread

7 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk (Too many open files)

Hi Team, This is my first post, hope I am doing it right. I have a large file, like 6 GB. Its a proxy file so vendor requested to change username from logs for saving the confidentiality of the user. This is the script I created (With the help of Google): awk '{ tmp="echo " $5 " |... (12 Replies)
Discussion started by: tealc
12 Replies

2. Shell Programming and Scripting

awk date too many open files

Dear Community; I have a csv file with msb and lsb in $3 and $5 fields which provides the epochtime (factor 65536). Further I need to convert epochtime to readable datetime. But am getting an error. File Sample: 5000a,1000,20671,0,16421,0,1,NULL,0 5000b,1000,20974,0,-16284,0,1,NULL,0... (4 Replies)
Discussion started by: mystition
4 Replies

3. Shell Programming and Scripting

awk output yields error: awk:can't open job_name (Autosys)

Good evening, Im newbie at unix specially with awk From an scheduler program called Autosys i want to extract some data reading an inputfile that comprises jobs names, then formating the output to columns for example 1. This is the inputfile: $ more MapaRep.txt ds_extra_nikira_usuarios... (18 Replies)
Discussion started by: alexcol
18 Replies

4. Shell Programming and Scripting

Calling shell script within awk script throws error

I am getting the following error while passing parameter to a shell script called within awk script. Any idea what's causing this issue and how to ix it ? Thanks sh: -c: line 0: syntax error near unexpected token `newline' sh: -c: line 0: `./billdatecalc.sh ... (10 Replies)
Discussion started by: Sudhakar333
10 Replies

5. Shell Programming and Scripting

awk script throws invalid char expression

Hello there, I am new to the awk scripting and getting the following error while running the script. Please can you advise me how to resolve this . Thanks ./sub_del_awk_lat.sh awk: cmd. line:5: warning: escape sequence `\/' treated as plain `/' awk: cmd. line:5: sed -n... (6 Replies)
Discussion started by: Sudhakar333
6 Replies

6. UNIX for Dummies Questions & Answers

scp throws error

My script is like STAMP=$(perl -e '($ss, $mm, $hh, $DD, $MM, $YY) = localtime(time() - 100); printf "%04d%02d%02d%02d%02d", $YY + 1900, $MM + 1, $DD, $hh, $mm') touch $STAMP /sasdata/copydata/ find /sasdata/copydata -type f ! -newer /sasdata/copydata/ -print > output3.txt awk '{print "scp ... (6 Replies)
Discussion started by: tushar_spatil
6 Replies

7. Forum Support Area for Unregistered Users & Account Problems

Search throws a wobbly

Dear Administrator, Rule No.5 for unregistered users is impossible to comply with because Google search thinks that any Unix command that I search for is an attack from a virus and blocks the search. I managed to find this site from an external search, and it looks really useful. Do you... (2 Replies)
Discussion started by: Ken N
2 Replies
Login or Register to Ask a Question