How to avoid "Too many arguments" error, when passing a long String literal as input to a command?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to avoid "Too many arguments" error, when passing a long String literal as input to a command?
# 1  
Old 03-06-2015
Computer How to avoid "Too many arguments" error, when passing a long String literal as input to a command?

Hi,

I am using awk here.
Inside an awk script, I have a variable which contains a very long XML data in string format (500kb).

I want to pass this data (as argument) to curl command using system function.
But getting Too many arguments error due to length of string data(payloadBlock).

I have tried the below ways, but both error out with code: 127 (Too many arguments)
Awk-extracted value inside the variable(payloadBlock) looks like:
payloadBlock="<?xml?><root> ... ... (lot of xml data) ... ... </root>";

Code:
CURL_RETURN_CODE=system("curl -s -S -X POST "URL" -H \"Content-Type: text/xml\" --data-binary '"payloadBlock"'");

Code:
CURL_RETURN_CODE=system("echo '"payloadBlock"' | curl -s -S -X POST "URL" -H \"Content-Type: text/xml\" --data-binary @-;");

Could you advise any better way to do this?
NOTE: I have to invoke curl from within awk only, as I am computing this string data inside awk script.
# 2  
Old 03-06-2015
What language is this? Shell language has no system() command nor any need for such.

To post a lot of data,
Code:
echo "all my post data" | curl -d '@-'

# 3  
Old 03-06-2015
Quote:
Originally Posted by Corona688
What language is this? Shell language has no system() command nor any need for such.

To post a lot of data,
Code:
echo "all my post data" | curl -d '@-'

I am using an awk script, and passing the script to awk command using -f switch.
awk has a function called system() to execute shell commands.


In your answer, the string "all my post data" is short.
But if it is a really long string, the command will fail with error: Too many arguments.
# 4  
Old 03-06-2015
The way to avoid too many arguments, is to not use arguments. Use a pipe.

Do this outside awk, and pipe its output into curl. @- tells it to read from standard input instead of arguments.

Code:
awk -f program | curl --data-binary "@-" ...

# 5  
Old 03-06-2015
Quote:
Originally Posted by Corona688
The way to avoid too many arguments, is to not use arguments. Use a pipe.

Do this outside awk, and pipe its output into curl. @- tells it to read from standard input instead of arguments.

Code:
awk -f program | curl --data-binary "@-" ...

This would work if I wanted to transmit all output of awk in one HTTP POST.
But if I want to POST each record extracted by awk separately, I have to perform curl from inside the awk script.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Using "mailx" command to read "to" and "cc" email addreses from input file

How to use "mailx" command to do e-mail reading the input file containing email address, where column 1 has name and column 2 containing “To” e-mail address and column 3 contains “cc” e-mail address to include with same email. Sample input file, email.txt Below is an sample code where... (2 Replies)
Discussion started by: asjaiswal
2 Replies

2. Shell Programming and Scripting

Error passing parameter in "sub" command in awk

I have to replace the pattern found in one file in another file with null/empty "" if found on the fields 3 or 4 ONLY File 1 ==== 10604747|Mxdef|9999|9999|9999|2012-03-04 00:00:59 10604747|Mcdef|8888|9999|8888|2012-03-04 00:00:59 . . . File 2 ==== 9999 8888 . . . Expected... (7 Replies)
Discussion started by: machomaddy
7 Replies

3. Shell Programming and Scripting

how to use "cut" or "awk" or "sed" to remove a string

logs: "/home/abc/public_html/index.php" "/home/abc/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" how to use "cut" or "awk" or "sed" to get the following result: abc abc xyz xyz xyz (8 Replies)
Discussion started by: timmywong
8 Replies

4. UNIX for Dummies Questions & Answers

Unix "look" Command "File too large" Error Message

I am trying to find lines in a text file larger than 3 Gb that start with a given string. My command looks like this: $ look "string" "/home/patrick/filename.txt" However, this gives me the following message: "look: /home/patrick/filename.txt: File too large" So, I have two... (14 Replies)
Discussion started by: shishong
14 Replies

5. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

6. Shell Programming and Scripting

Command to view full data "export MAESTRO_OUTPUT_STYLE=LONG"

Hi, Always when I login to Unix, I need to give the following command to view the data properly; export MAESTRO_OUTPUT_STYLE=LONG The reason is that by default the settings export MAESTRO_OUTPUT_STYLE=SHORT Please let me know how I could make LONG as the default and avoid giving the... (1 Reply)
Discussion started by: jmathew99
1 Replies

7. Shell Programming and Scripting

input string="3MMTQSZ348GGMZRQWMJM4SD6M";output string="3MMTQ-SZ348-GGMZR-QWMJM-4SD6

input string="3MMTQSZ348GGMZRQWMJM4SD6M" output string="3MMTQ-SZ348-GGMZR-QWMJM-4SD6M" using linux shell script (4 Replies)
Discussion started by: pankajd
4 Replies

8. Solaris

Compliation Error in solaris - macro "min" requires 2 arguments, but only 1 given

Hi, I am trying to compile our linux code base in solaris and came across the following issues. Am I suppose to do something special ? Can anyone help me to fix this issue. System : uname -a SunOS aspen 5.10 Generic_125100-08 sun4u sparc SUNW,Sun-Fire-280R The complier that I am using is... (0 Replies)
Discussion started by: learningkid
0 Replies

9. Shell Programming and Scripting

Avoid "++ requires lvalue" Error in Loop Calculation

Hi All, Please help me to perform sum of values in a loop, I am getting following error: "total=0++432907765772: ++ requires lvalue" where actual statement is as : total=$total+$amt where amt can have +ve or -ve values Thanks Sandeepb (3 Replies)
Discussion started by: sandeepb
3 Replies

10. UNIX for Dummies Questions & Answers

"Arguments too long"

Hi everyone, I know this sounds stupid but on my Sun solaris machine I have csh script that finds all files 7 days older and then moves it to another area. The find works but the move I get "Arguments too long". I know that in c shell can not handel more 1706 arguments. Can anyone give... (2 Replies)
Discussion started by: Peterh
2 Replies
Login or Register to Ask a Question