Check status of long running multiple curl commands in shell script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Check status of long running multiple curl commands in shell script
# 1  
Old 02-05-2019
Check status of long running multiple curl commands in shell script

I am working on script. it reads a file which contains multiple lines
Ex;
Code:
curl --write-out %{http_code} --silent  --output /dev/null  http://hostname:port/input=1
curl --write-out %{http_code} --silent  --output /dev/null  http://hostname:port/input=2
curl --write-out %{http_code} --silent  --output /dev/null  http://hostname:port/input=3
curl --write-out %{http_code} --silent  --output /dev/null  http://hostname:port/input=4

I can read file and put it background. each command can take anywhere from 1 min to 3 hours.
With this I am unable to capture http_code. I have to exit the script if any of the http_code is not 200.
# 2  
Old 02-05-2019
I'm lost. Smilie

Are you saying that you're using a shell where the string %{http_code} sometimes expands to one of the strings 200, %{200}, %200, or {200}?

If so, which one?

And, if so, isn't there something equivalent to the test utility in that shell that you can use to determine whether or not %{http_code} expands to the undesired string before you start the curl in the background instead of trying to kill it off after it has been started?

What operating system and shell are you using?

What shell will you be using to try to detect whether or not these bad forms of background jobs are running.?

Doesn't the operating system you're using have a ps command that can be used to display the command line arguments being used by running processes?
# 3  
Old 02-06-2019
oraclemanpt,

You might consider using the curl EXIT CODE in your script in some useful way.

Example:

Code:
#!/bin/sh
curl http://hostname.com/port
res=$?
if test "$res" != "0"; then
   echo "the curl command failed with: $res"
fi

According to the Linux man page:

Code:
https://www.unix.com/man-page/linux/1/curl/

Code:
EXIT CODES

       There  are  a  bunch  of different error codes and their corresponding error messages that may appear during bad conditions. At the time of
       this writing, the exit codes are:

       1      Unsupported protocol. This build of curl has no support for this protocol.

       2      Failed to initialize.

       3      URL malformed. The syntax was not correct.

       4      A feature or option that was needed to perform the desired request was not enabled or was explicitly disabled at build-time. To make
	      curl able to do this, you probably need another build of libcurl!

       5      Couldn't resolve proxy. The given proxy host could not be resolved.

       6      Couldn't resolve host. The given remote host was not resolved.

       7      Failed to connect to host.

       8      FTP weird server reply. The server sent data curl couldn't parse.

       9      FTP  access denied. The server denied login or denied access to the particular resource or directory you wanted to reach. Most often
	      you tried to change to a directory that doesn't exist on the server.

       11     FTP weird PASS reply. Curl couldn't parse the reply sent to the PASS request.

       13     FTP weird PASV reply, Curl couldn't parse the reply sent to the PASV request.

       14     FTP weird 227 format. Curl couldn't parse the 227-line the server sent.

       15     FTP can't get host. Couldn't resolve the host IP we got in the 227-line.

       17     FTP couldn't set binary. Couldn't change transfer method to binary.

       18     Partial file. Only a part of the file was transferred.

       19     FTP couldn't download/access the given file, the RETR (or similar) command failed.

       21     FTP quote error. A quote command returned error from the server.

       22     HTTP page not retrieved. The requested url was not found or returned another error with the HTTP error code being 400 or above. This
	      return code only appears if -f/--fail is used.

       23     Write error. Curl couldn't write data to a local filesystem or similar.

       25     FTP couldn't STOR file. The server denied the STOR operation, used for FTP uploading.

       26     Read error. Various reading problems.

       27     Out of memory. A memory allocation request failed.

       28     Operation timeout. The specified time-out period was reached according to the conditions.

       30     FTP PORT failed. The PORT command failed. Not all FTP servers support the PORT command, try doing a transfer using PASV instead!

       31     FTP couldn't use REST. The REST command failed. This command is used for resumed FTP transfers.

       33     HTTP range error. The range "command" didn't work.

       34     HTTP post error. Internal post-request generation error.

       35     SSL connect error. The SSL handshaking failed.

       36     FTP bad download resume. Couldn't continue an earlier aborted download.

       37     FILE couldn't read file. Failed to open the file. Permissions?

       38     LDAP cannot bind. LDAP bind operation failed.

       39     LDAP search failed.

       41     Function not found. A required LDAP function was not found.

       42     Aborted by callback. An application told curl to abort the operation.

       43     Internal error. A function was called with a bad parameter.

       45     Interface error. A specified outgoing interface could not be used.

       47     Too many redirects. When following redirects, curl hit the maximum amount.

       48     Unknown  option  specified  to  libcurl.	This  indicates  that  you passed a weird option to curl that was passed on to libcurl and
	      rejected. Read up in the manual!

       49     Malformed telnet option.

       51     The peer's SSL certificate or SSH MD5 fingerprint was not OK.

       52     The server didn't reply anything, which here is considered an error.

       53     SSL crypto engine not found.

       54     Cannot set SSL crypto engine as default.

       55     Failed sending network data.

       56     Failure in receiving network data.

       58     Problem with the local certificate.

       59     Couldn't use specified SSL cipher.

       60     Peer certificate cannot be authenticated with known CA certificates.

       61     Unrecognized transfer encoding.

       62     Invalid LDAP URL.

       63     Maximum file size exceeded.

       64     Requested FTP SSL level failed.

       65     Sending the data requires a rewind that failed.

       66     Failed to initialise SSL Engine.

       67     The user name, password, or similar was not accepted and curl failed to log in.

       68     File not found on TFTP server.

       69     Permission problem on TFTP server.

       70     Out of disk space on TFTP server.

       71     Illegal TFTP operation.

       72     Unknown TFTP transfer ID.

       73     File already exists (TFTP).

       74     No such user (TFTP).

       75     Character conversion failed.

       76     Character conversion functions required.

       77     Problem with reading the SSL CA cert (path? access rights?).

       78     The resource referenced in the URL does not exist.

       79     An unspecified error occurred during the SSH session.

       80     Failed to shut down the SSL connection.

       82     Could not load CRL file, missing or wrong format (added in 7.19.0).

       83     Issuer check failed (added in 7.19.0).

       84     The FTP PRET command failed

       85     RTSP: mismatch of CSeq numbers

       86     RTSP: mismatch of Session Identifiers

       87     unable to parse FTP file list

       88     FTP chunk callback reported error

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl script running multiple commands at once

Hi All, I have put a perl script together to go and collect some information from multiple nodes/endpoints. The script works absolutly fine however I want to make it quicker. You will see in the below that my script calls an expect script called ssh_run_cmd2.exp followed by the IP of... (7 Replies)
Discussion started by: mutley2202
7 Replies

2. Shell Programming and Scripting

UNIX script to check multiple jobs runninng status

Hi Folks, Please help me ,I need a unix shell script to check for multiple jobs running. if there are multiple backup Jobs running then it should be trigger an email . Thanks, Anand T (1 Reply)
Discussion started by: nandu67
1 Replies

3. Linux

Check up the status of a Script (running or not)

Hello, i allready search on google und here in the local Forum, but can't found something. I need a query in php, that check whether a process (script) is running or not. Like this: php query: /usr/bin/Script01 >> if runnig, then: "Script01 is Online", if not "Script01 is Offline" I... (2 Replies)
Discussion started by: ProTechEx
2 Replies

4. Shell Programming and Scripting

Shell Script for continuously checking status of a another script running in background, and immedia

Hi, I want to write a script which continuously checking status of a script running in background by nohup command. And if same script is not running then immediately start the script...please help.. i am using below command to run script nohup system_traps.sh & but in some... (9 Replies)
Discussion started by: ketanraut
9 Replies

5. Shell Programming and Scripting

Alert for long running commands in script

Hi All, Is there a way in which a long running command can terminate by itself inside a script? I need something like below: echo Start <command> exit If the <command> is taking more than say 100 seconds to complete, the script should exit without manual intervention. Thanks, Deepak (2 Replies)
Discussion started by: deepakgang
2 Replies

6. UNIX for Dummies Questions & Answers

Job Status for running shell script

Hello, I am running a shell script whose execution often takes several hours to complete. Is there way I can get some kind of status update as the job is running? Something as simple as the start and the current time stamp. Thanks, Gussi (2 Replies)
Discussion started by: Gussifinknottle
2 Replies

7. Shell Programming and Scripting

Running multiple unix commands in a single script

Hi, I would like to write a script with include more than 6 unix commands. my script like below: echo " script started" ls -ld bdf | grep "rama" tail -10 log.txt ... .. ... now, i want to run above unix commands one by one. example: first the ls -ld command will be... (3 Replies)
Discussion started by: koti_rama
3 Replies

8. UNIX for Dummies Questions & Answers

running a simple script file with multiple commands

I'm trying to run a script file with multiple commands that I would normally type into the command line. The commands are: #!/bin/bash diff Test1.o0 /usr3/ronelso4/Desktop/verificationKPC/Test1.o0 > differences2 diff Test1a.o0 /usr3/ronelso4/Desktop/verificationKPC/Test1a.o0 >> differences2... (1 Reply)
Discussion started by: knelson
1 Replies

9. UNIX for Advanced & Expert Users

How to check since when (for how long) a particular process is running ?

How do I check if a particular process is running from a certain date and time ? (2 Replies)
Discussion started by: hpuxlxboy
2 Replies

10. UNIX for Advanced & Expert Users

how to check how long the process has been running.

I have a requirement to check how long a process is running on unix system.If i use ps -ef i am getting the following message guest 2453638 1998920 0 16:16:05 - 0:00 dsapi_slave 9 8 0 but this is showing only time not the date.Can any one please advice me any script to find out how... (2 Replies)
Discussion started by: ukatru
2 Replies
Login or Register to Ask a Question