Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Curl/http 503 error with bash script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Curl/http 503 error with bash script
# 8  
Old 12-05-2017
Thanks rbatte1. I tried setting the LINK variable as you suggested and I am still getting the same 503 http error. This is what my script looks like now.

Code:
#!/bin/bash

export HOME=/export/home/kfoley/scripts/REST_API
export TEMP_DIR=/export/home/kfoley/scripts/REST_API/Temp

echo "Enter the hostname:"
read HOSTNAME

curl -u username:Password  http://seieadpa01:9004/apollo-api/nodes/?query=name%3d$HOSTNAME > $TEMP_DIR/test_file.txt

HOST_ID=`cat $TEMP_DIR/test_file.txt | grep "<id>"`

cat $HOME/host_query_input_file.out | awk -v ID="$HOST_ID" '{if ($1~/\<id>/) $1=ID ;print}' > $TEMP_DIR/host_query_input_file.out

curl -u username:Password -X POST -H "Content-Type:application/vnd.emc.apollo-v1+xml" --data "@/export/home/kfoley/scripts/REST_API/Temp/host_query_input_file.out" http://seieadpa01:9004/dpa-api/report -o $TEMP_DIR/test_xml.out

LINK=$(grep link $TEMP_DIR/test_xml.out)
LINK="${LINK#*>}"
LINK="${LINK%<*}"

export LINK

curl -v -u username:Password $LINK

echo "curl -u username:Password $LINK"

This is the output from the script and as you can see I am getting the 503 Service Unavailable error.

Thanks for your reply Yoda. When I run the curl from the command line I never have an issue. I get the expected html output.

In the last line of my script I have an echo and if I run that from the command line it works. If I set the LINK variable on the command line and then un just as it looks in the script it work also.

That is if I do this from outside the script it works

curl -u username:Password http://seieadpa01:9004/dpa-api/repor...9-fb4ed490d13a

or if I do this it also works.

Code:
export LINK="http://seieadpa01:9004/dpa-api/report/result/9d193c23-db7d-4c0c-aa09-fb4ed490d13a"
curl -u username:Password $LINK | head

This is the "head" of the output I should expect from the script.

Code:
[root@seiemccli01 REST_API]# curl -u username:Password $LINK | head
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5523    0  5523    0     0   512k      0 --:--:-- --:--:-- --:--:--  512k<html>
<head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
        <title>All Jobs</title>
</head>
<style>
.g
{
        margin-top:0px;
    margin-left:3px;
[root@seiemccli01 REST_API]#


Also, when I try to do the same as above using wget I am having the same issues. i.e it work outside the script but fails within it.

---------- Post updated at 10:32 AM ---------- Previous update was at 10:31 AM ----------

Sorry I never posted the output from running the curl -v which shows the 503 http error

Code:
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   714    0   714    0     0  20638      0 --:--:-- --:--:-- --:--:--  348k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   116  100   116    0     0   2328      0 --:--:-- --:--:-- --:--:--     0
* About to connect() to seieadpa01 port 9004
*   Trying 172.16.120.152... connected
* Connected to seieadpa01 (172.16.120.152) port 9004
* Server auth using Basic with user 'admin'
> GET /dpa-api/report/result/9d193c23-db7d-4c0c-aa09-fb4ed490d13a HTTP/1.1
> Authorization: Basic YWRtaW46U3RvcmFnZTAwMSE=
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Host: seieadpa01:9004
> Accept: */*
>
< HTTP/1.1 503 Service Unavailable
< Server: Apache-Coyote/1.1
< Pragma: No-cache
< Cache-Control: no-cache
< Expires: Wed, 31 Dec 1969 19:00:00 EST
< X-FRAME-OPTIONS: SAMEORIGIN
< Retry-After: 2
< Content-Type: application/octet-stream
< Content-Length: 116
< Date: Tue, 05 Dec 2017 10:04:29 GMT
< Connection: close

---------- Post updated at 11:03 AM ---------- Previous update was at 10:32 AM ----------

This is the output when I use
Code:
set -x

within the script. Note that I have taken out the real usernameSmilieassword in the script in this post but the output shows that the username is actually admin.

Code:
+ export HOME=/export/home/kfoley/scripts/REST_API
+ HOME=/export/home/kfoley/scripts/REST_API
+ export TEMP_DIR=/export/home/kfoley/scripts/REST_API/Temp
+ TEMP_DIR=/export/home/kfoley/scripts/REST_API/Temp
+ echo 'Enter the hostname:'
Enter the hostname:
+ read HOSTNAME
seiemccli01
+ curl -u 'username:Password' 'http://seieadpa01:9004/apollo-api/nodes/?query=name%3dseiemccli01'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   714    0   714    0     0  24736      0 --:--:-- --:--:-- --:--:--  697k
++ cat /export/home/kfoley/scripts/REST_API/Temp/test_file.txt
++ grep '<id>'
+ HOST_ID='      <id>560775dc-e949-490d-ae1c-d76ec33b18b5</id>'
+ cat /export/home/kfoley/scripts/REST_API/host_query_input_file.out
+ awk -v 'ID=      <id>560775dc-e949-490d-ae1c-d76ec33b18b5</id>' '{if ($1~/\<id>/) $1=ID ;print}'
+ curl -v -u 'username:Password' -X POST -H Content-Type:application/vnd.emc.apollo-v1+xml --data @/export/home/kfoley/scripts/REST_API/Temp/host_query_input_file.out http://seieadpa01:9004/dpa-api/report -o /export/home/kfoley/scripts/REST_API/Temp/test_xml.out
* About to connect() to seieadpa01 port 9004
*   Trying 172.16.120.152... connected
* Connected to seieadpa01 (172.16.120.152) port 9004
* Server auth using Basic with user 'admin'
> POST /dpa-api/report HTTP/1.1
> Authorization: Basic YWRtaW46U3RvcmFnZTAwMSE=
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Host: seieadpa01:9004
> Accept: */*
> Content-Type:application/vnd.emc.apollo-v1+xml
> Content-Length: 476
>
> <runReportParameters><report><name>Backup All Jobs</name> <!-- Report template name --></report><nodes><node>      <id>560775dc-e949-490d-ae1c-d76ec33b18b5</id> <!-- scope - node id of the Host --></node></nodes><timeConstraints type="window"><window ><name>Last Week</name> <!-- time period name --></window></timeConstraints><formatParameters><formatType>HTML</formatType> <!-- format type, could be CSV, HTML, PDF, IMAGE, XML. --></formatParameters>  </runReportParameters>HTTP/1.1 201 Created
< Server: Apache-Coyote/1.1
< X-FRAME-OPTIONS: SAMEORIGIN
< Location: http://seieadpa01:9004/dpa-api/report/result/3b33e982-b899-4439-9a8b-ed8a3a89859a
< Content-Type: */*
< Content-Length: 116
< Date: Tue, 05 Dec 2017 10:57:19 GMT
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   116  100   116    0     0   2296      0 --:--:-- --:--:-- --:--:--     0Connection #0 to host seieadpa01 left intact

* Closing connection #0
++ grep link /export/home/kfoley/scripts/REST_API/Temp/test_xml.out
+ LINK='   <link>http://seieadpa01:9004/dpa-api/report/result/3b33e982-b899-4439-9a8b-ed8a3a89859a</link>'
+ LINK='http://seieadpa01:9004/dpa-api/report/result/3b33e982-b899-4439-9a8b-ed8a3a89859a</link>'
+ LINK=http://seieadpa01:9004/dpa-api/report/result/3b33e982-b899-4439-9a8b-ed8a3a89859a
+ export LINK
+ curl -v -u 'username:Password' -H Content-Type:application/vnd.emc.apollo-v1+xml http://seieadpa01:9004/dpa-api/report/result/3b33e982-b899-4439-9a8b-ed8a3a89859a
* About to connect() to seieadpa01 port 9004
*   Trying 172.16.120.152... connected
* Connected to seieadpa01 (172.16.120.152) port 9004
* Server auth using Basic with user 'admin'
> GET /dpa-api/report/result/3b33e982-b899-4439-9a8b-ed8a3a89859a HTTP/1.1
> Authorization: Basic YWRtaW46U3RvcmFnZTAwMSE=
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Host: seieadpa01:9004
> Accept: */*
> Content-Type:application/vnd.emc.apollo-v1+xml
>
< HTTP/1.1 503 Service Unavailable
< Server: Apache-Coyote/1.1
< Pragma: No-cache
< Cache-Control: no-cache
< Expires: Wed, 31 Dec 1969 19:00:00 EST
< X-FRAME-OPTIONS: SAMEORIGIN
< Retry-After: 2
< Content-Type: application/octet-stream
< Content-Length: 116
< Date: Tue, 05 Dec 2017 10:57:19 GMT
< Connection: close
<report>
   <link>http://seieadpa01:9004/dpa-api/report/result/3b33e982-b899-4439-9a8b-ed8a3a89859a</link>
Closing connection #0
</report>+ echo 'curl -u username:Password http://seieadpa01:9004/dpa-api/report/result/3b33e982-b899-4439-9a8b-ed8a3a89859a'
curl -u admin:Storage001! http://seieadpa01:9004/dpa-api/report/result/3b33e982-b899-4439-9a8b-ed8a3a89859a
[root@seiemccli01 REST_API]#

---------- Post updated at 12:08 PM ---------- Previous update was at 11:03 AM ----------

It must have something to do with the curl calls made earlier in the script that break this. If I just have a small script with just these 2 entries it works.

Code:
#!/bin/bash

LINK=http://seieadpa01:9004/dpa-api/report/result/8ffb82ec-c808-4a06-838a-9ac8dc7c93fa

curl -u username:Password $LINK

[root@seiemccli01 REST_API]#

The thing is that I need to generate the report first within the script using POST and then GET the report once I have generated it....

---------- Post updated at 12:23 PM ---------- Previous update was at 12:08 PM ----------

I decided to try to put a "
Code:
sleep 5

" before the last curl is executed and this has fixed the issue.

Thanks
This User Gave Thanks to kieranfoley For This Post:
Yoda
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Curl , download file with user:pass in bash script

Hello, My question is about curl command. (ubuntu14.04) In terminal, I am able to download my mainfile with: curl -u user1:pass1 http://11.22.33.44/******* When I convert it into bash script like this: #!/bin/bash cd /root/scripts computer_ip=11.22.33.44 curl -u $1:$2... (8 Replies)
Discussion started by: baris35
8 Replies

2. Shell Programming and Scripting

Access a complete flow with CURL + bash shell

Hello Experts , I have an use case which needed your help . I have been using google for 2 days buy couldn`t succed , i believe i can get the help here. Here is my use case to run on bash shell 1. Access an URL -- in script , it will be mentioned as inputURL 2. Once i accessed the URL... (5 Replies)
Discussion started by: radha254
5 Replies

3. Shell Programming and Scripting

Using curl in bash script

Hello. I have pick up a script from internet to track errors from curl command. #!/bin/bash # URL_TO_TEST="http://www.xxxxxx.yyy" MY_VAR=curl_init("$URL_TO_TEST") ; curl_setopt($MY_VAR, CURLOPT_HEADER, 1); curl_setopt($MY_VAR, CURLOPT_RETURNTRANSFER, 1); curl_setopt($MY_VAR,... (2 Replies)
Discussion started by: jcdole
2 Replies

4. Shell Programming and Scripting

awk script to find time difference between HTTP PUT and HTTP DELETE requests in access.log

Hi, I'm trying to write a script to determine the time gap between HTTP PUT and HTTP DELETE requests in the HTTP Servers access log. Normally client will do HTTP PUT to push content e.g. file_1.txt and 21 seconds later it will do HTTP DELETE, but sometimes the time varies causing some issues... (3 Replies)
Discussion started by: Juha
3 Replies

5. Shell Programming and Scripting

Sending awk variables into curl in a bash script

Hello experts! I have a file1 with the following format (yr,day, month, hour,minute): 201201132435 201202141210 201304132030 201410100110 ... What i want to do is to assign variables and then use them in the curl command to download the text of each event from a web page. What I have... (6 Replies)
Discussion started by: phaethon
6 Replies

6. Linux

Apt-get update wont run and gives 503 service unavailable error

Hi, I am running on Ubuntu 12 (precise) 64 bit and came to know that i cannot install any packages using apt-get and aptitude. Linux test01 3.5.0-23-generic #35~precise1-Ubuntu SMP Fri Jan 25 17:13:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux But the same /etc/apt/sources.list file is... (20 Replies)
Discussion started by: prash358
20 Replies

7. UNIX for Advanced & Expert Users

Syntax issue in curl for getting a file from http site

Hi All I am using curl in my client environment. We need to pull a file from the libraries site. All the files are having perticular URL which can be used to get the files. Only issue is, when we fire that command , it will take to authentication page 1st. once authentication is succed, it will... (1 Reply)
Discussion started by: prabirkumardhar
1 Replies

8. Shell Programming and Scripting

bash curl escape & in url

I'm running a curl command in bash, but the & in the middle causes the second half of the line to run in the background, here's what I'm trying to do: lat="37.451" lon="-122.18" url="http://ws.geonames.org/findNearestAddress?lat=$lat&lng=$lon" curl -s "$url" I tried escaping the & with \&,... (4 Replies)
Discussion started by: unclecameron
4 Replies

9. UNIX for Advanced & Expert Users

Issue in Curl to send http POST request with attachment/multipart

Hi, I am using curl to hit a url using http in solaris 10 using commandline, I want to transfer an attachment(using multipart curl -F) also as a part of the request. If anyone has used kindly help me with the syntax. I am using below command: Code: /usr/local/bin/curl -v... (1 Reply)
Discussion started by: manishmaha
1 Replies

10. Shell Programming and Scripting

Bash script idea using cUrl -- possible?

I have an alias already in my .bash_profile to download files using cUrl's -o (output to file, user provides the file name) option. I find I'm using it quite a bit, so I wanted to write a script to run "curl -o", taking the necessary inputs - file name and URL from which to download - and then... (3 Replies)
Discussion started by: SilversleevesX
3 Replies
Login or Register to Ask a Question