Filter output in curl Post: 302914017

Sponsored Content

Top Forums Shell Programming and Scripting Filter output in curl Post 302914017 by Corona688 on Thursday 21st of August 2014 11:13:56 AM

08-21-2014

Registered User

Once again I haul out my micro xml parser for these questions... It looks like you want the contents of the very first <div class="planed">...</div> block.

Code:

$ cat planed.xml

BEGIN {
        FS=">"
        RS="<"
}

NR==1 { next } # The first "line" is blank when RS=<
/^[!?]/ {       next    }               # Skip XML specification junk
{       gsub(/[\r\n]*$/, " ");  }       # Clean up newlines

# Handle open-tags
match($0, /^[^\/ \r\n\t>]+/) {
        TAG=substr(toupper($1), RSTART, RLENGTH);
        TAGS=TAG "%" TAGS;
}

# Handle close-tags
/^[\/]/ {
        sub(/^\//, "", $1);
        sub("^.*" toupper($1) "%", "", TAGS);
        next;
}

(TAGS ~ /^DIV%/) && ($1 ~ /planed/) {
        gsub(/[\r\n]/, "", $2);
        print $2;
        exit;
}

$ awk -f planed.awk input.html

05:08 ab

$

The code highlighted in red is the statement which extracts the tag you want. The rest is generic XML parsing code.

Corona688

View Public Profile for Corona688

Visit Corona688's homepage!

Find all posts by Corona688

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

let curl output to stdout AND save to a file

hello hackers. i have a curl process running as cgi directly pushing stdout to the client. but i want to additionally save that stream to a file at the same time. any directions madly welcome. thanks in advance

2. IP Networking

Filter wireshark output

Hi I have a wireshark file saved (from my network) and I have to analyze the flows inside it. The problem is that i have to analyze not the complete file (60.000 pkts!) but just a subset of it. In other words i have to sample the wireshark.file.dump and for example from 60.000 pkts take...

3. Shell Programming and Scripting

script to output curl result as html

hi, new to scripting and would like to know how can I have a script which will curl a few URLs and have the results such as the URLs being curled, dns lookup time, connection time, total time, etc save in a html format in a form of table with column and rows. thank you.

4. Shell Programming and Scripting

Getting cURL to output verbose to a file

This is about to drive me crazy. What I want to do is simple: output ALL the verbose information from curl to a file I have read the manual, tried several options and searched this forum but no salvation... I'm using curl -k -Q "command" --user user:passwd --ftp-pasv --ftp-ssl -v...

5. Shell Programming and Scripting

web service call: curl output to xsltproc input

I need to invoke a web service and extract what I need from the response using a combination of curl and xsltproc. However, any file-based parameters that must be supplied to both these programs must be from stdin and not actual files. At least with curl, it seems to think that I am supplying a...

6. Shell Programming and Scripting

ery weird wget/curl output - what should I do?

Hi, I'm trying to write a script to download RedHat's errata digest. It comes in a txt.gz format, and i can get it easily with firefox. HOWEVER: output is VERY strange when donwloading it in a script. It seems I'm getting a file of the same size - but partially text and partly binary! It...

7. Shell Programming and Scripting

Encapsulating output of CURL and/or WGET

i use curl and wget quite often. i set up alarms on their output. for instance, i would run a "wget" on a url and then search for certain strings within the output given by the "wget". the problem is, i cant get the entire output or response of my wget/curl command to show up correctly in...

8. Shell Programming and Scripting

How the filter output?

Hey, I'm using some sensors that can be read by http. If I run following command: curl -v 'http://192.168.111.23:8080/sensor/52f5c63cc4221fbbedcb499908a0a823?version=1.0&interval=minute&unit=watt&callback=realtime' I'm getting: I would like to put this now in a sheet with only the...

9. Shell Programming and Scripting

Extracting urls from curl output

Hello. I use curl to fetch a website, then, I want to extract the URLs from this curls output. I tried both sed and grep, but couldnt figure it out. Ive tried : sed -n 's/href="\(*\).*/\1/p' results.txt and grep -o grep -o '<a href="http://*.*.*/*">' results.txt. What pattern...

10. Web Development

Filename output in curl

How can I get the name of the default output filename from curl using the argument -O? Using -o one can choose a filename. I want to get the name of the original file, but don't understand how to get it. curl -o filename http://www.website.com curl -O http://www.website.com The...

LEARN ABOUT MOJAVE

curlinfo_http_version

CURLINFO_HTTP_VERSION(3)				     curl_easy_getinfo options					  CURLINFO_HTTP_VERSION(3)

NAME

       CURLINFO_HTTP_VERSION - get the http version used in the connection

SYNOPSIS

       #include <curl/curl.h>

       CURLcode curl_easy_getinfo(CURL *handle, CURLINFO_HTTP_VERSION, long *p);

DESCRIPTION

       Pass  a	pointer  to  a	long  to  receive the version used in the last http connection.  The returned value will be CURL_HTTP_VERSION_1_0,
       CURL_HTTP_VERSION_1_1, or CURL_HTTP_VERSION_2_0, or 0 if the version can't be determined.

PROTOCOLS

       HTTP

EXAMPLE

       CURL *curl = curl_easy_init();
       if(curl) {
	 CURLcode res;
	 curl_easy_setopt(curl, CURLOPT_URL, "http://example.com");
	 res = curl_easy_perform(curl);
	 if(res == CURLE_OK) {
	   long http_version;
	   curl_easy_getinfo(curl, CURLINFO_HTTP_VERSION, &http_version);
	 }
	 curl_easy_cleanup(curl);
       }

AVAILABILITY

       Added in 7.50.0

RETURN VALUE

       Returns CURLE_OK if the option is supported, and CURLE_UNKNOWN_OPTION if not.

SEE ALSO

       CURLINFO_RESPONSE_CODE(3), curl_easy_getinfo(3), curl_easy_setopt(3),

libcurl 7.54.0							   May 11, 2016 					  CURLINFO_HTTP_VERSION(3)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

let curl output to stdout AND save to a file

Discussion started by: scarfake

2. IP Networking

Filter wireshark output

Discussion started by: Dedalus

3. Shell Programming and Scripting

script to output curl result as html

Discussion started by: squidusr

4. Shell Programming and Scripting

Getting cURL to output verbose to a file

Discussion started by: caramandi