Visit The New, Modern Unix Linux Community


Filter output in curl


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Filter output in curl
# 1  
Filter output in curl

Hello guys,

I'm writing a little script which sends me sms with my shell script via api of a sms provider.

problem is I can't filter my curl output for this site:

site url:
Code:
http://fahrplan.oebb.at/bin/query.exe/dn?start=1&_charset=UTF-8&REQ0JourneyProduct_list=0:1111111111011000-000000.&SALL=1&ZALL=1&scotty_advancedSearchMode=0&S=mureck&Z=graz%20puntigam&REQ0JourneyDate=21.08.2014&time=04:00&timeSel=depart

this is what i got so far, but I don't know how to filter everything out, I just want the times and also only the "ab time" like
Quote:
05:08 ab
Code:
#!/bin/bash
URL='http://fahrplan.oebb.at/bin/query.exe/dn?start=1&_charset=UTF-8&REQ0JourneyProduct_list=0:1111111111011000-000000.&SALL=1&ZALL=1&scotty_advancedSearchMode=0&S=mureck&Z=graz%20puntigam&REQ0JourneyDate=21.08.2014&time=04:00&timeSel=depart'

echo "Now:" `date +%H:%M:%S`
for URL in $URLS; do
echo "$URL:" | sed 's/.*=.*=/Route /'
curl -s $URL \

done

cheers
# 2  
Once again I haul out my micro xml parser for these questions... It looks like you want the contents of the very first <div class="planed">...</div> block.

Code:
$ cat planed.xml

BEGIN {
        FS=">"
        RS="<"
}

NR==1 { next } # The first "line" is blank when RS=<
/^[!?]/ {       next    }               # Skip XML specification junk
{       gsub(/[\r\n]*$/, " ");  }       # Clean up newlines

# Handle open-tags
match($0, /^[^\/ \r\n\t>]+/) {
        TAG=substr(toupper($1), RSTART, RLENGTH);
        TAGS=TAG "%" TAGS;
}

# Handle close-tags
/^[\/]/ {
        sub(/^\//, "", $1);
        sub("^.*" toupper($1) "%", "", TAGS);
        next;
}

(TAGS ~ /^DIV%/) && ($1 ~ /planed/) {
        gsub(/[\r\n]/, "", $2);
        print $2;
        exit;
}

$ awk -f planed.awk input.html

05:08 ab

$

The code highlighted in red is the statement which extracts the tag you want. The rest is generic XML parsing code.

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #68
Difficulty: Easy
Awk is abbreviated from the names of the developers - Aho, Weinberger, and Kernighan.
True or False?

10 More Discussions You Might Find Interesting

1. Web Development

Filename output in curl

How can I get the name of the default output filename from curl using the argument -O? Using -o one can choose a filename. I want to get the name of the original file, but don't understand how to get it. curl -o filename http://www.website.com curl -O http://www.website.com The... (3 Replies)
Discussion started by: locoroco
3 Replies

2. Shell Programming and Scripting

Extracting urls from curl output

Hello. I use curl to fetch a website, then, I want to extract the URLs from this curls output. I tried both sed and grep, but couldnt figure it out. Ive tried : sed -n 's/href="\(*\).*/\1/p' results.txt and grep -o grep -o '<a href="http://*.*.*/*">' results.txt. What pattern... (6 Replies)
Discussion started by: jozo95
6 Replies

3. Shell Programming and Scripting

How the filter output?

Hey, I'm using some sensors that can be read by http. If I run following command: curl -v 'http://192.168.111.23:8080/sensor/52f5c63cc4221fbbedcb499908a0a823?version=1.0&interval=minute&unit=watt&callback=realtime' I'm getting: I would like to put this now in a sheet with only the... (9 Replies)
Discussion started by: brononius
9 Replies

4. Shell Programming and Scripting

Encapsulating output of CURL and/or WGET

i use curl and wget quite often. i set up alarms on their output. for instance, i would run a "wget" on a url and then search for certain strings within the output given by the "wget". the problem is, i cant get the entire output or response of my wget/curl command to show up correctly in... (3 Replies)
Discussion started by: SkySmart
3 Replies

5. Shell Programming and Scripting

ery weird wget/curl output - what should I do?

Hi, I'm trying to write a script to download RedHat's errata digest. It comes in a txt.gz format, and i can get it easily with firefox. HOWEVER: output is VERY strange when donwloading it in a script. It seems I'm getting a file of the same size - but partially text and partly binary! It... (5 Replies)
Discussion started by: jstilby
5 Replies

6. Shell Programming and Scripting

web service call: curl output to xsltproc input

I need to invoke a web service and extract what I need from the response using a combination of curl and xsltproc. However, any file-based parameters that must be supplied to both these programs must be from stdin and not actual files. At least with curl, it seems to think that I am supplying a... (3 Replies)
Discussion started by: webuser
3 Replies

7. Shell Programming and Scripting

Getting cURL to output verbose to a file

This is about to drive me crazy. What I want to do is simple: output ALL the verbose information from curl to a file I have read the manual, tried several options and searched this forum but no salvation... I'm using curl -k -Q "command" --user user:passwd --ftp-pasv --ftp-ssl -v... (1 Reply)
Discussion started by: caramandi
1 Replies

8. Shell Programming and Scripting

script to output curl result as html

hi, new to scripting and would like to know how can I have a script which will curl a few URLs and have the results such as the URLs being curled, dns lookup time, connection time, total time, etc save in a html format in a form of table with column and rows. thank you. (4 Replies)
Discussion started by: squidusr
4 Replies

9. IP Networking

Filter wireshark output

Hi I have a wireshark file saved (from my network) and I have to analyze the flows inside it. The problem is that i have to analyze not the complete file (60.000 pkts!) but just a subset of it. In other words i have to sample the wireshark.file.dump and for example from 60.000 pkts take... (2 Replies)
Discussion started by: Dedalus
2 Replies

10. Shell Programming and Scripting

let curl output to stdout AND save to a file

hello hackers. i have a curl process running as cgi directly pushing stdout to the client. but i want to additionally save that stream to a file at the same time. any directions madly welcome. thanks in advance (3 Replies)
Discussion started by: scarfake
3 Replies

Featured Tech Videos