wget performance


 
Thread Tools Search this Thread
Operating Systems Linux wget performance
# 1  
Old 04-14-2011
wget performance

I am looking for a way to measure performance metrics of streaming audio/video from a contecnt server, e.g. YouTube for example. Im keen to see if I can look at duration it took for contecnt to download.
I know from the output of wget's log file you can see duration a url is downloaded in.
However I need to get the duration it takes for the streaming content.
Thanks
# 2  
Old 04-14-2011
For streaming, wget is not going to work. If you try to download a youtube page with it, it will simply download the source code of the page, the JS and the HTML, but not the video itself.
Here's a python script I use for downloading youtube videos to my workstation. You can see that it's not a simple one. I guess you should either write your own utility and work directly on the server providing the content, or look for commercial tools.
This User Gave Thanks to sysgate For This Post:
# 3  
Old 04-14-2011
Many Thanks Sysgate, however your python script is currently way beyond me. I was hoping there was some alternative to wget to actual download the video / audio content so I can peform some performanc eanalysis on it
# 4  
Old 04-14-2011
Yes, it's a bit complicated, for me, too, but it works. Actually, seeing your reply, I did search with another keywords, and I found this - if it's really that simple as it claims, you can work it out.
Else, I'm out of suggestions at this point.
This User Gave Thanks to sysgate For This Post:
# 5  
Old 04-14-2011
unfortunately I dont have clive installed currently and actually not sure how to install. It also mentions the requirement on having Adobe Flash to view the content. I have no desire to view. I simply want to cron a script that continually streams some content and to be able to work out how slow/fast it does so.

Many thanks
# 6  
Old 04-14-2011
Quote:
Originally Posted by rob171171
I am looking for a way to measure performance metrics of streaming audio/video from a contecnt server, e.g. YouTube for example. Im keen to see if I can look at duration it took for contecnt to download.
I know from the output of wget's log file you can see duration a url is downloaded in.
However I need to get the duration it takes for the streaming content.
Thanks
A file's a file's a file. Things like Youtube and the like work real hard to obscure where the actual URL comes from, though. The only even slightly reliable way I've found to figure that out is to set up your own proxy server, configure your web browser to use it, and watch the URLs it downloads for anything relevant. And even that sometimes doesn't work because when you try and download it by hand, you won't have the same cookies, etc.
# 7  
Old 04-15-2011
have downloaded the clive 2.2.24 but now need some advice as to how I install so if anyone wants a quick win today pls let me know. Thanks
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies

2. Shell Programming and Scripting

Wget and gz

Can wget be used to goto a site and piped into a .gz extrated command? wget ftp://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_GRCh37 | gunzip -d clinvar_20150603.vcf.gz (1 Reply)
Discussion started by: cmccabe
1 Replies

3. Red Hat

Wget

If I run the following command wget -r --no-parent --reject "index.html*" 10.11.12.13/backups/ A local directory named 10.11.12.13/backups with the content of web site data is created. What I want to do is have the data placed in a local directory called $HOME/backups. Thanks for... (1 Reply)
Discussion started by: popeye
1 Replies

4. Shell Programming and Scripting

WGET help!

Hi Friends, I have an url like this https://www.unix.com/help/ In this help directory, I have more than 300 directories which contains file or files. So, the 300 directories are like this http://unix.com/help/ dir1 file1 dir2 file2 dir3 file3_1 file3_2... (4 Replies)
Discussion started by: jacobs.smith
4 Replies

5. UNIX for Dummies Questions & Answers

Wget

...... (1 Reply)
Discussion started by: hoo
1 Replies

6. Shell Programming and Scripting

wget help?

can someone please help in understanding this shell script? wget --progress=dot:mega --cut-dirs=4 -r -c -nH -np --reject index.html*,icons/*.gif \ http://*****.oz.xxxxx.com:<portnum>/omcsm/releases/dew/${UPGRADE_VERSION}/ (1 Reply)
Discussion started by: dnam9917
1 Replies

7. Shell Programming and Scripting

Performance Metrics using wget

I am looking for a way to measure performance metrics of streaming audio/video from a contecnt server, e.g. YouTube for example. Im keen to see if I can look at duration it took for contecnt to download. I know from the output of wget's log file you can see duration a url is downloaded in.... (5 Replies)
Discussion started by: rob171171
5 Replies

8. News, Links, Events and Announcements

Announcing collectl - new performance linux performance monitor

About 4 years ago I wrote this tool inspired by Rob Urban's collect tool for DEC's Tru64 Unix. What makes this tool as different as collect was in its day is its ability to run at a low overhead and collect tons of stuff. I've expanded the general concept and even include data not available in... (0 Replies)
Discussion started by: MarkSeger
0 Replies

9. Shell Programming and Scripting

wget help

i am trying to ftp files/dirs with wget. i am having an issue where the path always takes me to my home dir even when i specify something else. For example: wget -m ftp://USER:PASS@IP_ADDRESS/Path/on/remote/box ...but if that path on the remote box isn't in my home dir it doesn't change to... (0 Replies)
Discussion started by: djembeplayer
0 Replies

10. Shell Programming and Scripting

wget -r

I have noticed a lot of expensive books appearing online so I have decided to copy them to CD. I was going to write a program in java to do this, but remembered that wget GNU program some of you guys were talking about. Instead of spending two hours or so writing a program to do this.... (1 Reply)
Discussion started by: photon
1 Replies
Login or Register to Ask a Question