Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Querying a website at random bunches simultaneously
Top Forums Shell Programming and Scripting Querying a website at random bunches simultaneously Post 302750821 by phpchick on Wednesday 2nd of January 2013 02:35:25 PM
Old 01-02-2013
Querying a website at random bunches simultaneously
I have a script that hits my website for stock quotes and pulls prices. currently it does this one by one until all the symbols have been pulled. I've solved for bandwidth issues by putting in a random 1-10 second sleep function, but this will take too long.

I know that I can do everything at once by adding & at the end, but that will bomb out my website's resources.

Is there a way to hit my site with a random count of 5-10 symbols pulled all at once, every 1-10 second randomly?

Code:
while read line
do
    array+=("$line")
done < "../stocks/aaa-stockuniverse.txt"
#this reads all the tickers into an array


for ((i=0; i < ${#array[*]}; i++))
do
	eval $(curl -s "http://www.website.com/api?stock=${array[i]}"|sed 's/</\n</g' |sed '/data=/!d; s/ data=/=/g; s/\/>/; /g; s/</GF_/g' |tee ~/stocks/${array[i]}.txt)
	echo "${array[i]},$(date +%Y-%m-%d),$GF_open,$GF_high,$GF_low,$GF_last,$GF_volume"
	sleep $[ ( $RANDOM % 10 )  + 1 ]s
done

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Connecting to DB2 database on Mainframe and Querying DB

I am trying to develop a script that can connect to a DB2 database mainframe and Query the database and display the results. I've been researching, but I have yet to find a definitive solution where I can enter in the Mainframe DB2 address db2:// and query the database and return the results. If... (2 Replies)
Discussion started by: developncode
2 Replies

2. Shell Programming and Scripting

Querying database from unix

Hi, I have a shell script to query the database to get the statistics of tables like sum,max,min,etc of all numeric columns,max and min of date columns and length of varchar columns for each and every table. There are nearly 1600 tables. My script queries few tables like about 100 tables and... (3 Replies)
Discussion started by: ragavhere
3 Replies

3. HP-UX

Querying HP VA array

Hello, all... I've got an older HP-UX 11i machine hooked up to an HP VA array. I've never worked with VA arrays, so I was wondering if anyone could point me in the direction of some documentation on querying and manipulating VAs (HP VAs for Dummies or something). I'm used to EMC arrays, so... (2 Replies)
Discussion started by: kknigga
2 Replies

4. AIX

Servers still querying old DNS server?

Hello, I've created new DNS servers and changed all of the clients /etc/resolv.conf to point to them, but when I check the old DNS logs, I see that the clients are still querying it. Does anybody know why? thanks, (2 Replies)
Discussion started by: ctcuser
2 Replies

5. Solaris

commands hanging when querying hardware

Hi all Got another strange one. If I try to enquire about the hardware, the command hangs implying Ive got a hardware issue. So, if I execute :- iostat -en sysdef - ( stops at the devices part ) format cfgadm -al Anything that searches the devices, then the command hangs. ... (4 Replies)
Discussion started by: sbk1972
4 Replies

6. Red Hat

Error in querying file information

Hi, We have one application running where there is Linux Machine. For some some process I found the following message came, error 5.8/116455-02/116455-02.zip Error in querying file information: Permission denied :confused: If anybody has any idea for this please let me... (2 Replies)
Discussion started by: sptr
2 Replies

7. Ubuntu

expect script for random password and random commands

Hi I am new to expect. Please if any one can help on my issue its really appreciable. here is my issue: I want expect script for random passwords and random commands generation. please can anyone help me? Many Thanks in advance (0 Replies)
Discussion started by: vanid
0 Replies

8. Shell Programming and Scripting

Generic script for Querying a DB

I have a few databases that i need to get some basic output from... so I'll be returning 1 or 2 records from a variety of different databases and the input queries will always differ. What im looking for is a generic script that I can use across of the databases. Theory being i have: ... (2 Replies)
Discussion started by: atelford
2 Replies

9. Shell Programming and Scripting

Need to generate a file with random data. /dev/[u]random doesn't exist.

Need to use dd to generate a large file from a sample file of random data. This is because I don't have /dev/urandom. I create a named pipe then: dd if=mynamed.fifo do=myfile.fifo bs=1024 count=1024 but when I cat a file to the fifo that's 1024 random bytes: cat randomfile.txt >... (7 Replies)
Discussion started by: Devyn
7 Replies

10. Shell Programming and Scripting

Querying v$session table

Hi All, I'm using below code entry_data_control=`sqlplus ${sysuser}/${syspwd} << data_control whenever sqlerror exit sql.sqlcode; whenever oserror exit failure; INSERT INTO LSHADMIN.STUDY_LEVEL_TABLE(SESSION_SID,SESSION_SERIAL_NO,FILE_NAME) ... (3 Replies)
Discussion started by: Pratiksha Mehra
3 Replies

LEARN ABOUT SUSE

finance::quote::trustnet

Finance::Quote::Trustnet(3)				User Contributed Perl Documentation			       Finance::Quote::Trustnet(3)

NAME

       Finance::Quote::Trustnet - Obtain unit trust prices from trustnet.co.uk

SYNOPSIS

	   use Finance::Quote;

	   $q = Finance::Quote->new;

	   %stockinfo = $q->fetch("uk_unit_trusts","trust-name"); # Can failover to other methods
	   %stockinfo = $q->fetch("trustnet","trust-name"); # Use this module only.

DESCRIPTION

       This module obtains information about UK unit trust prices from www.trustnet.co.uk.  The information source "uk_unit_trusts" can be used if
       the source of prices is irrelevant, and "trustnet" if you specifically want to use trustnet.co.uk.

UNIT TRUST NAMES

       Unfortunately there is no unique identifier for unit trust names.  Therefore enough of the name should be given including spaces to yield a
       unique match.  Trustnet sometimes uses abbreviated names, and the string given should match the abbreviation.

       Consult http://www.trustnet.co.uk/ut/funds/perf.asp?sort=0 to find a match for your unit trusts.

       Example "jupiter income"

LABELS RETURNED

       Information available from Trustnet may include the following labels: exchange method source name currency bid ask yield price.	In case of
       a unit trust, "price" returns the offer (ask) price.  In case of an OIEC, the unique price is returned in "bid", "ask" and "price".

SEE ALSO

       Trustnet website - http://www.trustnet.co.uk/

       Finance::Quote::Yahoo::USA

perl v5.12.1							    2009-10-05					       Finance::Quote::Trustnet(3)
All times are GMT -4. The time now is 12:56 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy