Sponsored Content
Special Forums UNIX and Linux Applications Virtualization and Cloud Computing Amazon CloudFront / S3 Small Object Test Results Post 302347596 by Neo on Wednesday 26th of August 2009 05:32:19 AM
Old 08-26-2009
I've not posted for a while, happy with CF/S3 performance - a happy camper as they say.....

However, something has always "bugged me" about CF/S3, and that is the need to create an origin-server side sync method to update S3/CF when dynamic content on an origin server is added (or existing content expires in the CDN - pull only from S3, etc).

For example, if you have a site where users upload their avatars, you need to sync with S3, either in near-time (best) or batch to keep S3 current. The same is true for any site with dynamic content. It is a lot of unnecessary extra work to initiate this on the origin service side, when it can be initiate from the client-user side.

In fact, I just noticed that SimpleCDN offers this service.... during a discussion at the link below, titled /clientscripts over to Amazon S3/Cloudfront?

/clientscripts over to Amazon S3/Cloudfront? - vBulletin.org Forum

And in particular, this reply to someone who introduced the SimpleCDN feature in the discussion thread:

vBulletin.org Forum - View Single Post - /clientscripts over to Amazon S3/Cloudfront?

I have been informed that SimpleCDN does have a feature where, if a file is
required from the CDN and it is not yet uploaded, SimpleCDN will pull it from the origin server. This is a very good feature and we are considering to try it now because of this. (!!)

Also, I note that S3/CF do not have this functionality (presently) and is not currently on the AWS roadmap.

I think this feature is critical now, especially them more I think about it. In fact, it is a "game changer".... as folks can move to the CDN and then (effortlessly)"on demand" populate the CDN by having the CDN (or S3 in the AWS case) pull from the origin server. Also dynamic changes to any site means almost "zero configuration" and no requires sync / upload to the CDN... and yea this is goodness for all!

As a fan of Amazon CF/S3, I would like to see this feature soon. More than likely, we will move some user-dynamic content over to SimpleCDN because of this feature that, if effect, works like "cache-miss, pull from origin server"). Very user friendly.... and also aligns with "the spirit of the Internet".... (like DNS, etc. where content is cached and pulled from the origin server when not available, cache timeouts, etc.) Having S3 as "the final frontier" is sub-optimal.

Right now, it is (1) Try CF, if CF does not have a file, (2) pull from S3, if S3 has no file, return "file not found" (404 I guess...) .... Better would be:

(1) Try CF, if CF does not have a file (2) pull from S3, if S3 has no file, (3) try origin server, if origin server has not file, then return "file not found".

Or, if you really want to serve AWS customers, offer this option..... :-)

(1) Try CF, if CF does not have a file (2) pull from origin server, if origin server has not file, then return "file not found" (or try S3 .... :-)

I assume that AWS wants to require people to store in S3 and will not permit CF to pull from the origin server..... OK... business is business, but please permit, at least, S3 to pull from origin server.

I will update some stats from our SimpleCDN tests soon.
 

3 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Printer Error(the Object Instance Test Does Not Exist)

Hello, i need some help about how to set up a high velocity impact printer in UNIX SCO 5.05, this printer is attached with a parallel port in a PC(host), the host use tunemul to access unix.(this reference is just to ask you if this is a local or remote connection, just to be sure), so, i... (2 Replies)
Discussion started by: jav_v
2 Replies

2. Virtualization and Cloud Computing

CEP as a Service (CEPaaS) with MapReduce on Amazon EC2 and Amazon S3

Tim Bass 11-25-2008 01:02 PM Just as I was starting to worry that complex event processing community has been captured by RDBMS pirates off the coast of Somalia, I rediscovered a new core blackboard architecture component, Hadoop. Hadoop is a framework for building applications on large... (0 Replies)
Discussion started by: Linux Bot
0 Replies

3. Shell Programming and Scripting

PERL - traverse sub directories and get test case results

Hello, I need help in creating a PERL script for parsing test result files to get the results (pass or fail). Each test case execution generates a directory with few files among which we are interested in .result file. Lets say Testing is home directory. If i executed 2 test cases. It will... (4 Replies)
Discussion started by: ravi.videla
4 Replies
Amazon::SQS::Simple(3pm)				User Contributed Perl Documentation				  Amazon::SQS::Simple(3pm)

NAME
Amazon::SQS::Simple - OO API for accessing the Amazon Simple Queue Service SYNOPSIS
use Amazon::SQS::Simple; my $access_key = 'foo'; # Your AWS Access Key ID my $secret_key = 'bar'; # Your AWS Secret Key # Create an SQS object my $sqs = new Amazon::SQS::Simple($access_key, $secret_key); # Create a new queue my $q = $sqs->CreateQueue('queue_name'); # Send a message $q->SendMessage('Hello world!'); # Retrieve a message my $msg = $q->ReceiveMessage(); print $msg->MessageBody() # Hello world! # Delete the message $q->DeleteMessage($msg->ReceiptHandle()); # Delete the queue $q->Delete(); INTRODUCTION
Amazon::SQS::Simple is an OO API for the Amazon Simple Queue Service. IMPORTANT
This version of Amazon::SQS::Simple defaults to work against version 2009-02-01 of the SQS API. Earlier API versions may or may not work. CONSTRUCTOR
new($access_key, $secret_key, [%opts]) Constructs a new Amazon::SQS::Simple object $access_key is your Amazon Web Services access key. $secret_key is your Amazon Web Services secret key. If you don't have either of these credentials, visit <http://aws.amazon.com/>. Options for new: Timeout => SECONDS Set the HTTP user agent's timeout (default is 180 seconds) Version => VERSION_STRING Specifies the SQS API version you wish to use. E.g.: my $sqs = new Amazon::SQS::Simple($access_key, $secret_key, Version => '2008-01-01'); METHODS
GetQueue($queue_endpoint) Gets the queue with the given endpoint. Returns a "Amazon::SQS::Simple::Queue" object. (See Amazon::SQS::Simple::Queue for details.) CreateQueue($queue_name, [%opts]) Creates a new queue with the given name. Returns a "Amazon::SQS::Simple::Queue" object. (See Amazon::SQS::Simple::Queue for details.) Options for CreateQueue: DefaultVisibilityTimeout => SECONDS Set the default visibility timeout for this queue ListQueues([%opts]) Gets a list of all your current queues. Returns an array of "Amazon::SQS::Simple::Queue" objects. (See Amazon::SQS::Simple::Queue for details.) Options for ListQueues: QueueNamePrefix => STRING Only those queues whose name begins with the specified string are returned. FUNCTIONS
No functions are exported by default; if you want to use them, export them in your use line: use Amazon::SQS::Simple qw( timestamp ); timestamp($seconds) Takes a time in seconds since the epoch and returns a formatted timestamp suitable for using in a Timestamp or Expires optional method parameter. STANDARD OPTIONS
The following options can be supplied with any of the listed methods. AWSAccessKeyId => STRING The AWS Access Key Id to use with the method call. If not provided, Amazon::SQS::Simple uses the value passed to the constructor. SecretKey => STRING The Secret Key to use with the method call. If not provided, Amazon::SQS::Simple uses the value passed to the constructor. Timestamp => TIMESTAMP All methods are automatically given a timestamp of the time at which they are called, but you can override this value if you need to. The value for this key should be a timestamp as returned by the Amazon::SQS::Simple::timestamp() function. You generally do not need to supply this option. Expires => TIMESTAMP All methods are automatically given a timestamp of the time at which they are called. You can alternatively set an expiry time by providing an Expires option. The value for this key should be a timestamp as returned by the "Amazon::SQS::Simple::timestamp()" function. You generally do not need to supply this option. ACKNOWLEDGEMENTS
Bill Alford wrote the code to support basic functionality of older API versions in release 0.9. AUTHOR
Copyright 2007-2008 Simon Whitaker <swhitaker@cpan.org> This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.10.1 2010-03-31 Amazon::SQS::Simple(3pm)
All times are GMT -4. The time now is 05:50 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy