The UNIX and Linux Forums  
Hello and Welcome from United States to the UNIX and Linux Forums! Thank You for Visiting and Joining Our Global Community.

Go Back   The UNIX and Linux Forums > Special Forums > Virtualization and Cloud Computing
.
google unix.com



Virtualization and Cloud Computing Xen, VMWare, hypervisors, Amazon Elastic Computing Cloud, EC2, virtual storage, Amazon S3 and other cloud computing topics.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Uploading Data to Amazon S3 Using CloudFront iBot Complex Event Processing RSS News 0 05-07-2009 03:45 PM
CloudFront LogAnalyzer on Amazon Elastic MapReduce iBot Complex Event Processing RSS News 0 05-07-2009 11:30 AM
CEP as a Service (CEPaaS) with MapReduce on Amazon EC2 and Amazon S3 iBot Virtualization and Cloud Computing 0 11-25-2008 02:08 PM
Printer Error(the Object Instance Test Does Not Exist) jav_v UNIX for Advanced & Expert Users 2 09-23-2002 10:42 PM

Reply
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 05-22-2009
Neo's Avatar
Neo Neo is online now Forum Staff  
Administrator
  
 

Join Date: Sep 2000
Location: Asia Pacific
Posts: 6,680
Amazon CloudFront / S3 Small Object Test Results

After some initial confusing results measuring "before-and-after CloudFront/S3" performance, Dennis Opacki, Systems Operations Manager for Amazon Cloudfront offered to help. Working with the test results from a number of members of The UNIX and Linux Forums pulling a small 1623 byte gif object with curl -s -w, we compiled the test results. For each location, we discarded the fastest and slowest times for the Origin Web Server (in Ireland) and the CloudFront (CF) sites and averaged the remaining three samples. The last two columns in the table below represents the difference in performance between the objects hosted. The data seems to suggest a fairly significant performance improvement (with Amazon CloudFront) for all locations worldwide except Ireland, which is not surprising, since Ireland is the location of the original server.


.Web Server (WS) in IrelandCloudFront (CF)Compare Performance (CF/WS)
Client LocationWS First-byte(s)WS Total Transfer(s)CF First-byte(s)CF Total(s)First-byte (%)Total Transfer (%)
Ireland 0.096 0.0970.1010.1016%4%
Netherlands0.1230.1240.0810.081-34%-35%
St Paul, MN0.4720.4740.2840.286-39%-40%
Sofia, Bulgaria0.1340.1340.0810.081-40%-40%
Milan, Italy0.1110.1120.0420.042-63%-63%
Seattle, WA0.5150.5160.1330.134-74%-74%
Redwood City, CA0.3240.3270.0410.045-86%-86%
Ashburn, VA0.1880.1890.0220.022-88%-88%
Hong Kong0.9440.9440.0480.048-95%-95%

As expected, the most improvement was from users in the US and Asia (Hong Kong), since the original web server resides in Ireland.

I want to thank all the members of The Unix and Linux Forums who took the time to run the tests and for Dennis of Amazon for helping compile the results.

Congratulations Amazon Web Services and CloudFront!!

Note: Our test results suggest that static web content hosted on Amazon's CDN can certainly help global user response time downloading (small) objects. This is what you would expect from a CDN.
  #2 (permalink)  
Old 06-15-2009
Neo's Avatar
Neo Neo is online now Forum Staff  
Administrator
  
 

Join Date: Sep 2000
Location: Asia Pacific
Posts: 6,680
FYI: I have also posted these results to the AWS CF forum.

-----Post Update-----

Earlier, I forgot to thank Stefan for his vB forum BBCODE that makes posting a nice table possible, BB Code [table]. Thanks Stefan!!
  #3 (permalink)  
Old 08-26-2009
Neo's Avatar
Neo Neo is online now Forum Staff  
Administrator
  
 

Join Date: Sep 2000
Location: Asia Pacific
Posts: 6,680
I've not posted for a while, happy with CF/S3 performance - a happy camper as they say.....

However, something has always "bugged me" about CF/S3, and that is the need to create an origin-server side sync method to update S3/CF when dynamic content on an origin server is added (or existing content expires in the CDN - pull only from S3, etc).

For example, if you have a site where users upload their avatars, you need to sync with S3, either in near-time (best) or batch to keep S3 current. The same is true for any site with dynamic content. It is a lot of unnecessary extra work to initiate this on the origin service side, when it can be initiate from the client-user side.

In fact, I just noticed that SimpleCDN offers this service.... during a discussion at the link below, titled /clientscripts over to Amazon S3/Cloudfront?

/clientscripts over to Amazon S3/Cloudfront? - vBulletin.org Forum

And in particular, this reply to someone who introduced the SimpleCDN feature in the discussion thread:

vBulletin.org Forum - View Single Post - /clientscripts over to Amazon S3/Cloudfront?

I have been informed that SimpleCDN does have a feature where, if a file is
required from the CDN and it is not yet uploaded, SimpleCDN will pull it from the origin server. This is a very good feature and we are considering to try it now because of this. (!!)

Also, I note that S3/CF do not have this functionality (presently) and is not currently on the AWS roadmap.

I think this feature is critical now, especially them more I think about it. In fact, it is a "game changer".... as folks can move to the CDN and then (effortlessly)"on demand" populate the CDN by having the CDN (or S3 in the AWS case) pull from the origin server. Also dynamic changes to any site means almost "zero configuration" and no requires sync / upload to the CDN... and yea this is goodness for all!

As a fan of Amazon CF/S3, I would like to see this feature soon. More than likely, we will move some user-dynamic content over to SimpleCDN because of this feature that, if effect, works like "cache-miss, pull from origin server"). Very user friendly.... and also aligns with "the spirit of the Internet".... (like DNS, etc. where content is cached and pulled from the origin server when not available, cache timeouts, etc.) Having S3 as "the final frontier" is sub-optimal.

Right now, it is (1) Try CF, if CF does not have a file, (2) pull from S3, if S3 has no file, return "file not found" (404 I guess...) .... Better would be:

(1) Try CF, if CF does not have a file (2) pull from S3, if S3 has no file, (3) try origin server, if origin server has not file, then return "file not found".

Or, if you really want to serve AWS customers, offer this option..... :-)

(1) Try CF, if CF does not have a file (2) pull from origin server, if origin server has not file, then return "file not found" (or try S3 .... :-)

I assume that AWS wants to require people to store in S3 and will not permit CF to pull from the origin server..... OK... business is business, but please permit, at least, S3 to pull from origin server.

I will update some stats from our SimpleCDN tests soon.
Reply

Bookmarks

Tags
amazon web services, aws, cloudfront, cloudfront performance, cloudfront test results, s3

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 05:41 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0