Summary
I am going to design and implement an Elastic Webserver Platform using AWS that allows you to specify scheduling and scaling policies programmatically utilizing characteristics of the incoming requests and knowledge of the type of workload. I will them attempt to beat Amazon's Elastic Load Balancer(ELB)[1] and Auto Scale Groups[2] performance at the same cost/hour for a variety of workload/cost combinations.
Background
Amazon offers an existing solution to create an Elastic Webserver on AWS using the Elastic Load Balancer and Auto Scale Group combination which fulfill some of the requirements of an Elastic Webserver like Availability and Elasticity however this approach has several limitations for real workloads which I look to address with my solution. Some of the limitations are:-
- The ELB uses a naive Round Robin Load Balancing approach to balance incoming requests among various servers however this approach is inefficient when the workload associated with each request is imbalanced which is usually the case for real world loads.
- Auto Scale Group can scale based upon only a limited set of Cloud Watch Metrics[3].
- Auto Scale Group only allows Homogeneous scaling out of resources. We can achieve better performance for the same cost if we scale out heterogeneously utilizing the knowledge of the type of workload which has increased(For e.g. launching a C series compute optimized instance may be more cost effective than launching multiple smaller instances).
The Challenge
- An Elastic Webserver must provide Availability and Elasticity. I would need to keep track of worker node health to mark workers as dead or alive. Also the load balancer is a bottleneck in the system and there is no way to scale a EC2 instance vertically on AWS in real-time. I will have to figure out a solution for this.
- Beating the performance at lower cost points will be difficult as the ELB is cheap at $0.025 an hour and a additional $0.8 for every 100GB through the load balancer whereas I would have to run the load balancer on an EC2 instance and even reasonably low power instances are much more expensive than the ELB. I will have to make up the cost in scheduling and scaling efficiency.
- I may face several hurdles along the way due to limitations of the AWS API and would have to find workarounds for those.
- Maintaining worker state if multiple load balancers are deployed to mitigate a SPOF.
Resources
The entire platform will be built on top of Amazon EC2 and will be built using AWS Java API. I plan on using the Undertow Web Framework[4] for the underlying Load Balancers and Web Server's HTTP components. I will look into using JMeter[5] as the load generator.
Goals and Deliverables
Must Haves
- Framework for scheduling and scaling based on desired algorithm with access to different worker state metrics.
- Horizontal scaling and Fault tolerance of Worker nodes
- Cost/Performance ratio competitive with ELB and ASG for the tested configurations with the implemented scheduling algorithms.
- Detailed analysis of the results explaining/demonstrating how work distribution and heterogeneity helped in achieving the performance goals.
Should Haves
- Vertical scaling of the Load Balancer.
- Easy to use configuration file to get setup and implement a solution based on the framework on AWS.
Platform Choice
I plan to use Java and use Java Web Framework for this as the AWS Java SDK is the most feature rich and stable API and Java is a popular language for Web Development and unlike PHP allows to explore the underlying parallelism. Amazon AWS is the cloud platform of choice for a majority of Salable Web Apps which require Platform as a Service and hence I chose AWS.
Schedule
| Time | Proposed Schedule | Tasks Accomplished |
|---|---|---|
| Week April 2-8 | ++Decide on all the libraries/frameworks to be used and read about them ++Build a basic Proxy with fixed set of Workers along with porting various workloads from assignment and writing more if need arises. |
++Read about JMeter and Undertow. ++Read about AWS Java SDK ++Built basic Undertow proxy |
| Week April 9-15 | ++Build basic Autoscaling API on top of AWS and get have the API get access to all the worker metrics. ++Build API to perform health checks on workers and mark/unmark unhealthy/healthy instances. ++Build various ELB and ASG configurations and run tests to get the baseline performance/cost comparisons. |
++Build Basic Java API on top of AWS for scaling and basic load balancing ++Thought about types of Traces |
| Week April 16-22 | ++Work on scheduling and scaling strategies and optimize performance. | ++Worked on Traces |
| Week April 23-29 | ++Write additional test traces and workloads. ++Further optimize performance |
|
| Week April 30-Competition | ++Finish any pending work and additional features. ++Finish Final write-up and Presentation. |