CASE STUDY OF AWS SQS

What is Amazon Simple Queue Service?

📝Amazon Simple Queue Service (Amazon SQS) is a pay-per-use web service for storing messages in transit between computers. Developers use SQS to build distributed applications with decoupled components without having to deal with the overhead of creating and maintaining message queues.

Amazon Simple Queue Service supports tasks that process asynchronously. Instead of one application having to invoke another application directly, the service enables an application to submit a message to a queue, which another application can then pick up at a later time.

An SQS queue can be FIFO (first-in, first-out) or standard. A FIFO queue maintains the exact order in which message strings are sent and received. Standard queues attempt to preserve the order of messages, but can be flexible and when processing demands require it. FIFO queues provide exactly one delivery, while standard queues provide at least one delivery.

SQS is compatible with other Amazon Web Services, including Amazon Relational Database Service, Amazon Elastic Compute Cloud, and Amazon Simple Storage Service.

Let’s understand a few things about Amazon SQS:

• SQS stands for Simple Queue Service.
• SQS was the first service available in AWS.
• Amazon SQS is a web service that gives you access to a message queue that can be used to store messages while waiting for a computer to process them.
• Amazon SQS is a distributed queue system that enables web service applications to quickly and reliably queue messages that one component in the application generates to be consumed by another component where a queue is a temporary repository for messages that are awaiting processing.
• With the help of SQS, you can send, store and receive messages between software components at any volume without losing messages.
• Using Amazon SQS, you can separate the components of an application so that they can run independently, easing message management between components.
• Any component of a distributed application can store the messages in the queue.
• Messages can contain up to 256 KB of text in any format such as JSON, XML, etc.
• Any component of an application can later retrieve the messages programmatically using the Amazon SQS API.
• The queue acts as a buffer between the component producing and saving data, and the component receives the data for processing. This means that the queue resolves issues that arise if the producer is producing work faster than the consumer can process it, or if the producer or consumer is only intermittently connected to the network.
• If you got two EC2 instances that are pulling the SQS Queue. You can configure the auto-scaling group if a number of messages go over a certain limit. Suppose the number of messages exceeds 10, then you can add an additional EC2 instance to process the job faster. In this way, SQS provides elasticity.

Let’s understand with an example:

📝Let’s look at a website that generates a Meme. Suppose the user wants to upload a photo and wants to convert into Meme. User uploads a photo on a website and website might store a photo in s3. As soon as it finished uploads, it triggers a Lambda function. Lambda analyzes the data about this particular image to SQS, and this data can be “what the top of the meme should say”, “what the bottom of the meme should say”, the location of the S3 bucket, etc. The data sits inside the SQS as a message. An EC2 instance looks at the message and performs its job. An EC2 instance creates a Meme and stores it in S3 bucket. Once the EC2 instance completed its job, it moves back to the SQS. The best thing is that if you lose your EC2 instance, then also you would not lose the job as the job sits inside the S3 bucket.

Types of Queue:

There are two types of Queue:

👉Standard Queues (default)

👉FIFO Queues (First-In-First-Out)

Standard Queue:

• SQS offers a standard queue as the default queue type.
• It allows you to have an unlimited number of transactions per second.
• It guarantees that a message is delivered at least once. However, sometimes, more than one copy of a message might be delivered out of order.
• It provides best-effort ordering which ensures that messages are generally delivered in the same order as they are sent but it does not provide a guarantee.

FIFO Queue:

• The FIFO Queue complements the standard Queue.
• It guarantees to order, i.e., the order in which they are sent is also received in the same order.
• The most important features of a queue are FIFO Queue and exactly-once processing, i.e., a message is delivered once and remains available until the consumer processes and deletes it.
• FIFO Queue does not allow duplicates to be introduced into the Queue.
• It also supports message groups that allow multiple ordered message groups within a single Queue.
• FIFO Queues are limited to 300 transactions per second but have all the capabilities of standard queues.

Let’s understand about visibility timeout:

📝Amazon SQS doesn’t automatically delete the message once the consumer receives and processes the message from the queue. As SQS is distributed in nature, they can be shared across multiple consumers to consume the messages.

To prevent other consumers from processing the message again, the message is hidden from other consumers for a specific period of time. The time window is called Visibility Timeout. The consumer should make a delete call to delete the messages from the queue. Amazon SQS requires you to delete the message, just that in the JMS world, the consumer acknowledges that it is done processing and the JMS broker takes care of removing the message from the queue after processing or consumed by the consumer.

Deletion should happen within that message visibility timeout period because it is during this period that the message is not available to other consumers. Once the message is visible again, there may be chances for reprocessing it.

The below diagram explains the concept of Message Visibility Timeout.

Advantages:

• Fully managed turn-key messaging infrastructure. SQS queues require very little effort to set up, maintain and manage over time.
• Integrates seamlessly with other services provided by AWS, such as IAM, CloudWatch, and Lambda. For organizations already committed to AWS, SQS is a natural choice.
• Can be used as a gateway between endpoints that may not have direct connectivity to each-other.
• Can send and receive large messages that exceed the queue limitations by storing large payloads in S3. For more information review the documentation for the transport topology and configuration options.

Disadvantages:

• Like other message brokers, there is no local store-and-forward mechanism available. If an endpoint cannot reach SQS, either due to network problems or if SQS is unavailable, the endpoint will not be able to send nor receive messages.
• Can be relatively expensive when using larger volumes of messages.

RedBus Case Study

📝redBus is an Indian travel agency that specializes in bus travel throughout India by selling bus tickets throughout the country. Tickets are purchased through the company’s Website or through the Web services of its agents and partners. The company also offers software, on a Software as a Service (SaaS) basis, which gives bus operators the option of handling their own ticketing and managing their own inventories. To date, the company says they have sold over 30 million bus tickets and has more than 1750 bus operators using the software to manage their operations.

With the time savings that the IT and development staffs obtain from the AWS solution, AWS gives us an overall cost benefit of about 30–40%.

Charan Padmaraju
Chief Technology Officer, redBus

The Challenge:

📝The company previously ran its operations from a traditional data center by purchasing and renting its systems and infrastructure. In addition to the expense, several logistical problems evolved from this arrangement. The biggest problem was that the infrastructure could not effectively handle processing fluctuations, which had a negative impact on productivity. Additionally, the procurement of servers or upgrading the server configuration was an extremely time-consuming endeavor. Over time, redBus realized that a better solution was imperative a solution that offered scalability to handle the company’s processing fluctuations. redBus looked to Amazon Web Services (AWS) for a solution.

Why Amazon Web Services?

📝After testing the AWS solution on a small application for several months, the travel agency determined that it was very workable and convenient. Although redBus was quite enthusiastic about the on-demand instances and variety of instance types, several other features cemented the company’s decision to migrate completely to AWS. These features included the ability to easily manage access to servers through security groups, the easy-to-use, self-service management console, the concept of Elastic IPs, and superior support.

The company has incorporated many of the AWS products into its solution, including Amazon Elastic Compute Cloud (Amazon EC2), Elastic Load Balancing, Amazon Relational Database Service (Amazon RDS), Amazon Simple Storage Service (Amazon S3), Amazon Elastic Block Store (Amazon EBS), and Amazon CloudWatch. Charan Padmaraju, Chief Technology Officer believes that “with features like Elastic Load Balancing and multiple availability zones, AWS provides the required infrastructure to build for redundancy and auto-failover. When you incorporate these in your system/application design, you can achieve high reliability and scale.”

The Benefits:

📝Since migrating to AWS, redBus has seen measurable improvements in the bottom line. Padmaraju says, “By scaling up and down dynamically based on the load, we maintain performance as well as minimize cost. With the time savings that the IT and development staffs obtain from the AWS solution, AWS gives us an overall cost-benefit of about 30–40%.” He adds, “By hosting at [the AWS Asia Pacific (Singapore) region], redBus.in gained significantly in terms of website performance by way of reduced latency (about 4x). This is a great advantage when the customers are from India.”

Of the many excellent characteristics of AWS, perhaps the most significant to redBus is the ability to “instantly replicate the whole setup on-demand for testing by creating and destroying instances on demand for experimentation, thereby reducing the time to market.” Less time to market translates to increased profitability and success.

The travel agency anticipates expanding the AWS solution to include Amazon Simple Notification Service (Amazon SNS) and Amazon Simple Queue Service (Amazon SQS) for monitoring, alerts, and intercommunication. “Amazon SQS is an especially good solution for enabling messaging between external applications and our applications,” says Padmaraju.

Since joining forces with AWS, redBus has gained the freedom to experiment on new solutions and applications at minimal cost, increased the efficiency of its operations, and improved its profitability.

#vimaldaga #righteducation #educationredefine
#rightmentor #worldrecordholder #linuxworld
#makingindiafutureready #righeducation #arthbylw
#aws #awssqs #amazonwebservice