The What, Why, and When of Single-Table Design with DynamoDB

Yet data modeling with DynamoDB is tricky for those used to the relational databases that have dominated for the past few decades. There are a number of quirks around data modeling with DynamoDB, but the biggest one is the recommendation from AWS to use a single table for all of your records.

In this post, we’ll do a deep dive on the concepts behind single-table design. You’ll learn:

https://www.alexdebrie.com/posts/dynamodb-single-table/

Z-Order Indexing for Multifaceted Queries in Amazon DynamoDB

Using Z-order indexing, you can efficiently run range queries on any combination of fields in your schema. Although Amazon DynamoDB doesn’t natively support Z-order indexing, you can implement the functionality entirely from the client side. A single Z-order index can outperform and even replace entire collections of secondary indexes, saving you money and improving your throughput.

https://aws.amazon.com/blogs/database/z-order-indexing-for-multifaceted-queries-in-amazon-dynamodb-part-1/

In a previous AWS Database Blog post, I introduced Z-order indexing, a way in which you can sort your data to efficiently query an Amazon DynamoDB table by using range bounds on multiple attributes. In this post, we explore the process of creating a schema for your index. We look at how to decide which attributes to include in your schema, how your index’s schema impacts query efficiency, and how to work with a variety of data types.

This post builds on concepts that are described in Part 1, so I recommend taking some time to review it before diving in.

https://aws.amazon.com/blogs/database/z-order-indexing-for-multifaceted-queries-in-amazon-dynamodb-part-2/

Using artificial intelligence to detect product defects with AWS Step Functions

Factories that produce a high volume of inventory must ensure that defective products are not shipped. This is often accomplished with human workers on the assembly line or through computer vision.

You can build an application that uses a custom image classification model to detect and report back any defects in a product, then takes appropriate action. This method provides a powerful, scalable, and simple solution for quality control. It uses Amazon S3Amazon SQSAWS LambdaAWS Step Functions, and Amazon SageMaker.

To simulate a production scenario, the model is trained using an example dataset containing images of an open-source printed circuit board, with defects and without. An accompanying AWS Serverless Application Repository application deploys the Step Functions workflow for handling image classification and notifications.

https://aws.amazon.com/blogs/compute/using-artificial-intelligence-to-detect-product-defects-with-aws-step-functions/

HTTP APIs is a new flavor of API Gateway. Build low-latency & cost-effective REST APIs

Announcing the API Gateway HTTP API

We talk to customers every day that use API Gateway for critical production applications. This includes everything ranging from simple HTTP proxies to full-blown API management with request transformation, authentication, and validation. API Gateway supports REST APIs and WebSocket APIs, but customers have told us they want more features, lower latency, and lower cost.

Customers have explained their need for the core features of API Gateway at a lower price along with an easier developer experience. Based on this feedback, we are excited to announce the availability of HTTP APIs (Preview).

HTTP APIs is a new flavor of API Gateway. It focuses on delivering enhanced features, improved performance, and an easier developer experience for customers building with API Gateway. Today, we’re launching the first phase, and we will continue to enhance HTTP APIs over the next few months.

We are introducing a new pricing model for HTTP APIs that better represents customer usage patterns. Staying true to our serverless principles, you will only pay for the requests you receive.  With existing REST APIs, you pay $3.50/million requests plus data transferred out.

With HTTP APIs, we have reduced request pricing to $1.00/million requests for the first 300 million requests, and $0.90/million for all requests after that. Most customers will see an average cost saving up to 70%, when compared to API Gateway REST APIs. In addition, you will see significant performance improvements in the API Gateway service overhead.

https://go.aws/2FbVBoG

Fast Cloudfront log queries using AWS Athena and Serverless

Recently we’ve needed to run queries on our Cloudfront web logs, for example, to determine the number of Googlebot hits to certain page types. We configured Cloudfront to write logs to an S3 bucket, and we set up an AWS Athena table to query those logs.

(By the way, if you haven’t used Athena before, I think it’s nothing short of amazing — the ability to use SQL to query fields from a bunch of gzipped text files sitting in S3? That’s incredible.)

But over time our queries became really slow. For every query, Athena had to scan the entire log history, reading through all the log files in our S3 bucket. This churned through a lot of data (about 120 GB) and made the queries slow and pretty expensive — taking over 65 seconds to run and costing about $0.70 per query.

So we did some searching and came across this AWS blog post, which describes how to set up Athena partitioning to vastly speed up date-based queries. We decided to use AWS Lambda to partition our logs, along with the Serverless framework to easily deploy the Lambda code.

https://medium.com/compass-true-north/fast-cloudfront-log-queries-using-aws-athena-and-serverless-ef117393c5a6

How Uber stores financial transactions in ledgers using Amazon DynamoDB

Each day, millions of people move around the world with Uber. The associated financial transactions are stored in Uber’s Ledger Store with provable completeness backed by Amazon DynamoDB. In this session, we discuss why provable completeness is key for compliant storage of financial and other ledger-like use cases, and we explain how this can be implemented at global scale by using DynamoDB.

https://www.youtube.com/watch?v=iN6mhI5hFt4

How Verizon Media implemented push notification using Amazon DynamoDB

Verizon Media had to create a better, stronger, and faster push notification system to serve the requirements of iconic Verizon brands, fulfill push notification completion time of 27 million devices in under three minutes, and consistently show the push “toast” on all users’ lock screens. Verizon decided to use Amazon DynamoDB and other AWS services such as Amazon ElastiCache and Amazon SQS in conjunction with its own Vespa search engine to power all the use cases of its brands. It also uses Kubernetes to orchestrate microservices across many Amazon EC2 instances.

https://www.youtube.com/watch?v=FwWT6a3ikZ4

AWS Step Functions Adds Support for Workflow Execution Events

AWS Step Functions now supports workflow execution events, which make it faster and easier to build and monitor event-driven, serverless workflows. Execution event notifications can be automatically delivered when a workflow starts or completes through CloudWatch Events, reaching targets like AWS Lambda, Amazon SNS, Amazon Kinesis, or AWS Step Functions for automated response to the event.

You can enable CloudWatch Event notifications for Step Functions in minutes. Customers simply have to identify the state machine, what event changes are of interest, and the targets for the notification.

AWS Step Functions allows you to add resilient workflow automation to your applications. The steps of your workflow can exist anywhere, including in AWS Lambda functions, on Amazon EC2, or on-premises. AWS Step Functions is also integrated with Amazon ECS, AWS Fargate, Amazon DynamoDB, Amazon SNS, Amazon SQS, AWS Batch, AWS Glue, and Amazon SageMaker. You can connect and coordinate all these services in minutes—without writing code

To learn more:

https://aws.amazon.com/pt/about-aws/whats-new/2019/05/aws-step-functions-adds-support-for-workflow-execution-events/