You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit e91221f
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: README.md
+69-1Lines changed: 69 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,7 @@ Why even use this feature? If you have many code bases that are writing data to
9
9
Things to consider:
10
10
- If you fire off lambdas for row updates that themselves update the DB then you can quickly exaust connection and lambda resources. If you update 1000 rows then you could have hundreds of lambdas fire and connect to the DB bringing everything to a grinding halt. Therefore you should set [concurrency limits](https://aws.amazon.com/blogs/compute/managing-aws-lambda-function-concurrency/) on these lambda functions in most cases.
11
11
- When you exceed the limit, you get a throttle on the lambda, and these will delay processing. In some cases this is fine. In the Geo coding use case we don't need the Geo coding to complete immediately, so it's OK. In testing 1K+ row updates for Geo coding some row updates took over 1 minute to complete because of the retry back off.
12
-
- If you need smoother execution of the events, then consider using a simple NodeJS Lambda (that won't be throttled) that takes the event payload and puts it on a SQS queue that another Lambda will process from the queue. See a NodeJS example of this in the [ForwardToSQS.js](ForwardToSQS.js) file.
12
+
- If you need smoother execution of the events, then consider using a simple NodeJS Lambda (which remains subject to Lambda concurrency limits) that takes the event payload and puts it on a SQS queue that another Lambda will process from the queue. See a NodeJS example of this in the [ForwardToSQS.js](ForwardToSQS.js) file.
13
13
14
14
Three use cases are covered in this demo:
15
15
- A Lambda([PostgresAddressTrigger.java](src/main/java/demo/PostgresAddressTrigger.java)) that will update the same row that the trigger is firing on. In this case, we have an address table, and when addresses are inserted or updated, they will be geo-coded by the [AWS Location API](https://docs.aws.amazon.com/location/latest/APIReference/API_SearchPlaceIndexForText.html). Special care is needed in this case to prevent recursive triggering of the function.
Run the offline routing and action-response tests with `mvn clean test` before `sam build`.
52
52
53
+
## Runtime behavior and failure handling
54
+
55
+
| Component | Success | Failure and retry behavior |
56
+
| --- | --- | --- |
57
+
| UI (`WebHandler`) | Delegates directly to the view/action; actions redirect to `/`| Acquisition and action/query errors return HTTP 500. Each request closes its JDBC connection. |
58
+
| Direct database triggers | Validate the top-level event, perform the side effect, then acknowledge | Validation and persistence errors fail the invocation so the configured Lambda asynchronous retry policy applies. |
59
+
| SQS audit consumer | Processes every record; successful records are acknowledged | Returns failed message IDs using `ReportBatchItemFailures`. After five receives, unsuccessful messages move to `AuditDeadLetterQueue`, which retains them for 14 days. |
60
+
| Address geocoding | Looks up inserts and changes to street, city, district, or postal code | A zero-row update retries at most ten times, with 100–900 ms delays. Exhaustion, interruption, service errors, and SQL errors fail the invocation. |
61
+
| Database setup | Loads all packaged SQL assets, executes them in order, then inserts the sample address | Stops at the first failure and returns CloudFormation `FAILED`; previously applied scripts may remain. UPDATE and DELETE do not rerun initialization. |
62
+
63
+
The SQS source queue's visibility timeout is 180 seconds, six times the consumer's
64
+
30-second timeout, following [AWS's SQS configuration guidance](https://docs.aws.amazon.com/lambda/latest/dg/services-sqs-configure.html).
65
+
The consumer and its `FunctionResponseTypes: [ReportBatchItemFailures]` configuration must
66
+
be deployed together for [partial batch retries](https://docs.aws.amazon.com/lambda/latest/dg/services-sqs-errorhandling.html).
67
+
The batch size remains one, but the implementation also handles larger standard-queue batches.
68
+
Inspect the dead-letter queue and correct the cause before intentionally replaying its messages.
69
+
70
+
`PostgresDataSource.getDSL()` returns a `CloseableDSLContext` that owns its connection.
71
+
Always use `try (var dsl = PostgresDataSource.getDSL()) { ... }`; passing an existing JDBC
72
+
connection to `DSL.using(connection)` does not transfer connection ownership. No live JDBC
73
+
connection is retained at initialization or across warm invocations. This is connection
74
+
cleanup, not pooling: a connection pool or RDS Proxy would need separate load testing.
75
+
76
+
The geocoding SQL trigger still requires `requires_geo_coding = true`. Callers editing an
77
+
already-geocoded address must set this flag if they want another lookup. A lookup with no
78
+
match leaves the row unchanged. Each Location call has a five-second API timeout and each
79
+
geocoding UPDATE a two-second query timeout; retry limits do not guarantee a fixed end-to-end
80
+
latency because connection establishment and service execution also take time.
81
+
82
+
### Delivery and deployment limits
83
+
84
+
- Database-trigger invocations are asynchronous external side effects. They may run before
85
+
the source transaction commits, and rolling back that transaction does not retract them.
86
+
This demo does not implement a transactional outbox.
87
+
- Delivery is at-least-once. Audit tables do not deduplicate; a successful write followed by
88
+
a lost acknowledgement can be repeated. Standard SQS and asynchronous Lambda delivery also
89
+
do not guarantee ordering, so concurrent address edits can race with geocoding results.
90
+
- Direct-trigger retries remain finite (`MaximumRetryAttempts: 2`, event age one hour),
91
+
and those functions have no failure destination configured. The new dead-letter queue
92
+
covers the SQS consumer only.
93
+
- Setup is not one atomic migration, and repeated CREATE processing can insert another sample
94
+
row. Changing packaged SQL does not apply it to an existing database: the custom resource's
95
+
UPDATE callback deliberately performs no SQL migrations. The trigger argument guard in
96
+
`LambdaTriggerFuction.sql` therefore requires an explicit migration on an existing database.
97
+
- This is an unauthenticated demo with GET routes that mutate data and a publicly reachable
98
+
database security-group rule. Restrict access and add authentication/CSRF protection before
99
+
using it with sensitive data. The UI currently renders all public tables without pagination.
100
+
101
+
### Local validation
102
+
103
+
Run `mvn clean test` for the offline tests, then `sam validate --lint` and `sam build --no-cached`.
104
+
The JDBC fixture exercises statement execution and connection closure without a live database;
105
+
service fakes exercise validation, retries, and failure responses. These checks do not prove a
106
+
live Aurora migration, IAM permissions, AWS retry delivery, or geocoding latency.
107
+
108
+
For strict documentation validation, include production, private, and test declarations:
Before deploying merged PRs, switch to current `main` and rebuild; `sam deploy` can otherwise
118
+
use stale `.aws-sam/build` artifacts. Verify `EngineVersion` in both `postgres.yaml` and
119
+
`.aws-sam/build/RDSStack/template.yaml`, plus the handlers and runtimes in the built template.
120
+
53
121
## Contents
54
122
This project contains source code and supporting files for a serverless application that you can deploy with the SAM CLI. It includes the following files and folders.
0 commit comments