feat: add decorator factory to create your own middleware by heitorlessa · Pull Request #17 · aws-powertools/powertools-lambda-python

heitorlessa · 2020-04-10T15:54:32Z

Issue #, if available: #20

Description of changes:

This is required for Serverless Airline project to move from the modified version of powertools, and create own middlewares to improve reusability as much as possible.

Instead of creating reusable methods, we're at the stage that makes more sense to give flexibility to create own middlewares as #9, so customers can create one middleware with many custom processing before, after and exception handling instead.

This gives flexibility to control before, after, and exception handling logic in the way you'd like. It also lay the foundations to allow us to create a base Trace provider so you can use any Tracer that follow our protocol.

One step closer from GA 🎉 - We'll tackle log level next!

User Experience

Create a middleware no params

from aws_lambda_powertools.middleware_factory import lambda_handler_decorator

@lambda_handler_decorator
def log_response(handler, event, context):
    any_code_to_execute_before_lambda_handler()                
    response = handler(event, context)
    any_code_to_execute_after_lambda_handler()
    print(f"Lambda handler response: {response}")
    return response

@log_response
def lambda_handler(event, context):
    return True

Create a middleware with params

from aws_lambda_powertools.middleware_factory import lambda_handler_decorator

@lambda_handler_decorator
def obfuscate_sensitive_data(handler, event, context, fields=None):
    # Obfuscate email before calling Lambda handler
    if fields:
        for field in fields:
            field = event.get(field, "")
            event[field] = obfuscate_pii(field)

    return handler(event, context)
    

@obfuscate_sensitive_data(fields=["email"])
def lambda_handler(event, context):
    return True

Trace execution of custom middleware - Creates sub-segments in X-Ray

```python
from aws_lambda_powertools.utils import lambda_handler_decorator

@lambda_handler_decorator(trace_execution=True)
def my_custom_middleware(handler, event, context):
    ...

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

heitorlessa · 2020-04-12T18:07:57Z

Local perf tests to rule out major performance impact on multiple callables - Decorator only prints before handler and after.

While Lambda will have less CPU it'll continue to be less than a microsecond, so it worth using this factory to refactor codebase to make it more contributor friendly (dev time is more expensive)

No decorator

50000000 loops, best of 5: 6.88 nsec per loop

Simpler decorator

50000000 loops, best of 5: 7.18 nsec per loop

Decorator factory

50000000 loops, best of 5: 7.25 nsec per loop

Simpler Stacked decorator

50000000 loops, best of 5: 7.05 nsec per loop

Factory Stacked decorator

50000000 loops, best of 5: 7.05 nsec per loop

python -m timeit -s 'import handler; handler.lambda_handler({}, {})'

def call_handler_simpler(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print("SIMPLER - BEGIN")
        response = func(*args, **kwargs)
        print("SIMPLER - END")
        return response
    return wrapper

@lambda_handler_decorator
def call_handler_factory(handler, event, context):
    print("[CALL HANDLER] BEGIN")
    response = handler(event, context)
    print("[CALL HANDLER] END")
    return response

def lambda_handler(event, context):
    a = {
        "statusCode": 200,
        "body": "Hello world"
    }
    print("-------------LAMBDA FN CALLED!-------------")
    return a

* develop: chore: clean up CI workflows fix: CI attempt 4 fix: CI attempt 3 fix: CI attempt 3 fix: CI attempt 2 feat: add docs to CI chore: fix github badge typo chore: pypi monthly download badge

python/README.md

python/aws_lambda_powertools/tracing/tracer.py

python/aws_lambda_powertools/middleware_factory/factory.py

* docs: add pypi badge * fix: add missing single_metric example; test var name * chore: pypi monthly download badge * chore: fix github badge typo * feat: add docs to CI * fix: CI attempt 2 * fix: CI attempt 3 * fix: CI attempt 3 * fix: CI attempt 4 * chore: clean up CI workflows * Decorator factory Feat: Create your own middleware (#17) * feat(utils): add decorator factory * improv: use partial to reduce complexity * improv: add error handling * chore: type hint * docs: include pypi downloads badge * feat: opt in to trace each middleware that runs * improv: add initial util tests * improv: test explicit and implicit trace_execution * improv: test decorator with params * chore: linting * docs: include utilities * improv: correct tests, dec_factory only for func * improv: make util name more explicit * improv: doc trace_execution, fix casting * docs: add limitations, improve syntax * docs: use new docs syntax * fix: remove middleware decorator from libs * feat: build docs in CI * chore: linting * fix: CI python-version type * chore: remove docs CI * chore: kick CI * chore: include build badge master branch * chore: refactor naming * fix: rearrange tracing tests * improv(tracer): toggle default auto patching * feat(tracer): retrieve registered class instance * fix(Makefile): make cov target more explicit * improv(Register): support multiple classes reg. * improv(Register): inject class methods correctly * docs: add how to reutilize Tracer * improv(tracer): test auto patch method * improv: address nicolas feedback * improv: update example to reflect middleware feat * fix: metric dimension in root blob * chore: version bump Co-authored-by: heitorlessa <lessa@amazon.co.uk> Co-authored-by: heitorlessa <lessa@amazon.co.uk>

* develop: (21 commits) bugfix: #32 Runtime Error for nested sync fns chore: renamed history to changelog dependabot bugfix: resolves #31 aiohttp lazy import chore: grammar issues improv: add project tenets Improv tracer - async support, patch, test coverage and X-Ray escape hatch (#29) Bugfix: "per second" metric units (#27) fix: #24 correct example test and docs chore: bump example to use 0.8.0 features Adopt logging best practices (#23) Decorator factory Feat: Create your own middleware (#17) chore: clean up CI workflows fix: CI attempt 4 fix: CI attempt 3 fix: CI attempt 3 fix: CI attempt 2 feat: add docs to CI chore: fix github badge typo chore: pypi monthly download badge fix: add missing single_metric example; test var name ...

* [Sync Master] 0.7.0 release (#22) * docs: add pypi badge * fix: add missing single_metric example; test var name * chore: pypi monthly download badge * chore: fix github badge typo * feat: add docs to CI * fix: CI attempt 2 * fix: CI attempt 3 * fix: CI attempt 3 * fix: CI attempt 4 * chore: clean up CI workflows * Decorator factory Feat: Create your own middleware (#17) * feat(utils): add decorator factory * improv: use partial to reduce complexity * improv: add error handling * chore: type hint * docs: include pypi downloads badge * feat: opt in to trace each middleware that runs * improv: add initial util tests * improv: test explicit and implicit trace_execution * improv: test decorator with params * chore: linting * docs: include utilities * improv: correct tests, dec_factory only for func * improv: make util name more explicit * improv: doc trace_execution, fix casting * docs: add limitations, improve syntax * docs: use new docs syntax * fix: remove middleware decorator from libs * feat: build docs in CI * chore: linting * fix: CI python-version type * chore: remove docs CI * chore: kick CI * chore: include build badge master branch * chore: refactor naming * fix: rearrange tracing tests * improv(tracer): toggle default auto patching * feat(tracer): retrieve registered class instance * fix(Makefile): make cov target more explicit * improv(Register): support multiple classes reg. * improv(Register): inject class methods correctly * docs: add how to reutilize Tracer * improv(tracer): test auto patch method * improv: address nicolas feedback * improv: update example to reflect middleware feat * fix: metric dimension in root blob * chore: version bump Co-authored-by: heitorlessa <lessa@amazon.co.uk> Co-authored-by: heitorlessa <lessa@amazon.co.uk> * feat: add algolia search for docs and api ref (#39) (#40) Signed-off-by: heitorlessa <lessa@amazon.co.uk> * fix: revert makefile build-docs-api Signed-off-by: heitorlessa <lessa@amazon.co.uk> * fix: metric_set reuse #43 Signed-off-by: heitorlessa <lessa@amazon.co.uk> * fix: clear metrics Signed-off-by: heitorlessa <lessa@amazon.co.uk> * fix: update serialize_metrics helper function to use MetricManager instead of Metrics * fix: clear metrics after lambda invocation #43 Signed-off-by: heitorlessa <lessa@amazon.co.uk> * improv: document metrics tests, remove redundants Signed-off-by: heitorlessa <lessa@amazon.co.uk> #43 * chore: linting Signed-off-by: heitorlessa <lessa@amazon.co.uk> * docs: add section to flush metrics manually Signed-off-by: heitorlessa <lessa@amazon.co.uk> * docs: include EMF Json object Signed-off-by: heitorlessa <lessa@amazon.co.uk> * chore: bump version 0.9.4 Signed-off-by: heitorlessa <lessa@amazon.co.uk> Co-authored-by: Tom McCarthy <thommcc@amazon.com>

* feat(utils): add decorator factory * improv: use partial to reduce complexity * improv: add error handling * chore: type hint * docs: include pypi downloads badge * feat: opt in to trace each middleware that runs * improv: add initial util tests * improv: test explicit and implicit trace_execution * improv: test decorator with params * chore: linting * docs: include utilities * improv: correct tests, dec_factory only for func * improv: make util name more explicit * improv: doc trace_execution, fix casting * docs: add limitations, improve syntax * docs: use new docs syntax * fix: remove middleware decorator from libs * feat: build docs in CI * chore: linting * fix: CI python-version type * chore: remove docs CI * chore: kick CI * chore: include build badge master branch * chore: refactor naming * fix: rearrange tracing tests * improv(tracer): toggle default auto patching * feat(tracer): retrieve registered class instance * fix(Makefile): make cov target more explicit * improv(Register): support multiple classes reg. * improv(Register): inject class methods correctly * docs: add how to reutilize Tracer * improv(tracer): test auto patch method * improv: address nicolas feedback * improv: update example to reflect middleware feat * fix: metric dimension in root blob * chore: version bump Co-authored-by: heitorlessa <lessa@amazon.co.uk>

* [Sync Master] 0.7.0 release (#22) * docs: add pypi badge * fix: add missing single_metric example; test var name * chore: pypi monthly download badge * chore: fix github badge typo * feat: add docs to CI * fix: CI attempt 2 * fix: CI attempt 3 * fix: CI attempt 3 * fix: CI attempt 4 * chore: clean up CI workflows * Decorator factory Feat: Create your own middleware (#17) * feat(utils): add decorator factory * improv: use partial to reduce complexity * improv: add error handling * chore: type hint * docs: include pypi downloads badge * feat: opt in to trace each middleware that runs * improv: add initial util tests * improv: test explicit and implicit trace_execution * improv: test decorator with params * chore: linting * docs: include utilities * improv: correct tests, dec_factory only for func * improv: make util name more explicit * improv: doc trace_execution, fix casting * docs: add limitations, improve syntax * docs: use new docs syntax * fix: remove middleware decorator from libs * feat: build docs in CI * chore: linting * fix: CI python-version type * chore: remove docs CI * chore: kick CI * chore: include build badge master branch * chore: refactor naming * fix: rearrange tracing tests * improv(tracer): toggle default auto patching * feat(tracer): retrieve registered class instance * fix(Makefile): make cov target more explicit * improv(Register): support multiple classes reg. * improv(Register): inject class methods correctly * docs: add how to reutilize Tracer * improv(tracer): test auto patch method * improv: address nicolas feedback * improv: update example to reflect middleware feat * fix: metric dimension in root blob * chore: version bump Co-authored-by: heitorlessa <lessa@amazon.co.uk> Co-authored-by: heitorlessa <lessa@amazon.co.uk> * feat: add algolia search for docs and api ref (#39) (#40) Signed-off-by: heitorlessa <lessa@amazon.co.uk> * fix: revert makefile build-docs-api Signed-off-by: heitorlessa <lessa@amazon.co.uk> * fix: metric_set reuse #43 Signed-off-by: heitorlessa <lessa@amazon.co.uk> * fix: clear metrics Signed-off-by: heitorlessa <lessa@amazon.co.uk> * fix: update serialize_metrics helper function to use MetricManager instead of Metrics * fix: clear metrics after lambda invocation #43 Signed-off-by: heitorlessa <lessa@amazon.co.uk> * improv: document metrics tests, remove redundants Signed-off-by: heitorlessa <lessa@amazon.co.uk> #43 * chore: linting Signed-off-by: heitorlessa <lessa@amazon.co.uk> * docs: add section to flush metrics manually Signed-off-by: heitorlessa <lessa@amazon.co.uk> * docs: include EMF Json object Signed-off-by: heitorlessa <lessa@amazon.co.uk> * chore: bump version 0.9.4 Signed-off-by: heitorlessa <lessa@amazon.co.uk> Co-authored-by: Tom McCarthy <thommcc@amazon.com>

feat(utils): add decorator factory

3555b43

heitorlessa added the enhancement label Apr 10, 2020

heitorlessa self-assigned this Apr 10, 2020

heitorlessa added 26 commits April 12, 2020 19:35

improv: use partial to reduce complexity

11ff229

improv: add error handling

774ee3d

chore: type hint

5364ca9

docs: include pypi downloads badge

55c23d4

feat: opt in to trace each middleware that runs

31eb0e2

improv: add initial util tests

5a79b12

improv: test explicit and implicit trace_execution

ee52a15

improv: test decorator with params

f57d000

chore: linting

19d02dc

docs: include utilities

021a324

improv: correct tests, dec_factory only for func

b3cee6c

improv: make util name more explicit

3390c52

improv: doc trace_execution, fix casting

4baf118

docs: add limitations, improve syntax

e7c2bfe

docs: use new docs syntax

1e8af14

fix: remove middleware decorator from libs

1a62571

feat: build docs in CI

9d08ea0

chore: linting

333e5c8

fix: CI python-version type

abbb12d

chore: remove docs CI

600c9b1

chore: kick CI

4f1ed1f

Merge branch 'develop' into feat/dec_factory

ef5d901

* develop: chore: clean up CI workflows fix: CI attempt 4 fix: CI attempt 3 fix: CI attempt 3 fix: CI attempt 2 feat: add docs to CI chore: fix github badge typo chore: pypi monthly download badge

chore: include build badge master branch

4d1548f

chore: refactor naming

c0e3227

fix: rearrange tracing tests

69b3529

improv(tracer): toggle default auto patching

c3419c6

heitorlessa added 4 commits April 17, 2020 20:44

feat(tracer): retrieve registered class instance

365d561

fix(Makefile): make cov target more explicit

c3aad5f

improv(Register): support multiple classes reg.

98aeb77

improv(Register): inject class methods correctly

37d6a19

heitorlessa changed the title ~~[WIP] Decorator factory Feat: Create your own middleware~~ Decorator factory Feat: Create your own middleware Apr 18, 2020

heitorlessa added 2 commits April 18, 2020 13:48

docs: add how to reutilize Tracer

250e47f

improv(tracer): test auto patch method

bfe3404

nmoutschen reviewed Apr 20, 2020

View reviewed changes

heitorlessa added 3 commits April 20, 2020 14:27

improv: address nicolas feedback

1f0a6c9

improv: update example to reflect middleware feat

55ce3a1

fix: metric dimension in root blob

fddc26b

heitorlessa force-pushed the feat/dec_factory branch from b77dab7 to fddc26b Compare April 20, 2020 15:27

chore: version bump

073ee4f

heitorlessa mentioned this pull request Apr 20, 2020

Feature: Automate docs build #19

Closed

heitorlessa merged commit 6ce07e9 into develop Apr 20, 2020

heitorlessa deleted the feat/dec_factory branch April 20, 2020 15:56

heitorlessa mentioned this pull request Apr 20, 2020

[Sync Master] 0.7.0 release #22

Merged

heitorlessa changed the title ~~Decorator factory Feat: Create your own middleware~~ feat: add decorator factory to create your own middleware Jun 3, 2020

heitorlessa added feature New feature or functionality and removed enhancement labels Jun 3, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

feat: add decorator factory to create your own middleware#17

feat: add decorator factory to create your own middleware#17
heitorlessa merged 37 commits intodevelopfrom
feat/dec_factory

heitorlessa commented Apr 10, 2020 •

edited

Loading

Uh oh!

heitorlessa commented Apr 12, 2020 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Conversation

heitorlessa commented Apr 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

User Experience

Uh oh!

heitorlessa commented Apr 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

heitorlessa commented Apr 10, 2020 •

edited

Loading

heitorlessa commented Apr 12, 2020 •

edited

Loading