Microsoft.Extensions.Logging.ILogger interface support for Lambda

[dhhoang]

Describe the feature

The dotnet6 Lambda runtime should offer a log provider that supports the Microsoft.Extensions.Logging.ILogger logging interface.

Use Case

We have lots of Lambdas written in C# that makes heavy uses of .NET built-in Dependency injection (aka IServiceProvider). The typical usage is to set up the DI container in the function's constructor, and uses it to resolve the classes in the function handler method. We believe this is the "right" way, as creating a new DI container per invocation is too expensive.
The provided ILambdaLogger interface used for logging however is wrapped inside ILambdaContext which is provided per-invocation, making it impossible to set up in the with the DI pipeline (without some ugly work-around). We believe using IServiceProvider is very common among .NET customers.

On top of that ILamdaLogger misses some features compared to ILogger:

• No parameterized logging based on log level. This means that the log message is formed regardless of the configured log level. For example, even when we configure AWS_LAMBDA_HANDLER_LOG_LEVEL to Warning, our Debug and Info messages are stilled formed. This is not very efficient.
• No support for templatized/structured logging or custom log formatting (e.g. JSON). Being able to output JSON logs will allow us to have flexible queries with CloudWatch Logs insight. This has also been raised in Allow log message formatting to be controlled programmatically #1000 and Amazon.Lambda.Logging.AspNetCore should support structural json logging #1144 .

Proposed Solution

We propose that the runtime offers an implementation of ILoggerProvider (aka LambdaLogProvider, see Microsoft docs).
There is currently an existing implementation in Amazon.Lambda.Logging.AspNetCore which delegates back to LambdaLogger but we would like to see it implemented in the core runtime so that other app types can benefit from it.
We have a PoC implementation here https://github.com/dhhoang/LambdaCustomLogger. Compared to the implementation in Amazon.Lambda.Logging.AspNetCore, it has some added features:

• Allows users to extend and support custom formats by registering ILogHandler.
• Correctly writes frames to Telemetry log file descriptor (i.e. _LAMBDA_TELEMETRY_LOG_FD). The current LambdaLogger has a bug that results in multi-line logs ending up as several CloudWatch Logs entries (instead of one with several lines). For example, when logging exception with default LambdaLogger:
  
  With our CustomLogger
  
• Log data can be written directly to _LAMBDA_TELEMETRY_LOG_FD which should make it more efficient.

Other Information

We are interested in submitting PR for this feature.
Our current idea is to allow users configure a log handler in the DI like so

services.AddLogging(b => b.AddLambdaProxyLogger(opts =>
{
    opts.HandlerName = "JSON";
}));

How the actual JSON logs (or any other format) look will be completely up to the handler implementation. Anyone can implement a log handler to support custom format. The default handler can output same format as is written by lambda today. We are interested in hearing feedback about what formats should be supported by default.

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

AWS .NET SDK and/or Package version used

"Amazon.Lambda.Core" Version="2.1.0"

Targeted .NET Platform

.NET 6

Operating System and version

AmazonLinux

[ashishdhingra]

Appears to be a useful feature request.

@dhhoang Thanks for opening the feature request. This needs to be reviewed with the team. Feel free to contribute your PR for further review.

Thanks,
Ashish

[piekstra]

Just ran into a desire for this capability myself. Would appreciate seeing it implemented.

[dhhoang]

Closing in favor of #1268

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.