Partially incompatible with structlog configuration

[james-certn]

Using structlog's "Rendering Using structlog-based Formatters Within logging" steps, new relic is unable to extract "event" -> "message"

Description
Log entries in New Relic are missing the "message" value

Expected Behavior
Log entries in New Relic have the "message" value populated correctly

Troubleshooting
Diagnosing the issue, I noticed:

• newrelic appends a new_relic_event_consumer processor to the structlog processors list, and ensures it is at the tail end
  • it does this in wrap__process_event(...)
• new_relic_event_consumer inspects the type of the event parameter:
  • if str-like it directly copies it to message
  • if dict-like it copies the "event" value from the dict into message
• this works for the 'simplified' structlog setup, but not for the 'ambitious' (structlog's wording, not mine!) setup (linked at the top)
  • in this setup, the last processor when configurating must be structlog.stdlib.ProcessorFormatter.wrap_for_formatter
  • wrap_for_formatter expects to always be the last processor prior to the ProcessorFormatter handling its output
    • as such, wrap_for_formatter knows it can safely change the shape of the data it returns to be compatible with ProcessorFormatter
    • this format is a tuple (specifically, the (*args, **kwargs}) that ProcessorFormatter expects to receive
  • however, because newrelic ensures its new_relic_event_consumer is always at the tail end of the processors list, it ends up:
    • receiving a tuple of (*args, **kwargs})
    • skips over it, because it doesn't know what to do with that data type
    • returns the tuple (as it should), which successfully gets passed to ProcessorFormatter

Steps to Reproduce

1. configure structlog in "Rendering Using structlog-based Formatters Within logging" fashion
   a. for us, we use django-structlog as well, but I don't think this is an influencing factor
2. configure new relic to receive logs
3. output some logs using structlog logging
4. structlog logging output to the console is correct, however new relic does not receive "message" in records is unable to extract "event" in new_relic_event_consumer structlog processor

Your Environment

• newrelic==10.3.1
• django-structlog==8.1.0
• structlog==24.4.0

Podman, linux container environment.

Additional context
Relates to and describes the issue in NR support ticket #00236985.

[workato-integration]

https://new-relic.atlassian.net/browse/NR-357206

[TimPansino]

Wow that's really specific, great writeup. I'll put this ticket into our queue and we'll see if we can do something about that.

[james-certn]

If I add in the following to new_relic_event_consumer in the initial relevant if clause, it works:

        elif isinstance(event, tuple):
            _event = event[0][0]
            message = original_message = _event.get("event", "")
            event_attrs = {k: v for k, v in _event.items() if k != "event"}