Changelog

Changed

• Improve collector status visibility. The collector now repeats its version and configuration details while it is running, making it easier to confirm which setup is active when reviewing logs.

This release can be installed through our collector packages and Docker image.

Changed

• Improve the sampling strategy used for error and performance issue tracking.

This release can be installed through our collector packages and Docker image.

Added

• Allow appsignal.group and appsignal.format attributes in OpenTelemetry logs to override the log line's group and format.

  When present, appsignal.group overrides the resource-level service name as the log group, and appsignal.format (one of plaintext, logfmt, json, autodetect) overrides the log source's parsing format.

Fixed

• Fix crashes when truncating attribute keys and values whose length limit fell in the middle of a multi-byte UTF-8 character. Truncation now rounds the cut point down to the nearest character boundary instead of panicking.
• Continue reporting non-disk host metrics when a mount is frozen.

This release can be installed through our collector packages and Docker image.

Added

• Support Render's OpenTelemetry metrics stream by converting its host-related metrics into AppSignal host metrics.

This release can be installed through our collector packages and Docker image.

Fixed

• Cap the amount of metric states stored for cumulative-to-delta conversion, preventing high-cardinality tag combinations from causing excessive memory usage.
• Evict and block traces that accumulate large amounts of orphaned spans, preventing them from growing unboundedly in memory.

This release can be installed through our collector packages and Docker image.

Fixed

• Fix an issue where certain noisy traces would not be ignored on Next.js and Flask.
• Fix an issue where traces representing requests performed from different HTTP ports on the client's end would be grouped as different traces.

This release can be installed through our collector packages and Docker image.

Added

• Use db.stored_operation.name as one of the attributes to determine if two spans are the same. When it is present, ignore db.statement and db.query.text for the same purpose.

Changed

• Improve recognition of HTTP client spans. This will help our 'slow requests' feature recognize grouped HTTP client spans and improve our N+1 detection.

This release can be installed through our collector packages and Docker image.

Changed

• Allow the use of app-level push API keys. When an app-level push API key is used, the resource attributes for the name and environment should not be specified. These resource attributes are now optional, although the use of one requires the other.

• Make most resource attributes optional. Allow for the resource attributes pertaining the revision, the hostname, the language integration and the application path to not be present. This allows the AppSignal collector to more easily ingest data from arbitrary OpenTelemetry sources.

  Note that usage of these resource attributes is still recommended. Traces sent without these attributes may be tagged with the wrong hostname or revision, or have their error backtraces parsed incorrectly.

This release can be installed through our collector packages and Docker image.

Changed

• Use heroku.release.version resource attribute as a fallback value for revision for Heroku Fir applications.

This release can be installed through our collector packages and Docker image.

Added

• Support first-party add-on metrics from Heroku Fir generation apps.
• Support Heroku Fir host metrics.

This release can be installed through our collector packages and Docker image.

Changed

• Change the enable_host_metrics configuration option default from true to false. Host metrics are now disabled by default and must be explicitly enabled in the configuration.

This release can be installed through our collector packages and Docker image.

Fixed

• Fix an issue where the collector would stop transmitting data after a payload's authentication was rejected.

This release can be installed through our collector packages and Docker image.

Fixed

• Report errors in AppSignal for exception events in Next.js render spans.

This release can be installed through our collector packages and Docker image.

Added

• Improve histogram support. When processing OpenTelemetry histograms, allow for histograms that do not have min, max or sum values. Use the exemplars provided alongside the histogram to improve the accuracy of the histogram.
• Return CORS headers from the export endpoints to make it possible to export OpenTelemetry data to these endpoints without the need of a proxy server. The Access-Control-Allow-Origin header value can be configured in the collector with the cors_origin config option or APPSIGNAL_CORS_ORIGIN environment variable.
• Support key-value list attributes in log lines. When an attribute sent in a log line is a key-value list, it will be converted into multiple attributes, one for each key-value pair in the list, prefixed with the key of the list attribute.

Changed

• Disable the default AppSignal request and response headers filter. By default, the OpenTelemetry SDK in the applications do not send any request and response headers and require configuration of to record these headers. This required double configuration in applications for AppSignal and OpenTelemetry. The appsignal.config.request_headers and appsignal.config.response_headers configuration options can still be used for additional header filtering if needed. Consult your application's language OpenTelemetry SDK for the configuration option to record request and response headers.
• Log a warning when making a GET request to a POST endpoint. The error page when making a GET request will also explain to make a POST request instead.
• Read the app language from the OpenTelemetry telemetry.sdk.language resource attribute if set. This attribute is set by the OpenTelemetry SDK for some languages, like Go, Java and PHP. If your application's OpenTelemetry SDK sets the telemetry.sdk.language resource attribute, you will not need to set the appsignal.config.language_integration attribute. If the appsignal.config.language_integration attribute is set, it will take precedence over the telemetry.sdk.language resource attribute.
• Report log lines with unspecified severity. When a log line does not have a severity, report it as a log line with info level severity.

Fixed

• Fix child spans not always being reported. This was due to the collector not properly traversing all the levels the trace span hierarchy.

This release can be installed through our collector packages and Docker image.

Added

• Add an ignore_logs config option, which can be set using the appsignal.config.ignore_logs resource attribute. When this resource attribute is set, logs whose message matches any of the values in the ignore_logs list of values will be discarded.

  The values on the ignore_logs configuration option support the ^, $ and .* meta-characters, with their expected meanings in regular expressions. When neither ^ or $ are specified at the beginning or ending of the value respectively, ignore_logs will match a log message if its value is present at any point in the message.

  For example, to ignore logs starting with "success" or containing the word "OK", when setting the resource attributes in a Go application:

  Go

  res := resource.NewWithAttributes(
  	attribute.StringSlice("appsignal.config.ignore_logs", []string{"^success", "OK"}),
  	// And other resource attributes
  )

  The list of patterns can also be provided as a comma-separated string -- this is useful when using the OTEL_RESOURCE_ATTRIBUTES environment variable, such as in a PHP application:

  Shell

  function encode() {
    echo -n "$@" | sed 's/,/%2C/g'
  }
   
  export OTEL_RESOURCE_ATTRIBUTES="\
    appsignal.config.ignore_errors=$(encode "^success,OK"),\
    ..."

Changed

• Improve AppSignal compatibility with traces across multiple services.

This release can be installed through our collector packages and Docker image.

Added

• When reporting a boolean resource attribute configuration option, such as send_function_parameters, send_request_payload, send_request_query_parameters and send_request_session_data, allow it to be provided as a string containing the case-insensitive values "true" or "false".

  This makes it possible to set these configuration options using the OTEL_RESOURCE_ATTRIBUTES environment variable.

Changed

• Allow overriding automatically set namespaces. When a namespace is automatically by AppSignal, set it only if a different namespace has not been set in the trace using the appsignal.namespace attribute.
• Improve communication from the collector's exporter server to the client by returning the expected OpenTelemetry export response messages from the collector's HTTP server. This includes updating the tracing, metrics and logs endpoints by returning their respective response messages.

Fixed

• Truncate the keys of attributes and the overall size of the attribute list for spans, events, scopes, resources and links.

This release can be installed through our collector packages and Docker image.

Added

• Add ignore_errors option. Configure this option using the appsignal.config.ignore_errors resource attribute. The errors are matched by class/type.

  The value can be set both as an array or a comma-separated string. The following examples are equivalent:

  Shell

  appsignal.config.ignore_errors=["RuntimeError", "NoMethodError"]
  appsignal.config.ignore_errors="RuntimeError,NoMethodError"
  

• Allow log records with non-string values as bodies.

  When a log record does not have a string as its body, but it has attributes, serialise those attributes as JSON and use that as the log message instead of the body.

  When a log record has a key-value list as its body, use those key-value pairs as attributes.

• Extract attributes in Laravel logs, encoded into the context attribute, into separate attributes that can be explored individually in AppSignal.

Fixed

• Support Express instrumentation version 0.200.0, by ensuring that exception events from Express request handler spans are always reported.

This release can be installed through our collector packages and Docker image.

Added

• Add support for OpenTelemetry logs. Logs sent to the collector's /v1/logs endpoint in OTLP/HTTP format will be delivered to AppSignal's logging feature.
• Enrich Sidekiq events by setting the action name. Now Sidekiq jobs are reported as <WorkerClass>#perform, e.g. MyWorker#perform.

Changed

• BREAKING CHANGE: Use a better optimized protocol to send data to our API. This change is backwards incompatible and we recommend you upgrade to the latest version of the collector. Older versions of the collector will not be able to send data to our API. Please refer to the documentation for more information on how to upgrade your collector.

This release can be installed through our collector packages and Docker image.

Added

• Update our support OpenTelemetry Semantic Conventions for code attributes used in unique span detection.

Changed

• The shutdown behavior on internal errors is improved, and now logs what crashed and why.

• Limit appsignal.tag.<key> tag values to 256 characters.

• Rename an internal span event attribute for metadata distributions. Older collectors will no longer report metadata distributions for error incidents.

  This will create new incidents for existing errors. This is intentional.

Fixed

• Fix a crash on truncating Unicode values.

This release can be installed through our collector packages and Docker image.

Added

• Configure AppSignal via HTTP request headers on export requests. If it's not possible to configure resource attributes in OpenTelemetry export data, configure AppSignal via request headers set on the export requests made to the collector's HTTP endpoints.

  Supported headers are:

  AppSignal-Config-Name: app name (required) AppSignal-Config-Environment: app environment (required) AppSignal-Config-PushApiKey: push API key (required) AppSignal-Config-Revision: app revision (required) AppSignal-Config-LanguageIntegration: language integration like ruby, elixir, php, etc. (required) OpenTelemetry-ServiceName: service name of the app OpenTelemetry-HostName: name of the app's host (required)

  For more information, see our configuration documentation.

Changed

• Close HTTP connections from OpenTelemetry exports when we have successfully received the request. This fixes an issue with the server not shutting down when the connection is kept open.

This release can be installed through our collector packages and Docker image.