Logging
The different components can be configured to provide logging information. These log messages are intended for operational processes only. They provide informative output of the inner workings of the application servers. As such, depending on the loglevel, the information may contain personal data and must be stored, processed and purged in accordance with data privacy legislation.
skp-server, skp-storage
The Java-based application servers use the logback framework to facilitate logging. Three appenders (outputstreams) can be used:
- local file
- syslog
- logstash
The default configuration is overruled by changes to the config files:
/etc/skp/server.logback.xml/etc/skp/storage.logback.xml
These files are monitored, and changes are applied within 10 seconds. A reload or restart of the application server process is not necessary.
Be aware that the file may be under configuration management control.
Log levels
The following log levels exist:
ALLTRACEDEBUGINFOWARNERROROFF
The minimum log level for a message to be passed on to the outputstreams can be configured.
Configuration options:
| Option | Default | Comment |
|---|---|---|
| logging.level | INFO |
Filters the messages before they are passed to the output streams. Allowed values: ALL, TRACE, DEBUG, INFO, WARN, ERROR, OFF |
Local file
This outputstream writes text messages to a file in the local file system. Directory, loglevel and message pattern can be configured.
The application server will not rotate this file by itself, instead it will detect and respond to external rotation of the logfile (for example by logrotate) within 5 seconds. If the logfile has been rotated externally, the new file will be reopened / recreated.
The message pattern can access fields from the Mapped Diagnostic Context.
Configuration options:
| Option | Default | Comment |
|---|---|---|
| file.level | ALL |
Filters the messages that pass through the outputstream. Allowed values: ALL, TRACE, DEBUG, INFO, WARN, ERROR, OFF |
| file.directory | /var/log/skp |
Base directory, in which the application-server specific logfiles are being written. |
| file.pattern | %d{yyyy-MM-dd HH:mm:ss.SSS} %mdc{req.xForwardedFor} %mdc{req.host:-localhost} [%thread] %-5level %logger{36} - %msg%n} |
Defines the structure and content of the output message. See the documentation for details. |
Syslog
The syslog output sends text-messages to a syslog daemon running on the local host. This allows the syslog daemon to forward log messages and/or write them to file.
Messages are encoded according to the syslog protocol. By default the message-pattern uses the | character as a delimiter between fields, enabling a syslog daemon to separate the messages into individual files. The message pattern can access fields from the Mapped Diagnostic Context.
The syslog messages are always sent via UDP to localhost:514.
Configuration options:
| Option | Default | Comment |
|---|---|---|
| syslog.level | OFF |
Filters the messages that pass through the outputstream. Allowed values: ALL, TRACE, DEBUG, INFO, WARN, ERROR, OFF |
| syslog.facility | LOCAL1 |
Syslog facility, to be used for message routing. |
| syslog.tag | none | Optional syslog tag, can be used for message routing |
| syslog.pattern | %d{yyyy-MM-dd HH:mm:ss.SSS}|%thread|%level|%logger{36}|%msg} |
Defines the structure and content of the output message. See the documentation for details. |
Logstash
The logstash outputstream sends messages in JSON-encoding to a Logstash instance. The benefit of this stream is easier parsing through the self-describing fields of the JSON log message. The complete Mapped Diagnostic Context is provided in the message.
Messages are sent over the network via UDP in syslog protocol. To accept the messages, configure a logstash input as follows:
syslog {
host => '0.0.0.0'
port => '10515'
codec => 'json'
use_labels => true
type => 'skp'
}
Configuration options:
| Option | Default | Comment |
|---|---|---|
| logstash.level | OFF |
Filters the messages that pass through the outputstream. Allowed values: ALL, TRACE, DEBUG, INFO, WARN, ERROR, OFF |
| logstash.hostname | localhost |
Hostname or IP of the logstash server. |
| logstash.port | 514 | Port of the correct input on the logstash server. |
Mapped Diagnostic Context
The application servers intend to provide extended context to each log message, such as the hostname of the application server or the IP-address of the remote client. This enables downstream processing of multiple messages, creating a more complete picture of what is happening.
See here for more details.
Here is a list of keywords that are used for MDC. Be aware that not every keyword will always be present in each message.
| Keyword | Purpose |
|---|---|
| appserver | Hostname of the server processing the request. |
| program | Name of the program processing the request. |
| req.url | URL of the request, without query parameters. |
| req.method | HTTP request method. |
| req.xForwardedFor | Content of HTTP header X-Forwarded-For. |
| req.userAgent | Content of HTTP header User-Agent. |
| recipientId | Maps to the recipient-ID of a transfer. |
| objectId | Maps to the object-ID of a transfer-object. |
| storagehost | FQDN of the requested external hostname. |
| principal | Email address of the authenticated user who initiated the request. |
| messageId | SMTP message ID of the generated email address. |
Access Logging
The application servers can log access events to a local file. This functionality is somewhat limited in its configuration options. If enabled, it logs entries in Apache mod_log format to the configured local file, which is rotated daily by the appserver itself.
Access logging is configured in the application server specific config files:
/etc/skp/server.properties/etc/skp/storage.properties
These settings are not reloadable and require a server restart to become effective.
Configuration options:
| Option | Default | Comment |
|---|---|---|
| http.accessLogEnabled | false |
If true, access logging is enabled. |
| http.accessLogFile | /var/log/skp/storage.access.log, /var/log/skp/server.access.log |
Location and filename of the logfile. |
| http.accessLogFormat | %a %t \"%r\" %s (%D) |
Message pattern. |
Transaction Logging
The Transaction Log feature can be configured to
send its events, in JSON encoding via Syslog to a configured external server. While this behaviour is controlled in config file /etc/skp/server.logback.xml, it is not considered logging in its diagnostic meaning. It simply uses the same framework.
Transaction log events intended for the CUSTOMER-logger do not need to pass through the logging.level-filter.
SkpServer
The PHP-based application server uses a custom logger. It provides the following outputstreams:
- syslog
- local file
- logstash via syslog
The streams are enabled through the configuration chain:
- Default value
- Appserver defaults:
/etc/teambeam/appserver.env.php - Storagehost-specific config file:
/var/teambeam/{hostname}/config/local.env.php
This allows specific storagehosts to use different logging configuration from others.
Due to the nature of PHP, settings are effective immediately, no restart is necessary.
Log levels
The following log levels are available:
TRACE= 16DEBUG= 8INFO= 4WARN= 2ERROR= 1NOTHING= 0
They are mapped to global constants: SKP_LOG_TRACE, SKP_LOG_DEBUG, etc.
Configration options:
| Option | Default | Comment |
|---|---|---|
| $ENV['loglevel'] | SKP_LOG_INFO |
Allowed values: 0 - 16, or appropriate constants. |
Syslog
The syslog output sends messages to the syslog daemon on the localhost.
Configuration options:
| Option | Default | Comment |
|---|---|---|
| $ENV['log_to_syslog'] | true | Enables the syslog output. |
| $ENV['syslog_ident'] | SKP |
Prefixes all syslog messages. |
| $ENV['syslog_facility'] | LOG_LOCAL0 |
Syslog facility for routing purposes. |
| $ENV['syslog_delimiter'] | | |
ASCII 124 = |, must match syslog-parsing. |
Local file
This outputstream sends everything to a file in the local filesystem.
Please note that the appserver does not rotate the file.
Configuration options:
| Option | Default | Comment |
|---|---|---|
| $ENV['log_to_file'] | true | Enables the local file output. |
| $ENV['logfile'] | /var/log/teambeam/skp-infoserver.log |
Defines the filename where messages are logged. |
Logstash
The application server has experimental support for logstash via the syslog daemon on localhost.
Configuration options:
| Option | Default | Comment |
|---|---|---|
| $ENV['log_to_logstash'] | true | Enables the logstash output. |
| $ENV['logstash_facility'] | LOG_LOCAL2 |
Syslog facility for routing purposes. |