Skip to content

Configuration file

The Control Center configuration file, /etc/default/serviced, contains Bash environment variables that are read by the serviced daemon startup script. The order of the following list matches the order of the variables in the file.

HOME

Default: (the value of shell variable HOME)

The path Docker clients use to locate the .docker/config.json authentication file, which contains Docker Hub credentials.

TMPDIR

Default: (the value of shell variable TMPDIR)

The path serviced uses for temporary files.

GOMAXPROCS

Default: 2

The maximum number of CPU cores serviced uses. This setting is relevant only on the master host.

The value can safely be set to 50% of the available processors on the host, and higher if necessary, but must always be fewer than the total number of available processors.

SERVICED_MASTER

Default: 1 (true)

Assigns the role of a serviced instance, either master or delegate. The master runs the application services scheduler and other internal services. Delegates run the application services assigned to the resource pool to which they belong.

Only one serviced instance can be the master; all other instances must be delegates. The default value assigns the master role. To assign the delegate role, set the value to 0 (false). This variable must be explicitly set on all Control Center hosts.

SERVICED_MASTER_IP

Default: 127.0.0.1

A convenience variable, for use in places where the IP address or hostname of the master host is required. This variable is unused unless it is both set here and referenced elsewhere. (For example, by replacing {{SERVICED_MASTER_IP}} with $SERVICED_MASTER_IP.)

SERVICED_MASTER_POOLID

Default: default

The name of the default resource pool. This variable is only used the first time serviced is started.

SERVICED_ZK

Default: (none)

The list of endpoints in the serviced ZooKeeper ensemble, separated by the comma character (,). Each endpoint identifies an ensemble node. Each Control Center server and in-container proxy uses SERVICED_ZK to create a randomized, round-robin list, and cycles through the list when it attempts to establish a connection with the lead ZooKeeper host.

SERVICED_DOCKER_REGISTRY

Default: localhost:5000

The endpoint of the local Docker registry, which serviced uses to store internal services and application images.

If the default value is changed, the host's Docker configuration file must include the --insecure-registry flag with the same value as this variable.

The safest replacement for localhost is the IPv4 address of the registry host. Otherwise, the fully-qualified domain name of the host must be specified.

SERVICED_OUTBOUND_IP

Default: (none)

The IPv4 address that delegates use to connect to the master host. When no address is specified, serviced attempts to discover its public IP address by pinging google.com.

This variable must be set on all Control Center hosts in either of the following scenarios:

  • Control Center is deployed behind a firewall and google.com is not reachable. Set the value to the IPv4 address of the master host.
  • Control Center is deployed in a high-availability cluster. Set the value to the virtual IPv4 address of the high-availability cluster (HA-Virtual-IP).

Setting the Docker HTTP_PROXY or HTTPS_PROXY environment variables prevents access to the IP address defined with this variable. To enable access, unset the Docker variables, and then reboot the host.

SERVICED_STATIC_IPS

Default: (none)

A list of one or more static IP addresses that are available for IP assignment. Use the comma character (,) to separate addresses.

SERVICED_ENDPOINT

Default: {{SERVICED_MASTER_IP}}:4979

The endpoint of the serviced RPC server. Replace {{SERVICED_MASTER_IP}} with the IP address or hostname of the serviced master host. The port number of this endpoint must match the value of the SERVICED_RPC_PORT variable defined on the serviced master host.

SERVICED_MAX_RPC_CLIENTS

Default: 3

The preferred maximum number of simultaneous connections a serviced delegate uses for RPC requests. The value is used to create a pool of sockets, which are reused as needed. Increasing the value increases the number of open sockets and the use of socket-related operating system resources.

When the demand for connections exceeds the supply of open sockets, serviced opens more sockets. When demand eases, serviced reduces the number of open sockets to the preferred maximum.

SERVICED_RPC_PORT

Default: 4979

The port on which the serviced RPC server listens for connections. The value of this variable must match the port number defined for the SERVICED_ENDPOINT) variable on all serviced delegate hosts.

SERVICED_RPC_CERT_VERIFY

Default: false

Determines whether serviced performs TLS certificate verification for RPC connections. The certificate is defined by the SERVICED_CERT_FILE. variable.

SERVICED_RPC_DISABLE_TLS

Default: false

Determines whether serviced encrypts RPC traffic with TLS.

SERVICED_RPC_TLS_MIN_VERSION

Default: VersionTLS12

The version of TLS serviced accepts for RPC connections.

SERVICED_RPC_TLS_CIPHERS

Default: (list of ciphers)

The list of TLS ciphers serviced prefers for RPC connections, separated by the comma character (,):

  • TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
  • TLS_RSA_WITH_AES_128_CBC_SHA
  • TLS_RSA_WITH_AES_256_CBC_SHA
  • TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
  • TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

Other ciphers are supported; the preceding ciphers provide strong security for relatively low processing overhead.

An instance of serviced is on both ends of an RPC connection, so both daemons use the first cipher in the list. To use a different cipher, put it first in the list, on all Control Center hosts.

SERVICED_UI_PORT

Default: :443

The port on which the serviced HTTP server listens for requests for its internal services and for tenant services. The value may be expressed as follows:

IP-Address:Port-Number

:Port-Number

Port-Number

Tenant applications can specify alternative ports with the port public endpoint feature.

The value of this variable must be identical on all Control Center hosts in a deployment.

SERVICED_UI_POLL_FREQUENCY

Default: 3

The number of seconds between polls from Control Center browser interface clients. The value is included in a JavaScript library that is sent to the clients.

SERVICED_MUX_PORT

Default: 22250

The port serviced uses for traffic among Docker containers.

SERVICED_MUX_DISABLE_TLS

Default: 0

Determines whether inter-host traffic among Docker containers is encrypted with TLS. Intra-host traffic among Docker containers is not encrypted. To disable encryption, set the value to 1.

SERVICED_MUX_TLS_MIN_VERSION

Default: VersionTLS12

The version of TLS serviced accepts for mux traffic.

SERVICED_MUX_TLS_CIPHERS

Default: (list of ciphers)

The list of TLS ciphers serviced prefers for mux traffic, separated by the comma character (,):

  • TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
  • TLS_RSA_WITH_AES_128_CBC_SHA
  • TLS_RSA_WITH_AES_256_CBC_SHA
  • TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
  • TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

Other ciphers are supported; the preceding ciphers provide strong security for relatively low processing overhead.

An instance of serviced is on both ends of a mux connection, so both daemons use the first cipher in the list. To use a different cipher, put it first in the list, on all Control Center hosts.

SERVICED_ISVCS_PATH

Default: /opt/serviced/var/isvcs

The location of serviced internal services data.

SERVICED_VOLUMES_PATH

Default: /opt/serviced/var/volumes

The location of serviced application data.

SERVICED_BACKUPS_PATH

Default: /opt/serviced/var/backups

The location of serviced backup files.

SERVICED_LOG_PATH

Default: /var/log/serviced

The location of serviced audit log files. Non-audit (operations) messages are written to journald.

SERVICED_KEY_FILE

Default: $TMPDIR/zenoss_key.[0-9]+

The path of a digital certificate key file. Choose a location that is not modified during operating system updates, such as /etc.

This key file is used for all TLS-encrypted communications (RPC, mux, and HTTPS). The default, insecure key file is created when the serviced web server first starts, and is based on a public key that is compiled into serviced.

SERVICED_CERT_FILE

Default: $TMPDIR/zenoss_cert.[0-9]+

The path of a digital certificate file. Choose a location that is not modified during operating system updates, such as /etc. Certificates with passphrases are not supported.

This certificate file is used for all TLS-encrypted communications (RPC, mux, and HTTPS). The default, insecure certificate file is created when the serviced web server first starts, and is based on a public certificate that is compiled into serviced.

SERVICED_TLS_MIN_VERSION

Default: VersionTLS12

The version of TLS that serviced accepts for HTTPS traffic.

SERVICED_TLS_CIPHERS

Default: (list of ciphers)

The list of TLS ciphers that serviced accepts for HTTPS traffic, separated by the comma character (,):

  1. TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  2. TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  3. TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  4. TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
  5. TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
  6. TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
  7. TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
  8. TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
  9. TLS_RSA_WITH_AES_256_CBC_SHA
  10. TLS_RSA_WITH_AES_128_CBC_SHA
  11. TLS_RSA_WITH_AES_128_GCM_SHA256
  12. TLS_RSA_WITH_AES_256_GCM_SHA384

To disable support for most ciphers, you can remove them from the list. The following rules apply to the list:

  • The first cipher, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, must always be present in the list of ciphers.
  • The first four ciphers in the list must always precede any of the ciphers that appear after the first four. The first four ciphers are valid for HTTP/2, while the remaining ciphers are not.
SERVICED_FS_TYPE

Default: devicemapper

The driver to manage application data storage on the serviced master host. Only devicemapper is supported in production deployments.

The only supported storage layout for the devicemapper driver is an LVM thin pool. To create a thin pool, use the serviced-storage utility. To specify the name of the thin pool device, use SERVICED_DM_THINPOOLDEV.

SERVICED_DM_ARGS

Default: (none)

Customized startup arguments for the devicemapper storage driver.

SERVICED_DM_BASESIZE

Default: 100G

The base size of virtual storage devices for tenants in the application data thin pool, in gigabytes. The units symbol (G) is required. This variable is used when serviced starts for the first time, to set the initial size of tenant devices, and when a backup is restored, to set the size of the restored tenant device.

The base size device is sparse device that occupies at most 1MB of space in the application data thin pool; its size has no immediate practical impact. However, the application data thin pool should have enough space for twice the size of each tenant device it supports, to store both the data itself and snapshots of the data. Since the application data thin pool is an LVM logical volume, its size can be increased at any time. Likewise, the size of a tenant device can be increased, as long as the available space in the thin pool can support the larger tenant device plus snapshots.

SERVICED_DM_LOOPDATASIZE

Default: 100G

Specifies the size of the data portion of the loop-back file. This setting is ignored when SERVICED_ALLOW_LOOP_BACK is false.

SERVICED_DM_LOOPMETADATASIZE

Default: 2G

Specifies the size of the metadata portion of the loop-back file. This setting is ignored when SERVICED_ALLOW_LOOP_BACK is false.

SERVICED_DM_THINPOOLDEV

Default: (none)

The name of the thin pool device to use with the devicemapper storage driver.

SERVICED_STORAGE_STATS_UPDATE_INTERVAL

Default: 300 (5 minutes)

The number of seconds between polls of kernel statistics about the application data thin pool.

This setting is ignored when the operating system kernel version is less than 3.10.0-366.

SERVICED_ALLOW_LOOP_BACK

Default: false

Determines whether loop-back files can be used with the devicemapper storage driver. This option is not supported for production use.

SERVICED_MAX_CONTAINER_AGE

Default: 86400 (24 hours)

The number of seconds serviced waits before removing a stopped container.

SERVICED_VIRTUAL_ADDRESS_SUBNET

Default: 10.3.0.0/16

The private subnet for containers that use virtual IP addresses on a host. This value may be unique on each Control Center host, if necessary.

RFC 1918 restricts private networks to the 10.0/24, 172.16/20, and 192.168/16 address spaces. However, serviced accepts any valid IPv4 address space.

Specify the value in CIDR notation. A /29 network provides sufficient address space.

SERVICED_LOG_LEVEL

Default: 0

The log level serviced uses when writing to the system log. Valid values are 0 (normal) and 2 (debug).

See also Enabling serviced debug messages.

SERVICED_LOG_ADDRESS

Default: {{SERVICED_MASTER_IP}}:5042

The endpoint of the logstash service. Replace {{SERVICED_MASTER_IP}} with the IP address or hostname of the serviced master host.

SERVICED_LOGSTASH_ES

Default: {{SERVICED_MASTER_IP}}:9100

The endpoint of the Elasticsearch service for logstash. On delegate hosts, replace {{SERVICED_MASTER_IP}} with the IP address or hostname of the Elasticsearch host, which by default is the serviced master host.

SERVICED_LOGSTASH_STDOUT

Default: false

Determines whether the logstash services writes its output to STDOUT, for aggregated logging.

SERVICED_LOGSTASH_MAX_DAYS

Default: 14

The maximum number of days to keep application logs in the logstash database before purging them.

SERVICED_LOGSTASH_MAX_SIZE

Default: 10

The maximum size of the logstash database, in gigabytes.

SERVICED_LOGSTASH_CYCLE_TIME

Default: 6

The amount of time between logstash purges, in hours.

SERVICED_STATS_PORT

Default: {{SERVICED_MASTER_IP}}:8443

The endpoint of the serviced metrics consumer service. Replace {{SERVICED_MASTER_IP}} with the IP address or hostname of the serviced master host.

SERVICED_STATS_PERIOD

Default: 10

The frequency, in seconds, at which delegates gather metrics to send to the serviced metrics consumer service on the master host.

SERVICED_SVCSTATS_CACHE_TIMEOUT

Default: 5

The number of seconds to cache statistics about services. The cache is used by Control Center browser interface clients.

SERVICED_DEBUG_PORT

Default: 6006

The port on which serviced listens for HTTPS requests for the Go profiler. To stop listening for requests, set the value to -1.

SERVICED_ISVCS_ENV_[0-9]+

Default: (none)

Startup arguments to pass to internal services. You may define multiple arguments, each for a different internal service. The variables themselves, and their arguments, use the following syntax:

SERVICED_ISVCS_ENV_%d

Each variable name ends with a unique integer in place of %d.

Service-Name:Key=Value

The value of each variable includes the following elements, in order:

  1. Service-Name, the internal service name. The following command returns the internal service names that may be used for Service-Name:

    docker ps | awk '/serviced-isvcs:/{print $NF}'

  2. The colon character (:).

  3. Key, a variable to pass to the internal service.

  4. The equals sign character (=).

  5. Value, the definition of the variable to pass to the internal service.

The following example variable passes ES_JAVA_OPTS=-Xmx4g to the Elasticsearch internal service.

SERVICED_ISVCS_ENV_0=serviced-isvcs_elasticsearch-logstash:ES_JAVA_OPTS=-Xmx4g

SERVICED_ADMIN_GROUP

Default: wheel

The name of the Linux group on the serviced master host whose members are authorized to use the serviced browser interface. You may replace the default group with a group that does not have superuser privileges.

SERVICED_ALLOW_ROOT_LOGIN

Default: 1 (true)

Determines whether the root user account on the serviced master host may be used to gain access to the serviced browser interface.

SERVICED_IPTABLES_MAX_CONNECTIONS

Default: 655360

The default value of this variable ensures that a serviced delegate does not run out of connections if the serviced master goes down. The connections are automatically cleaned up by the kernel soon after the serviced master comes back online.

SERVICED_SNAPSHOT_TTL

Default: 12

The number of hours an application data snapshot is retained before removal. To disable snapshot removal, set the value to zero. The application data storage can fill up rapidly when this value is zero or too high.

SERVICED_NFS_CLIENT

Default: 1

DEPRECATED: Prevent a delegate host from mounting the DFS.

SERVICED_SERVICE_MIGRATION_TAG

Default: 1.0.2

Overrides the default value for the service migration image.

SERVICED_ISVCS_START

Default: (none)

Enables one or more internal services to run on a delegate host. Currently, only zookeeper has been tested.

SERVICED_ISVCS_ZOOKEEPER_ID

Default: (none)

The unique identifier of a ZooKeeper ensemble node. The identifier must be a positive integer.

SERVICED_ISVCS_ZOOKEEPER_QUORUM

Default: (none)

The comma-separated list of nodes in a ZooKeeper ensemble. Each entry in the list specifies the ZooKeeper ID, IP address or hostname, peer communications port, and leader communications port of a node in the ensemble. Each quorum definition must be unique, so the IP address or hostname of the "current" host must be 0.0.0.0.

The following example shows the syntax of a node entry:

ZooKeeper-ID@Host-IP-Or-Name:2888:3888

SERVICED_ISVCS_ZOOKEEPER_USERNAME

Default: (none)

The user account name to use to create a DIGEST-MD5 hash for secure communications during the Zookeeper leader-election phase.

SERVICED_ISVCS_ZOOKEEPER_PASSWD

Default: (none)

The password to use to create a DIGEST-MD5 hash for secure communications during the Zookeeper leader-election phase.

SERVICED_ISVCS_OPENTSDB_USERNAME

Default: (none)

The account name to use to gain access to the internal services OpenTSDB instance.

SERVICED_ISVCS_OPENTSDB_PASSWORD

Default: (none)

The password to use to gain access to the internal services OpenTSDB instance.

SERVICED_ZOOKEEPER_ACL_USER

Default: (none)

The user account name to use to create a DIGEST-MD5 hash for controlling access to Zookeeper data nodes.

SERVICED_ZOOKEEPER_ACL_PASSWD

Default: (none)

The password to use to create a DIGEST-MD5 hash for controlling access to Zookeeper data nodes.

SERVICED_DOCKER_LOG_DRIVER

Default: json-file

The log driver for all Docker container logs, including containers for Control Center internal services. Valid values:

  • json-file
  • syslog
  • journald
  • gelf
  • fluentd
  • none

This is a direct port of the Docker --log-driver option.

SERVICED_DOCKER_LOG_CONFIG

Default: max-file=5,max-size=10m

A comma-separated list of Docker --log-opt options as key=value pairs. To specify the default values for a log driver, or for drivers that need no additional options, such as journald, use a single comma character (,) as the value of this variable.

SERVICED_DOCKER_DNS

Default: (empty)

The IP address of one or more DNS servers. The value of this variable is injected into each Docker container that serviced starts. Separate multiple values with the comma character (,).

SERVICED_OPTS

Default: (empty)

Special options for the serviced startup command.

SERVICED_SNAPSHOT_USE_PERCENT

Default: 20

The amount of free space in the thin pool specified with SERVICED_DM_THINPOOLDEV, expressed as a percentage the total size. This value is used to determine whether the thin pool can hold a new snapshot.

SERVICED_ZK_SESSION_TIMEOUT

Default: 15

The number of seconds the lead ZooKeeper host waits before flushing an inactive connection.

SERVICED_ZK_CONNECT_TIMEOUT

Default: 1

The number of seconds Control Center waits for a connection to the lead ZooKeeper host.

SERVICED_ZK_PER_HOST_CONNECT_DELAY

Default: 0

The number of seconds serviced waits before attempting to connect to the next host in its round-robin list of ZooKeeper hosts. For more information about the round-robin list, see SERVICED_ZK.

SERVICED_ZK_RECONNECT_START_DELAY

Default: 1

SERVICED_ZK_RECONNECT_START_DELAY and SERVICED_ZK_RECONNECT_MAX_DELAY are used together when Control Center is unable to re-establish a connection with the lead ZooKeeper host.

To prevent unnecessary spikes in TCP traffic, serviced waits a randomized amount of time that is equal to plus or minus 20% of the value of SERVICED_ZK_RECONNECT_START_DELAY. If serviced is unable to reconnect after contacting all of the hosts in its round-robin list of ZooKeeper hosts, the wait time is increased by a randomized value and the process of attempting to reconnect begins again. If the attempts fail again, the process repeats until the wait time reaches the value of SERVICED_ZK_RECONNECT_MAX_DELAY, and the wait time of subsequent reconnection attempts is capped at SERVICED_ZK_RECONNECT_MAX_DELAY. Once connection is re-established, the wait time is reset to SERVICED_ZK_RECONNECT_START_DELAY.

For more information about the round-robin list, see SERVICED_ZK.

SERVICED_ZK_RECONNECT_MAX_DELAY

Default: 1

See SERVICED_ZK_RECONNECT_START_DELAY.

SERVICED_ES_STARTUP_TIMEOUT

Default: 240

The number of seconds to wait for the Elasticsearch service to start.

SERVICED_MAX_DFS_TIMEOUT

Default: 300

The number of seconds until a DFS snapshot attempt times out.

SERVICED_RPC_DIAL_TIMEOUT

Default: 30

The number of seconds until an RPC connection attempt times out.

SERVICED_AUTH_TOKEN_EXPIRATION

Default: 3600 (1 hour)

The expiration time, in seconds, of delegate authentication tokens. This timeout affects RPC, mux, and serviced internal services endpoint communications.

SERVICED_CONTROLLER_BINARY

Default: /opt/serviced/bin/serviced-controller

The path of the serviced-controller binary, which runs in every container that serviced manages.

SERVICED_HOME

Default: /opt/serviced

The path of the home directory for serviced.

SERVICED_ETC_PATH

Default: /opt/serviced/etc

The path of the directory for serviced configuration files. The default is SERVICED_HOME/etc.

SERVICED_VHOST_ALIASES

Default: (none)

A list of hostname aliases for a host; for example, localhost. Separate multiple values with the comma character (,).