Stopping and starting Control Center

Before performing maintenance, such as operating system upgrades or applying patches, properly stop and start Control Center. This section provides procedures for single-host and multi-host deployments.

Stopping Control Center on the master host

Use this procedure to stop the Control Center service (serviced) on the master host.

Log in to the master host as root or as a user with superuser privileges.
Stop the top-level service serviced is managing, if necessary.
1. Show the status of running services.
```
serviced service status
```
  The top-level service is the service listed immediately below the headings line. - If the status of the top-level service and all child services is stopped, proceed to the next step. - If the status of the top-level service and all child services is not stopped, perform the remaining substeps.
2. Stop the top-level service.
```
serviced service stop Zenoss.resmgr
```
3. Monitor the stop.
```
serviced service status
```
  When the status of the top-level service and all child services is stopped, proceed to the next step.
Stop the Control Center service.
```
systemctl stop serviced
```
Ensure that no containers remain in the local repository.
1. Display the identifiers of all containers, running and exited.
```
docker ps -qa
```
  - If the command returns no result, stop. This procedure is complete.
  - If the command returns a result, perform the following substeps.
2. Remove all remaining containers.
```
docker ps -qa | xargs --no-run-if-empty docker rm -fv
```
3. Display the identifiers of all containers, running and exited.
```
docker ps -qa
```
  - If the command returns no result, stop. This procedure is complete.
  - If the command returns a result, perform the remaining substeps.
4. Disable the automatic startup of serviced.
```
systemctl disable serviced
```
5. Reboot the host.
```
reboot
```
6. Log in to the master host as root, or as a user with superuser privileges.
7. Enable the automatic startup of serviced.
```
systemctl enable serviced
```

Stopping Control Center on a delegate host

Use this procedure to stop the Control Center service (serviced) on a delegate host in a multi-host deployment. Repeat this procedure on each delegate host in your deployment.

Before performing this procedure on any delegate host, stop Control Center on the master host.

Log in to the delegate host as root or as a user with superuser privileges.
Stop the Control Center service.
```
systemctl stop serviced
```
Ensure that no containers remain in the local repository.
1. Display the identifiers of all containers, running and exited.
```
docker ps -qa
```
  - If the command returns no result, proceed to the next step.
  - If the command returns a result, perform the following substeps.
2. Remove all remaining containers.
```
docker ps -qa | xargs --no-run-if-empty docker rm -fv
```
  - If the remove command completes, proceed to the next step.
  - If the remove command does not complete, the most likely cause is an NFS conflict. Perform the following substeps.
3. Stop the NFS and Docker services.
```
systemctl stop nfs && systemctl stop docker
```
4. Start the NFS and Docker services.
```
systemctl start nfs && systemctl start docker
```
5. Repeat the attempt to remove all remaining containers.
```
docker ps -qa | xargs --no-run-if-empty docker rm -fv
```
  - If the remove command completes, proceed to the next step.
  - If the remove command does not complete, perform the remaining substeps.
6. Disable the automatic startup of serviced.
```
systemctl disable serviced
```
7. Reboot the host.
```
reboot
```
8. Log in to the delegate host as root, or as a user with superuser privileges.
9. Enable the automatic startup of serviced.
```
systemctl enable serviced
```
Dismount all filesystems mounted from the Control Center master host.

This step ensures no stale mounts remain when the storage on the master host is replaced.
1. Identify filesystems mounted from the master host.
```
awk '/serviced/ { print $1, $2 }' < /proc/mounts | grep -v '/opt/serviced/var/isvcs'
```
  - If the preceding command returns no result, stop. This procedure is complete.
  - If the preceding command returns a result, perform the following substeps.
2. Force the filesystems to dismount.
```
for FS in $(awk '/serviced/ { print $2 }' < /proc/mounts | grep -v '/opt/serviced/var/isvcs')
do
  umount -f $FS
done
```
3. Identify filesystems mounted from the master host.
```
awk '/serviced/ { print $1, $2 }' < /proc/mounts | grep -v '/opt/serviced/var/isvcs'
```
  - If the preceding command returns no result, stop. This procedure is complete.
  - If the preceding command returns a result, perform the following substeps.
4. Perform a lazy dismount.
```
for FS in $(awk '/serviced/ { print $2 }' < /proc/mounts | grep -v '/opt/serviced/var/isvcs')
do
  umount -f -l $FS
done
```
5. Restart the NFS service.
```
systemctl restart nfs
```
6. Determine whether any filesystems remain mounted from the master host.
```
awk '/serviced/ { print $1, $2 }' < /proc/mounts | grep -v '/opt/serviced/var/isvcs'
```
  - If the preceding command returns no result, stop. This procedure is complete.
  - If the preceding command returns a result, perform the remaining substeps.
7. Disable the automatic startup of serviced.
```
systemctl disable serviced
```
8. Reboot the host.
```
reboot
```
9. Log in to the delegate host as root, or as a user with superuser privileges.
10. Enable the automatic startup of serviced.
```
systemctl enable serviced
```

Starting Control Center (single-host deployment)

Use this procedure to start Control Center in a single-host deployment. The default configuration of the Control Center service (serviced) is to start when the host starts. This procedure is only needed after stopping serviced to perform maintenance tasks.

Log in to the master host as root or as a user with superuser privileges.

Enable serviced, if necessary.

systemctl is-enabled serviced || systemctl enable serviced

Start the Control Center service.
```
systemctl start serviced
```
Optional: Monitor the startup, if desired.
```
journalctl -u serviced -f -o cat
```

Once Control Center is started, it is ready to start managing applications. For more information, see Adding Resource Manager to a Control Center deployment.

Starting Control Center (multi-host deployment)

Use this procedure to start Control Center in a multi-host deployment. The default configuration of the Control Center service (serviced) is to start when the host starts. This procedure is only needed after stopping serviced to perform maintenance tasks.

Log in to the master host as root or as a user with superuser privileges.

Enable serviced, if necessary.

systemctl is-enabled serviced || systemctl enable serviced

Identify the hosts in the ZooKeeper ensemble.
```
grep -E '^[[:space:]]*SERVICED_ZK=' /etc/default/serviced
```
The result is a list of 1, 3, or 5 hosts, separated by the comma character (,). The master host is always a node in the ZooKeeper ensemble.
In separate windows, log in to each of the delegate hosts that are nodes in the ZooKeeper ensemble as root, or as a user with superuser privileges.
On all ensemble hosts, start serviced.

The window of time for starting a ZooKeeper ensemble is relatively short. The goal of this step is to start Control Center on each ensemble node at about the same time, so that each node can participate in electing the leader.
```
systemctl start serviced
```
On the master host, check the status of the ZooKeeper ensemble.
1. Attach to the container of the ZooKeeper service.
```
docker exec -it serviced-isvcs_zookeeper bash
```
2. Query the master host and identify its role in the ensemble.
  
  Replace Master with the hostname or IP address of the master host:
```
{ echo stats; sleep 1; } | nc Master 2181 | grep Mode
```
  The result includes leader or follower. When multiple hosts rely on the ZooKeeper instance on the master host, the result includes standalone.
3. Query the other delegate hosts to identify their role in the ensemble.
  
  Replace Delegate with the hostname or IP address of a delegate host:
```
{ echo stats; sleep 1; } | nc Delegate 2181 | grep Mode
```
4. Detach from the container of the ZooKeeper service.
```
exit
```
If none of the nodes reports that it is the ensemble leader within a few minutes of starting serviced, reboot the ensemble hosts.
Log in to each of the delegate hosts that are not nodes in the ZooKeeper ensemble as root, or as a user with superuser privileges, and then start serviced.
```
systemctl start serviced
```
Optional: Monitor the startup, if desired.
```
journalctl -u serviced -f -o cat
```

Once Control Center is started, it is ready to start managing applications. For more information, see Adding Resource Manager to a Control Center deployment.