- Configure Docker Desktop Ubuntu
- Create Docker Ubuntu Container
- Set Up Docker Ubuntu 20.04
- Create Docker Ubuntu Image
Estimated reading time: 11 minutes
After successfully installing and starting Docker, the dockerd
daemonruns with its default configuration. This topic shows how to customizethe configuration, start the daemon manually, and troubleshoot and debug thedaemon if you run into issues.
Start the daemon using operating system utilities
# docker run -name my-www -d -p 81:80 ubuntu-apache # docker ps Give Docker Container Name Now you can reference the container for manipulation (start, stop, top, stats, etc) only by using the assigned name. Check the correct page under Install Docker. To configure Docker to start automatically at system boot, see Configure Docker to start on boot. Start the daemon manually. If you don’t want to use a system utility to manage the Docker daemon, or just want to test things out, you can manually run it using the dockerd command.
On a typical installation the Docker daemon is started by a system utility,not manually by a user. This makes it easier to automatically start Docker whenthe machine reboots.
The command to start Docker depends on your operating system. Check the correctpage under Install Docker. To configure Dockerto start automatically at system boot, seeConfigure Docker to start on boot.
A Docker Swarm, or Docker cluster, is made up of one or more Dockerized hosts that function as manager nodes, and any number of worker nodes. Setting up such a system requires careful manipulation of the Linux firewall. The network ports required for a Docker Swarm to function correctly are: TCP port 2376 for secure Docker client communication. There are two versions of Docker – Docker CE (Community Edition) and Docker EE (Enterprise Edition). If you have a small-scale project, or you’re just learning, you will want to use Docker CE. In this tutorial, learn how to install Docker on Ubuntu 18.04. Docker includes multiple logging mechanisms to help you get information from running containers and services. These mechanisms are called logging drivers. Each Docker daemon has a default logging driver, which each container uses unless you configure it to use a different logging driver, or “log-driver” for short.
Start the daemon manually
If you don’t want to use a system utility to manage the Docker daemon, orjust want to test things out, you can manually run it using the dockerd
command. You may need to use sudo
, depending on your operating systemconfiguration.
When you start Docker this way, it runs in the foreground and sends its logsdirectly to your terminal.
To stop Docker when you have started it manually, issue a Ctrl+C
in yourterminal.
Configure the Docker daemon
There are two ways to configure the Docker daemon:
- Use a JSON configuration file. This is the preferred option, since it keepsall configurations in a single place.
- Use flags when starting
dockerd
.
You can use both of these options together as long as you don’t specify thesame option both as a flag and in the JSON file. If that happens, the Dockerdaemon won’t start and prints an error message.
To configure the Docker daemon using a JSON file, create a file at/etc/docker/daemon.json
on Linux systems, or C:ProgramDatadockerconfigdaemon.json
on Windows. On MacOS go to the whale in the taskbar > Preferences > Daemon > Advanced.
Here’s what the configuration file looks like:
With this configuration the Docker daemon runs in debug mode, uses TLS, andlistens for traffic routed to 192.168.59.3
on port 2376
.You can learn what configuration options are available in thedockerd reference docs
You can also start the Docker daemon manually and configure it using flags.This can be useful for troubleshooting problems.
Here’s an example of how to manually start the Docker daemon, using the sameconfigurations as above:
You can learn what configuration options are available in thedockerd reference docs, or by running:
Many specific configuration options are discussed throughout the Dockerdocumentation. Some places to go next include:
Docker daemon directory
The Docker daemon persists all data in a single directory. This tracks everythingrelated to Docker, including containers, images, volumes, service definition,and secrets.
By default this directory is:
/var/lib/docker
on Linux.C:ProgramDatadocker
on Windows.
You can configure the Docker daemon to use a different directory, using thedata-root
configuration option.
Since the state of a Docker daemon is kept on this directory, make sureyou use a dedicated directory for each daemon. If two daemons share the samedirectory, for example, an NFS share, you are going to experience errors thatare difficult to troubleshoot.
Troubleshoot the daemon
You can enable debugging on the daemon to learn about the runtime activity ofthe daemon and to aid in troubleshooting. If the daemon is completelynon-responsive, you can alsoforce a full stack trace of allthreads to be added to the daemon log by sending the SIGUSR
signal to theDocker daemon.
Troubleshoot conflicts between the daemon.json
and startup scripts
If you use a daemon.json
file and also pass options to the dockerd
command manually or using start-up scripts, and these options conflict,Docker fails to start with an error such as:
If you see an error similar to this one and you are starting the daemon manually with flags,you may need to adjust your flags or the daemon.json
to remove the conflict.
Note: If you see this specific error, continue to thenext section for a workaround.
If you are starting Docker using your operating system’s init scripts, you mayneed to override the defaults in these scripts in ways that are specific to theoperating system.
Use the hosts key in daemon.json with systemd
One notable example of a configuration conflict that is difficult to troubleshootis when you want to specify a different daemon address fromthe default. Docker listens on a socket by default. On Debian and Ubuntu systems using systemd
,this means that a host flag -H
is always used when starting dockerd
. If you specify ahosts
entry in the daemon.json
, this causes a configuration conflict (as in the above message)and Docker fails to start.
To work around this problem, create a new file /etc/systemd/system/docker.service.d/docker.conf
withthe following contents, to remove the -H
argument that is used when starting the daemon by default.
There are other times when you might need to configure systemd
with Docker, such asconfiguring a HTTP or HTTPS proxy.
Note: If you override this option and then do not specify a hosts
entry in the daemon.json
or a -H
flag when starting Docker manually, Docker fails to start.
Run sudo systemctl daemon-reload
before attempting to start Docker. If Docker startssuccessfully, it is now listening on the IP address specified in the hosts
key of thedaemon.json
instead of a socket.
Important: Setting hosts
in the daemon.json
is not supported on Docker Desktop for Windowsor Docker Desktop for Mac.
Out Of Memory Exceptions (OOME)
If your containers attempt to use more memory than the system has available,you may experience an Out Of Memory Exception (OOME) and a container, or theDocker daemon, might be killed by the kernel OOM killer. To prevent this fromhappening, ensure that your application runs on hosts with adequate memory andseeUnderstand the risks of running out of memory.
Read the logs
The daemon logs may help you diagnose problems. The logs may be saved in one ofa few locations, depending on the operating system configuration and the loggingsubsystem used:
Operating system | Location |
---|---|
RHEL, Oracle Linux | /var/log/messages |
Debian | /var/log/daemon.log |
Ubuntu 16.04+, CentOS | Use the command journalctl -u docker.service or /var/log/syslog |
Ubuntu 14.10- | /var/log/upstart/docker.log |
macOS (Docker 18.01+) | ~/Library/Containers/com.docker.docker/Data/vms/0/console-ring |
macOS (Docker <18.01) | ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/console-ring |
Windows | AppDataLocal |
Enable debugging
There are two ways to enable debugging. The recommended approach is to set thedebug
key to true
in the daemon.json
file. This method works for everyDocker platform.
Edit the
daemon.json
file, which is usually located in/etc/docker/
.You may need to create this file, if it does not yet exist. On macOS orWindows, do not edit the file directly. Instead, go toPreferences / Daemon / Advanced.If the file is empty, add the following:
If the file already contains JSON, just add the key
'debug': true
, beingcareful to add a comma to the end of the line if it is not the last linebefore the closing bracket. Also verify that if thelog-level
key is set,it is set to eitherinfo
ordebug
.info
is the default, and possiblevalues aredebug
,info
,warn
,error
,fatal
.Send a
HUP
signal to the daemon to cause it to reload its configuration.On Linux hosts, use the following command.On Windows hosts, restart Docker.
Instead of following this procedure, you can also stop the Docker daemon andrestart it manually with the debug flag -D
. However, this may result in Dockerrestarting with a different environment than the one the hosts’ startup scriptscreate, and this may make debugging more difficult.
Force a stack trace to be logged
If the daemon is unresponsive, you can force a full stack trace to be loggedby sending a SIGUSR1
signal to the daemon.
Linux:
Windows Server:
Download docker-signal.
Get the process ID of dockerd
Get-Process dockerd
.Run the executable with the flag
--pid=<PID of daemon>
.
This forces a stack trace to be logged but does not stop the daemon.Daemon logs show the stack trace or the path to a file containing thestack trace if it was logged to a file.
The daemon continues operating after handling the SIGUSR1
signal anddumping the stack traces to the log. The stack traces can be used to determinethe state of all goroutines and threads within the daemon.
View stack traces
The Docker daemon log can be viewed by using one of the following methods:
- By running
journalctl -u docker.service
on Linux systems usingsystemctl
/var/log/messages
,/var/log/daemon.log
, or/var/log/docker.log
on olderLinux systems
Note: It is not possible to manually generate a stack trace on Docker Desktop forMac or Docker Desktop for Windows. However, you can click the Docker taskbar icon andchoose Diagnose and feedback to send information to Docker if you run intoissues.
Look in the Docker logs for a message like the following:
The locations where Docker saves these stack traces and dumps depends on youroperating system and configuration. You can sometimes get useful diagnosticinformation straight from the stack traces and dumps. Otherwise, you can providethis information to Docker for help diagnosing the problem.
Check whether Docker is running
The operating-system independent way to check whether Docker is running is toask Docker, using the docker info
command.
You can also use operating system utilities, such assudo systemctl is-active docker
or sudo status docker
orsudo service docker status
, or checking the service status using Windowsutilities.
Finally, you can check in the process list for the dockerd
process, usingcommands like ps
or top
.
You can configure automatic log upload for continuous reports in Cloud App Security using a Docker on an on-premises Ubuntu, Red Hat Enterprise Linux (RHEL), or CentOS server.
Prerequisites
OS:
- Ubuntu 14.04, 16.04, and 18.04
- RHEL 7.2 or higher
- CentOS 7.2 or higher
Disk space: 250 GB
CPU: 2
RAM: 4 GB
Set your firewall as described in Network requirements
Note
If you have an existing log collector and want to remove it before deploying it again, or if you simply want to remove it, run the following commands:
Log collector performance
The Log collector can successfully handle log capacity of up to 50 GB per hour. The main bottlenecks in the log collection process are:
Network bandwidth - Your network bandwidth determines the log upload speed.
I/O performance of the virtual machine - Determines the speed at which logs are written to the log collector's disk. The log collector has a built-in safety mechanism that monitors the rate at which logs arrive and compares it to the upload rate. In cases of congestion, the log collector starts to drop log files. If your setup typically exceeds 50 GB per hour, it's recommended that you split the traffic between multiple log collectors.
Set up and configuration
Step 1 – Web portal configuration: Define data sources and link them to a log collector
Go to the Automatic log upload settings page.
- In the Cloud App Security portal, click the settings icon followed by Log collectors.
For each firewall or proxy from which you want to upload logs, create a matching data source.
- Click Add data source.
- Name your proxy or firewall.
- Select the appliance from the Source list. If you select Custom log format to work with a network appliance that isn't listed, see Working with the custom log parser for configuration instructions.
- Compare your log with the sample of the expected log format. If your log file format doesn't match this sample, you should add your data source as Other.
- Set the Receiver type to either FTP, FTPS, Syslog – UDP, or Syslog – TCP, or Syslog – TLS.
Note
Integrating with secure transfer protocols (FTPS and Syslog – TLS) often requires additional settings or your firewall/proxy.
f. Repeat this process for each firewall and proxy whose logs can be used to detect traffic on your network. It's recommended to set up a dedicated data source per network device to enable you to:
- Monitor the status of each device separately, for investigation purposes.
- Explore Shadow IT Discovery per device, if each device is used by a different user segment.
- Click Add data source.
Go to the Log collectors tab at the top.
- Click Add log collector.
- Give the log collector a name.
- Enter the Host IP address of the machine you'll use to deploy the Docker. The host IP address can be replaced with the machine name, if there is a DNS server (or equivalent) that will resolve the host name.
- Select all Data sources that you want to connect to the collector, and click Update to save the configuration.
Further deployment information will appear. Copy the run command from the dialog. You can use the copy to clipboard icon.
Export the expected data source configuration. This configuration describes how you should set the log export in your appliances.
Note
- A single Log collector can handle multiple data sources.
- Copy the contents of the screen because you will need the information when you configure the Log Collector to communicate with Cloud App Security. If you selected Syslog, this information will include information about which port the Syslog listener is listening on.
- For users sending log data via FTP for the first time, we recommend changing the password for the FTP user. For more information, see Changing the FTP password.
Step 2 – On-premises deployment of your machine
The following steps describe the deployment in Ubuntu.
Note
The deployment steps for other supported platforms may be slightly different.
Open a terminal on your Ubuntu machine.
Change to root privileges using the command:
sudo -i
To bypass a proxy in your network, run the following two commands:
If you accept the software license terms, uninstall old versions and install Docker CE by running the commands appropriate for your environment:
Remove old versions of Docker:
yum erase docker docker-engine docker.io
Install Docker engine prerequisites:
yum install -y yum-utils
Add Docker repository:
Install Docker engine:
yum -y install docker-ce
Start Docker
Test Docker installation:
docker run hello-world
Remove old versions of Docker:
yum erase docker docker-engine docker.io
Install Docker engine prerequisites:
Add Docker repository:
Install dependencies:
Install Docker engine:
sudo yum install docker-ce
Start Docker
Test Docker installation:
docker run hello-world
Remove the container-tools module:
yum module remove container-tools
Add the Docker CE repository:
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
Modify the yum repo file to use CentOS 8/RHEL 8 packages:
sed -i s/7/8/g /etc/yum.repos.d/docker-ce.repo
Install Docker CE:
yum install docker-ce
Start Docker
Test Docker installation:
docker run hello-world
Remove old versions of Docker:
apt-get remove docker docker-engine docker.io
If you are installing on Ubuntu 14.04, install the linux-image-extra package.
Install Docker engine prerequisites:
Verify that the apt-key fingerprint UID is docker@docker.com:
apt-key fingerprint | grep uid
Install Docker engine:
Test Docker installation:
docker run hello-world
Deploy the collector image on the hosting machine by importing the collector configuration. Import the configuration by copying the run command generated in the portal. If you need to configure a proxy, add the proxy IP address and port number. For example, if your proxy details are 192.168.10.1:8080, your updated run command is:
Verify that the collector is running properly with the following command:
docker logs <collector_name>
Configure Docker Desktop Ubuntu
You should see the message: Finished successfully!
Step 3 - On-premises configuration of your network appliances
Configure your network firewalls and proxies to periodically export logs to the dedicated Syslog port or the FTP directory according to the directions in the dialog. For example:
Step 4 - Verify the successful deployment in the Cloud App Security portal
Create Docker Ubuntu Container
Check the collector status in the Log collector table and make sure the status is Connected. If it's Created, it's possible the log collector connection and parsing haven't completed.
You can also go to the Governance log and verify that logs are being periodically uploaded to the portal.
Alternatively, you can check the log collector status from within the docker container using the following commands:
- Log in to the container by using this command:
docker exec -it <Container Name> bash
- Verify the log collector status using this command:
collector_status -p
If you have problems during deployment, see Troubleshooting Cloud Discovery.
Set Up Docker Ubuntu 20.04
Optional - Create custom continuous reports
Verify that the logs are being uploaded to Cloud App Security and that reports are generated. After verification, create custom reports. You can create custom discovery reports based on Azure Active Directory user groups. For example, if you want to see the cloud use of your marketing department, import the marketing group using the import user group feature. Then create a custom report for this group. You can also customize a report based on IP address tag or IP address ranges.
- In the Cloud App Security portal, under the Settings cog, select Cloud Discovery settings, and then select Continuous reports.
- Click the Create report button and fill in the fields.
- Under the Filters you can filter the data by data source, by imported user group, or by IP address tags and ranges.
Next steps
Create Docker Ubuntu Image
If you run into any problems, we're here to help. To get assistance or support for your product issue, please open a support ticket.