Docker19.03 新特性

2019-09-26

docker19.03新特性

Automatically use TLS

Update: Changes to GitLab CI/CD and Docker in Docker with Docker 19.03

安装使用都需明确指明版本，建议禁用latest或者stable

This is to help prevent users’ jobs randomly failing when a new update comes out.

如果出现

Client:
 Debug Mode: false

Server:
ERROR: error during connect: Get http://docker:2375/v1.40/info: dial tcp: lookup docker on 114.114.114.114:53: no such host
errors pretty printing info

在gitlab-runner里的volumes挂载上本地daemon.json

volumes = ["/certs/client", "/cache", "/etc/default/docker:/etc/default/docker", "/etc/docker/daemon.json:/etc/docker/daemon.json"]

给出两种解决方法：

To fix the problem above you have two options:

Configure GitLab Runner to use TLS.
Explicitly turn off TLS.

Configure TLS

Since the service docker:dind will create the certificates, we need to have the certificate shared between the service and the job container. To do this we have to add a mount inside of the volumes under the [runners.docker] section.

For example:

[[runners]]
  name = "My Docker Runner"
  url = "http://gitlab.com"
  token = ""
  executor = "docker"
  [runners.custom_build_dir]
  [runners.docker]
    privileged = true
    volumes = ["/certs/client", "/cache"]
    shm_size = 0

If you’re a GitLab.com user, we’ve already done the config change above for you on the Shared Runners.

Also, update .gitlab-ci.yml accordingly to specify the DOCKER_TLS_CERTDIR

image: docker:19.03.0

variables:
  DOCKER_DRIVER: overlay2
  # Create the certificates inside this directory for both the server
  # and client. The certificates used by the client will be created in
  # /certs/client so we only need to share this directory with the
  # volume mount in `config.toml`.
  DOCKER_TLS_CERTDIR: "/certs"

services:
  - docker:19.03.0-dind

before_script:
  - docker info

build:
  stage: build
  script:
    - docker build -t my-docker-image .
    - docker run my-docker-image /script/to/run/tests

Disable TLS

You might not have access to update the volume mounting inside of the config.toml, so the only option is to disable TLS. You can do this by setting the environment variable DOCKER_TLS_CERTDIR to an empty value.

For GitLab.com Shared Runners users this is done already using the environment settings, which works the same way.

image: docker:19.03.0

variables:
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: ""

services:
  - docker:19.03.0-dind

before_script:
  - docker info

build:
  stage: build
  script:
    - docker build -t my-docker-image .
    - docker run my-docker-image /script/to/run/tests

We would like to thank the rest of the community with all the feedback and help throughout #4501.

SSH keys when using the Docker executor

GitLab currently doesn’t have built-in support for managing SSH keys in a build environment (where the GitLab Runner runs).

The SSH keys can be useful when:

You want to checkout internal submodules
You want to download private packages using your package manager (e.g., Bundler)
You want to deploy your application to your own server, or, for example, Heroku（重点）
You want to execute SSH commands from the build environment to a remote server（重点）
You want to rsync files from the build environment to a remote server

If anything of the above rings a bell, then you most likely need an SSH key.

The most widely supported method is to inject an SSH key into your build environment by extending your .gitlab-ci.yml, and it’s a solution which works with any type of executor (Docker, shell, etc.)

How it works

Create a new SSH key pair locally with ssh-keygen
Add the private key as a variable to your project
Run the ssh-agent during job to load the private key.
Copy the public key to the servers you want to have access to (usually in ~/.ssh/authorized_keys) or add it as a deploy key if you are accessing a private GitLab repository.

Note: The private key will not be displayed in the job trace, unless you enable debug tracing. You might also want to check the visibility of your pipelines.

SSH keys when using the Docker executor

When your CI/CD jobs run inside Docker containers (meaning the environment is contained) and you want to deploy your code in a private server, you need a way to access it. This is where an SSH key pair comes in handy.【理解】

You will first need to create an SSH key pair. For more information, follow the instructions to generate an SSH key.Do not add a passphrase to the SSH key, or the before_script will prompt for it.
Create a new variable. As Key enter the name SSH_PRIVATE_KEY and in the Value field paste the content of your private key that you created earlier.
Modify your .gitlab-ci.yml with a before_script action. In the following example, a Debian based image is assumed. Edit to your needs:

before_script:
  ##
  ## Install ssh-agent if not already installed, it is required by Docker.
  ## (change apt-get to yum if you use an RPM-based image)
  ##
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'

  ##
  ## Run ssh-agent (inside the build environment)
  ##
  - eval $(ssh-agent -s)

  ##
  ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
  ## We're using tr to fix line endings which makes ed25519 keys work
  ## without extra base64 encoding.
  ## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556
  ##
  - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -

  ##
  ## Create the SSH directory and give it the right permissions
  ##
  - mkdir -p ~/.ssh
  - chmod 700 ~/.ssh

  ##
  ## Optionally, if you will be using any Git commands, set the user name and
  ## and email.
  ##
  #- git config --global user.email "user@example.com"
  #- git config --global user.name "User name"

Note: The before_script can be set globally or per-job

Make sure the private server’s SSH host keys are verified.
As a final step, add the public key from the one you created in the first step to the services that you want to have an access to from within the build environment. If you are accessing a private GitLab repository you need to add it as a deploy key.

Verifying the SSH host keys

GG在了这里

It is a good practice to check the private server’s own public key to make sure you are not being targeted by a man-in-the-middle attack. In case anything suspicious happens, you will notice it since the job would fail (the SSH connection would fail if the public keys would not match).

To find out the host keys of your server, run the ssh-keyscan command from a trusted network (ideally, from the private server itself):

## Use the domain name
ssh-keyscan example.com

## Or use an IP
ssh-keyscan 1.2.3.4

Create a new variable with SSH_KNOWN_HOSTS as “Key”, and as a “Value” add the output of ssh-keyscan.

Note: If you need to connect to multiple servers, all the server host keys need to be collected in the Value of the variable, one key per line.

Tip: By using a variable instead of ssh-keyscan directly inside .gitlab-ci.yml, it has the benefit that you don’t have to change .gitlab-ci.yml if the host domain name changes for some reason. Also, the values are predefined by you, meaning that if the host keys suddenly change, the CI/CD job will fail, and you’ll know there’s something wrong with the server or the network.

Now that the SSH_KNOWN_HOSTS variable is created, in addition to the content of .gitlab-ci.yml above, here’s what more you need to add:

before_script:
  ##
  ## Assuming you created the SSH_KNOWN_HOSTS variable, uncomment the
  ## following two lines.
  ##
  - echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
  - chmod 644 ~/.ssh/known_hosts

  ##
  ## Alternatively, use ssh-keyscan to scan the keys of your private server.
  ## Replace example.com with your private server's domain name. Repeat that
  ## command if you have more than one server to connect to.
  ##
  #- ssh-keyscan example.com >> ~/.ssh/known_hosts
  #- chmod 644 ~/.ssh/known_hosts

  ##
  ## You can optionally disable host key checking. Be aware that by adding that
  ## you are susceptible to man-in-the-middle attacks.
  ## WARNING: Use this only with the Docker executor, if you use it with shell
  ## you will overwrite your user's SSH config.
  ##
  #- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'

gitlab-examples/ssh-private-key/blob/master/.gitlab-ci.yml

https://gitlab.com/gitlab-examples/ssh-private-key/blob/master/.gitlab-ci.yml

image: ubuntu

before_script:
  ##
  ## Install ssh-agent if not already installed, it is required by Docker.
  ## (change apt-get to yum if you use an RPM-based image)
  ##
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client git -y )'

  ##
  ## Run ssh-agent (inside the build environment)
  ##
  - eval $(ssh-agent -s)

  ##
  ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
  ## We're using tr to fix line endings which makes ed25519 keys work
  ## without extra base64 encoding.
  ## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556
  ##
  - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null

  ##
  ## Create the SSH directory and give it the right permissions
  ##
  - mkdir -p ~/.ssh
  - chmod 700 ~/.ssh

  ##
  ## Use ssh-keyscan to scan the keys of your private server. Replace gitlab.com
  ## with your own domain name. You can copy and repeat that command if you have
  ## more than one server to connect to.
  ##
  - ssh-keyscan gitlab.com >> ~/.ssh/known_hosts
  - chmod 644 ~/.ssh/known_hosts

  ##
  ## Alternatively, assuming you created the SSH_SERVER_HOSTKEYS variable
  ## previously, uncomment the following two lines instead.
  ##
  #- echo "$SSH_SERVER_HOSTKEYS" > ~/.ssh/known_hosts'
  #- chmod 644 ~/.ssh/known_hosts

  ##
  ## You can optionally disable host key checking. Be aware that by adding that
  ## you are suspectible to man-in-the-middle attacks.
  ## WARNING: Use this only with the Docker executor, if you use it with shell
  ## you will overwrite your user's SSH config.
  ##
  #- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'

  ##
  ## Optionally, if you will be using any Git commands, set the user name and
  ## email.
  ##
  - git config --global user.email "darth@empire.com"
  - git config --global user.name "Darth Vader"


Test SSH:
  script:
  # try to connect to GitLab.com
  - ssh git@gitlab.com

  # try to clone yourself. A *PUBLIC* key paired to the SSH_PRIVATE_KEY was added as deploy key to this repository
  - git clone git@gitlab.com:gitlab-examples/ssh-private-key.git

|阅读量()