Docker仓库之Harbor

知识就是力量，时间就是生命。这篇文章主要讲述Docker仓库之Harbor相关的知识，希望能为你提供帮助。
Docker仓库一、Docker单机仓库Docker Registry作为Docker的核心组件之一负责单主机的镜像内容的存储与分发，客户端的docker pull以及push命令都将直接与registry进行交互,最初版本的registry 由python实现，由于设计初期在安全性，性能以及API的设计上有着诸多的缺陷，该版本在0.9之后停止了开发，由新项目distribution（新的docker register被称为Distribution）来重新设计并开发下一代registry，新的项目由go语言开发，所有的API，底层存储方式，系统架构都进行了全面的重新设计已解决上一代registry中存在的问题，2016年4月份registry 2.0正式发布，docker 1.6版本开始支持registry 2.0，而八月份随着docker 1.8 发布，docker hub正式启用2.1版本registry全面替代之前版本 registry，新版registry对镜像存储格式进行了重新设计并和旧版不兼容，docker 1.5和之前的版本无法读取2.0的镜像，另外，Registry 2.4版本之后支持了回收站机制，也就是可以删除镜像了，在2.4版本之前是无法支持删除镜像的，所以如果你要使用最好是大于Registry 2.4版本的
官方文档地址: https://docs.docker.com/registry/
官方github 地址: https://github.com/docker/distribution
官方部署文档: https://github.com/docker/docker.github.io/blob/master/registry/deploying.md

下载docker registry 镜像

[root@localhost docker]# docker pull registry:2.7.1 2.7.1: Pulling from library/registry 79e9f2f55bf5: Pull complete 0d96da54f60b: Pull complete 5b27040df4a2: Pull complete e2ead8259a04: Pull complete 3790aef225b9: Pull complete Digest: sha256:169211e20e2f2d5d115674681eb79d21a217b296b43374b8e39f97fcf866b375 Status: Downloaded newer image for registry:2.7.1 docker.io/library/registry:2.7.1 [root@localhost docker]# docker images REPOSITORYTAGIMAGE IDCREATEDSIZE registry2.7.1b8604a3fe8543 months ago26.2MB

1、创建单机仓库
1.1 创建账号启动docker registry容器

[root@localhost docker]# yum -y install httpd [root@localhost docker]# htpasswd -Bbn test 123456 > /etc/docker/auth/registry [root@localhost docker]# cat /etc/docker/auth/registry test:$2y$05$ecpX/anNOrNRe7xgenNZVOqmA1DWlb8e1AAUWD2LGFrV5pFnBtMs2[root@localhost docker]# docker run -d -p 5000:5000 --restart=always --name registry -v /etc/docker/auth:/auth -e "REGISTRY_AUTH=htpasswd" -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/registry registry:2.7.1 a218fcf93146d0d2cf9888fc8bd65ef6a753ddb985996acf3a70487c469c037d[root@localhost docker]# docker ps CONTAINER IDIMAGECOMMANDCREATEDSTATUSPORTSNAMES a218fcf93146registry:2.7.1"/entrypoint.sh /etc…"About a minute agoUp About a minute0.0.0.0:5000-> 5000/tcpregistry [root@localhost docker]# ss -tln StateRecv-Q Send-QLocal Address:PortPeer Address:Port LISTEN0128*:5000*:* LISTEN0128*:22*:* LISTEN0128[::]:22[::]:*

1.2 登录仓库

#docker login 默认使用https登录,而docker registry为http,所以默认登录失败 [root@localhost docker]# docker login 192.168.187.10:5000 Username: test Password: Error response from daemon: Get "https://192.168.187.10:5000/v2/": http: server gave HTTP response to HTTPS client

将registry仓库地址加入到service单元文件

[root@localhost docker]# cat /etc/docker/daemon.json "registry-mirrors": ["https://boqr6s5g.mirror.aliyuncs.com"], "insecure-registry": ["192.168.187.10:5000"]#或者[root@localhost docker]# cat/usr/lib/systemd/system/docker.service [Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network-online.target firewalld.service containerd.service Wants=network-online.target Requires=docker.socket containerd.service[Service] Type=notify # the default is not to use systemd for cgroups because the delegate issues still # exists and systemd currently does not support the cgroup feature set required # for containers run by docker ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --insecure-registry 192.168.187.10:5000[root@localhost docker]#systemctl daemon-reload [root@localhost docker]#systemctl restart docker

[root@localhost docker]# docker login 192.168.187.10:5000 Username: test Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded

1.3 上传下载镜像

#需要先登录（dcoker login） [root@dockerserver2 ~]# docker tag centos:centos7.8.2003 192.168.187.10:5000/centos7:v1 [root@dockerserver2 ~]# docker push 192.168.187.10:5000/centos7:v1 The push refers to repository [192.168.187.10:5000/centos7] fb82b029bea0: Pushed v1: digest: sha256:50b9a3bc27378889210f88d6d0695938e45a912aa99b3fdacfb9a0fef511f15a size: 529#下载镜像 [root@dockerserver2 ~]# docker pull 192.168.187.10:5000/centos7:v1 v1: Pulling from centos7 9b4ebb48de8d: Pull complete Digest: sha256:50b9a3bc27378889210f88d6d0695938e45a912aa99b3fdacfb9a0fef511f15a Status: Downloaded newer image for 192.168.187.10:5000/centos7:v1 192.168.187.10:5000/centos7:v1 [root@dockerserver2 ~]# docker images REPOSITORYTAGIMAGE IDCREATEDSIZE 192.168.187.10:5000/centos7v1afb6fca791e021 months ago203MB [root@dockerserver2 ~]# docker run -it --rm 192.168.187.10:5000/centos7:v1 [root@14fd3b4ad4b8 /]# cat /etc/redhat-release CentOS Linux release 7.8.2003 (Core)

二、Docker分布式仓库Harbor 1、 Harhor
Harbor是一个用于存储和分发Docker镜像的企业级Registry服务器，由VMware开源，其通过添加一些企业必需的功能特性，例如安全、标识和管理等，扩展了开源 Docker Distribution。作为一个企业级私有Registry服务器，Harbor 提供了更好的性能和安全。提升用户使用Registry构建和运行环境传输镜像的效率。Harbor支持安装在多个Registry节点的镜像资源复制，镜像全部保存在私有 Registry 中，确保数据和知识产权在公司内部网络中管控，另外，Harbor也提供了高级的安全特性，诸如用户管理，访问控制和活动审计等
vmware 官方开源服务: https://vmware.github.io/
【Docker仓库之Harbor】harbor 官方github 地址: https://github.com/vmware/harbor
harbor 官方网址: https://goharbor.io/
harbor 官方文档: https://goharbor.io/docs/
github文档: https://github.com/goharbor/harbor/tree/master/docs
官方功能介绍

基于角色的访问控制: 用户与Docker镜像仓库通过“项目”进行组织管理，一个用户可以对多个镜像仓库在同一命名空间（project）里有不同的权限
镜像复制: 镜像可在多个Registry实例中复制（同步）。尤其适合于负载均衡，高可用，混合云和多云的场景
图形化用户界面: 用户可以通过浏览器来浏览，检索当前Docker镜像仓库，管理项目和命名空间
AD/LDAP 支: Harbor可以集成企业内部已有的AD/LDAP，用于鉴权认证管理
审计管理: 所有针对镜像仓库的操作都可以被记录追溯，用于审计管理
国际化: 已拥有英文、中文、德文、日文和俄文的本地化版本。更多的语言将会添加进来
RESTful API: 提供给管理员对于Harbor更多的操控, 使得与其它管理软件集成变得更容易
部署简单: 提供在线和离线两种安装工具，也可以安装到vSphere平台(OVA方式)虚拟设备

Harbor组成

文章图片

Proxy: 对应启动组件nginx。它是一个nginx反向代理，代理Notary client（镜像认证）、Dockerclient（镜像上传下载等）和浏览器的访问请求（Core Service）给后端的各服务
UI（Core Service）: 对应启动组件harbor-ui。底层数据存储使用mysql数据库，主要提供了四个
子功能:
- UI: 一个web管理页面ui
- API: Harbor暴露的API服务
- Auth: 用户认证服务，decode后的token中的用户信息在这里进行认证；auth后端可以接db、ldap、uaa三种认证实现
- Token服务（上图中未体现）: 负责根据用户在每个project中的role来为每一个dockerpush/pull命令发布一个token，如果从docker client发送给registry的请求没有带token，registry会重定向请求到token服务创建token
Registry: 对应启动组件registry。负责存储镜像文件，和处理镜像的pull/push命令。Harbor对镜像进行强制的访问控制，Registry会将客户端的每个pull、push请求转发到token服务来获取有效的token
Admin Service: 对应启动组件harbor-adminserver。是系统的配置管理中心附带检查存储用量，
ui和jobserver启动时候需要加载adminserver的配置
Job Sevice: 对应启动组件harbor-jobservice。负责镜像复制工作的，他和registry通信，从一个registry pull镜像然后push到另一个registry，并记录job_log
Log Collector: 对应启动组件harbor-log。日志汇总组件，通过docker的log-driver把日志汇总到
一起
DB: 对应启动组件harbor-db，负责存储project、 user、 role、replication、image_scan、
access等的metadata数据

2、安装Harbor
下载地址: https://github.com/vmware/harbor/releases
安装文档: https://github.com/goharbor/harbor/blob/master/docs/install-config/_index.md

安装docker ---- 滤过
安装docker compose

[root@dockerserver2 ~]# curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose % Total% Received % XferdAverage SpeedTimeTimeTimeCurrent DloadUploadTotalSpentLeftSpeed 100664100664009010 --:--:-- --:--:-- --:--:--900 100 12.1M100 12.1M004631k00:00:020:00:02 --:--:--9.7M [root@dockerserver2 ~]# [root@dockerserver2 ~]# ll /usr/local/bin/docker-compose -rw-r--r--. 1 root root 12737304 Feb 18 08:32 /usr/local/bin/docker-compose [root@dockerserver2 ~]# chmod +x/usr/local/bin/docker-compose [root@dockerserver2 ~]# ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose [root@dockerserver2 ~]# docker-compose version docker-compose version 1.29.2, build 5becea4c docker-py version: 5.0.0 CPython version: 3.7.10 OpenSSL version: OpenSSL 1.1.0l10 Sep 2019

安装Harbor

以下使用 harbor 稳定版本v1.10.10安装包
下载离线完整安装包,推荐使用

[root@dockerserver2 ~]#wget https://github.com/goharbor/harbor/releases/download/v1.10.10/harbor-offline-installer-v1.10.10.tgz [root@dockerserver2 src]# mkdir /apps [root@dockerserver2 src]# tar -zxf harbor-offline-installer-v1.10.10.tgz

编辑harbor配置文件

最新文档: https://github.com/goharbor/harbor/blob/master/docs/install-config/configure-yml-file.
md

[root@dockerserver2 ~]#vim /apps/harbor/harbor.cfg #只需要修改下面两行 hostname = 192.168.187.11 #修改此行,指向当前主机IP 或 FQDN harbor_admin_password = 123456 #修改此行指定harbor登录用户admin的,默认用户:admin/Harbor12345 #可选项 ui_url_protocol = http #默认即可,如果修改为https,需要指定下面证书路径 ssl_cert = /data/cert/server.crt #默认即可,https时，需指定下面证书文件路径 ss_cert_key = /data/cert/server.key #默认即可,https时，需指定下面私钥文件路径

运行安装脚本

[root@dockerserver2 harbor]# yum -y install python [root@dockerserver2 harbor]# ./install.sh [Step 0]: checking if docker is installed ...Note: docker version: 20.10.12[Step 1]: checking docker-compose is installed ...Note: docker-compose version: 1.29.2[Step 2]: loading Harbor images ... [Step 3]: preparing environment ... [Step 4]: preparing harbor configs ... prepare base dir is set to /apps/harbor /usr/src/app/utils/configs.py:100: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. configs = yaml.load(f) WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https /usr/src/app/utils/configs.py:90: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. versions = yaml.load(f) Generated configuration file: /config/log/logrotate.conf Generated configuration file: /config/log/rsyslog_docker.conf Generated configuration file: /config/nginx/nginx.conf Generated configuration file: /config/core/env Generated configuration file: /config/core/app.conf Generated configuration file: /config/registry/config.yml Generated configuration file: /config/registryctl/env Generated configuration file: /config/db/env Generated configuration file: /config/jobservice/env Generated configuration file: /config/jobservice/config.yml Generated and saved secret to file: /secret/keys/secretkey Generated certificate, key file: /secret/core/private_key.pem, cert file: /secret/registry/root.crt Generated configuration file: /compose_location/docker-compose.yml Clean up the input dir[Step 5]: starting Harbor ... Creating network "harbor_harbor" with the default driver Creating harbor-log ... done Creating registryctl... done Creating redis... done Creating registry... done Creating harbor-db... done Creating harbor-portal ... done Creating harbor-core... done Creating harbor-jobservice ... done Creating nginx... done ? ----Harbor has been installed and started successfully.----#安装harbor后会自动开启很多相关容器 [root@dockerserver2 harbor]# docker ps CONTAINER IDIMAGECOMMANDCREATEDSTATUSPORTSNAMES 1834217183fbgoharbor/nginx-photon:v1.10.10"nginx -g daemon of…"About a minute agoUp About a minute (healthy)0.0.0.0:80-> 8080/tcpnginx a726b631c455goharbor/harbor-jobservice:v1.10.10"/harbor/harbor_jobs…"About a minute agoUp About a minute (healthy)harbor-jobservice d93cd0e5c4c0goharbor/harbor-core:v1.10.10"/harbor/harbor_core"About a minute agoUp About a minute (healthy)harbor-core 62f4feeea29fgoharbor/harbor-portal:v1.10.10"nginx -g daemon of…"About a minute agoUp About a minute (healthy)8080/tcpharbor-portal 7b4100dfc0begoharbor/harbor-db:v1.10.10"/docker-entrypoint.…"About a minute agoUp About a minute (healthy)5432/tcpharbor-db 9ede3db336cbgoharbor/registry-photon:v1.10.10"/home/harbor/entryp…"About a minute agoUp About a minute (healthy)5000/tcpregistry 0a86c46beb82goharbor/redis-photon:v1.10.10"redis-server /etc/r…"About a minute agoUp About a minute (healthy)6379/tcpredis d8cc10dda240goharbor/harbor-registryctl:v1.10.10"/home/harbor/start.…"About a minute agoUp About a minute (healthy)registryctl 3a1e7e0eb2bcgoharbor/harbor-log:v1.10.10"/bin/sh -c /usr/loc…"About a minute agoUp About a minute (healthy)127.0.0.1:1514-> 10514/tcpharbor-log

开机启动harbor

[root@dockerserver2 harbor]# vim /lib/systemd/system/harbor.service [Unit] Description=Harbor After=docker.service systemd-networkd.service systemd-resolved.service Requires=docker.service Documentation=http://github.com/vmware/harbor[Service] Type=simple Restart=on-failure RestartSec=5 ExecStart=/usr/bin/docker-compose -f /apps/harbor/docker-compose.yml up ExecStop=/usr/bin/docker-compose -f /apps/harbor/docker-compose.yml down [Install] WantedBy=multi-user.target[root@dockerserver2 harbor]# systemctl daemon-reload [root@dockerserver2 harbor]# systemctl enable harbor Created symlink from /etc/systemd/system/multi-user.target.wants/harbor.service to /usr/lib/systemd/system/harbor.service.

3、使用Harbor
访问：http://192.168.187.11/

账号：admin/123456

文章图片

使用单机harbor
- 建立项目

harbor上必须先建立项目，才能上传镜像

文章图片

登录harbor上传镜像

[root@ubuntu1804 ~]#vim /lib/systemd/system/docker.service ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --insecure-registry 10.0.0.101 --insecure-registry 192.168.187.11[root@dockerserver2 src]# systemctl daemon-reload [root@dockerserver2 src]# systemctl restart docker [root@dockerserver2 src]# docker login 192.168.187.11 Username: admin Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded[root@dockerserver2 ~]# cat .docker/config.json"auths": "192.168.187.10:5000": "auth": "dGVzdDoxMjM0NTY=" , "192.168.187.11": "auth": "YWRtaW46MTIzNDU2"#上传镜像 #修改 images 的名称，不修改成指定格式无法将镜像上传到 harbor 仓库 #格式： Harbor主机IP/项目名/image名字:版本[root@dockerserver2 ~]# docker images REPOSITORYTAGIMAGE IDCREATEDSIZE ubuntu20.04ba6acccedd294 months ago72.8MB 192.168.187.10:5000/centos7v1afb6fca791e021 months ago203MB [root@dockerserver2 ~]# docker tag ubuntu:20.04 192.168.187.11/test/ubuntu-base:v1 [root@dockerserver2 ~]# docker push 192.168.187.11/test/ubuntu-base:v1 The push refers to repository [192.168.187.11/test/ubuntu-base] 9f54eef41275: Pushed v1: digest: sha256:7cc0576c7c0ec2384de5cbf245f41567e922aab1b075f3e8ad565f508032df17 size: 529

验证是否成功上传

文章图片

注意：如果不事先建立项目，上传镜像会失败

可查看日志记录

文章图片

下载镜像

下载前必须修改docker的service 文件，加入harbor服务器的地址才可以下载

[root@ubuntu1804 ~]#vim /lib/systemd/system/docker.service ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --insecure-registry 10.0.0.101 --insecure-registry 192.168.187.11[root@dockerserver2 src]# systemctl daemon-reload [root@dockerserver2 src]# systemctl restart docker [root@dockerserver1 src]# docker images REPOSITORYTAGIMAGE IDCREATEDSIZE registry2.7.1b8604a3fe8543 months ago26.2MB [root@dockerserver1 src]# docker pull 192.168.187.11/test/ubuntu-base:v1 v1: Pulling from test/ubuntu-base 7b1a6ab2e44d: Pull complete Digest: sha256:7cc0576c7c0ec2384de5cbf245f41567e922aab1b075f3e8ad565f508032df17 Status: Downloaded newer image for 192.168.187.11/test/ubuntu-base:v1 192.168.187.11/test/ubuntu-base:v1 [root@dockerserver1 src]# docker images REPOSITORYTAGIMAGE IDCREATEDSIZE registry2.7.1b8604a3fe8543 months ago26.2MB 192.168.187.11/test/ubuntu-basev1ba6acccedd294 months ago72.8MB

创建自动上传镜像脚本

[root@ubuntu1804 ~]#cd /data/dockerfile/web/nginx/1.16.1-alpine/ [root@ubuntu1804 1.16.1-alpine]#vim build.sh [root@ubuntu1804 1.16.1-alpine]#cat build.sh #!/bin/bash TAG=$1 docker build -t 10.0.0.101/example/nginx-alpine:1.16.1-$TAG . docker push 10.0.0.101/example/nginx-alpine:1.16.1-$TAG docker rmi -f 10.0.0.101/example/nginx-alpine:1.16.1-$TAG [root@ubuntu1804 1.16.1-alpine]#bash build.sh v1

修改harbor配置

#后期如果修改harbor配置，比如: 修改IP地址等，可执行以下步骤生效 [root@ubuntu1804 ~]#cd /apps/harbor/ [root@ubuntu1804 harbor]#docker-compose stop Stopping nginx ... done Stopping harbor-portal ... done Stopping harbor-jobservice ... done Stopping harbor-core ... done Stopping harbor-adminserver ... done Stopping harbor-db ... done Stopping registryctl ... done Stopping registry ... done Stopping redis ... done Stopping harbor-log ...#修改harbor配置 [root@dockerserver2 harbor]# vim /apps/harbor/harbor.yml#更新配置 [root@dockerserver2 harbor]#/apps/harbor/prepare#重新启动docker compose [root@dockerserver2 harbor]#docker-compose start

4、Harbor 高可用
Harbor支持基于策略的Docker镜像复制功能，这类似于MySQL的主从同步，其可以实现不同的数据中心、不同的运行环境之间同步镜像，并提供友好的管理界面，大大简化了实际运维中的镜像管理工作，已经有用很多互联网公司使用harbor搭建内网docker仓库的案例，并且还有实现了双向复制功能

安装第二台harbor主机
注意：harbor.cfg中配置 hostname = 192.168.187.12
创建相同的项目

文章图片

参考第一台主机信息,新建复制（同步）目标信息,将第一台主机设为复制的目标

文章图片

输入第一台主机信息

文章图片

第二台harbor上新建复制规则实现到第一台harbor的单向复制

文章图片

在第一台harbor主机上重复上面操作，在第一台harbor上再执行下面操作，才实现双向同步
确认同步成功

文章图片

注意：设置好同步规则后，已有镜像不会进行同步
5、Harbor https 配置

#安装docker步骤省略#生成私钥和证书 [root@dockerserver2 harbor]#touch /root/.rnd [root@dockerserver2 harbor]#mkdir /apps/harbor/certs/ [root@dockerserver2 harbor]#cd /apps/harbor/certs/#生成CA证书 [root@dockerserver2 harbor]#openssl req -newkey rsa:4096 -nodes -sha256 -keyout ca.key -x509 -subj "/CN=ca.org" -days 365 -out ca.crt#生成harbor主机的证书申请 [root@dockerserver2 harbor]#openssl req -newkey rsa:4096 -nodes -sha256 -subj "/CN=harbor.magedu.org" -keyout harbor.org.key -out harbo.org.csr#给harbor主机颁发证书 [root@dockerserver2 harbor]#openssl x509 -req -in harbor.org.csr -CA ca.crt - CAkey ca.key -CAcreateserial -out harbor.org.crt[root@dockerserver2 harbor]#tree /apps/harbor/certs /apps/harbor/certs ├── ca.crt ├── ca.key ├── ca.srl ├── harbor.org.crt ├── harbor.org.csr └── harbor.org.key 0 directories, 6 files [root@dockerserver2 harbor]#vim /apps/harbor/harbor.cfg hostname = harbor.magedu.org ui_url_protocol = https ssl_cert = /apps/harbor/certs/harbor.org.crt ssl_cert_key = /apps/harbor/certs/harbor.org.key harbor_admin_password = 123456 [root@dockerserver2 harbor]#apt -y install python [root@dockerserver2 harbor]#/apps/harbor/install.sh