版本:v4.24
1.4 组件与安全说明
| 标识符 | 说明 |
|---|---|
| ✔️ | 支持 |
| ⚠️ | 有限制支持 |
| 🔚 | 停止支持 |
组件第三方版本情况
| 组件名称 | 组件版本 | DP v3.7 | DP V4.15 | DP V4.17 | DP V4.18 | DP V4.19 | DP V4.20 | DP V4.21 | DP V4.22 | DP V4.23 | DP V4.24 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| MySQL配置库 | 8.0.34 | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
| 8.0.36 | ⚠️ 麒麟操作系统不支持MGR | ⚠️ 麒麟操作系统不支持MGR | ⚠️ 麒麟操作系统不支持MGR | ⚠️ 麒麟操作系统不支持MGR | ⚠️ 麒麟操作系统不支持MGR | ⚠️ 麒麟操作系统不支持MGR | ⚠️ 麒麟操作系统不支持MGR | ⚠️ 麒麟操作系统不支持MGR | ⚠️ 麒麟操作系统不支持MGR | ⚠️ 麒麟操作系统不支持MGR | |
| Oceanbase配置库 | V3.2.4.1 | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
| V4.0.0 | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | |
| 达梦配置库 | V8 | ✔️ | ✔️ | ✔️ | |||||||
| KingbaseES配置库 | V8 | ✔️ | ✔️ | ✔️ | |||||||
| PostGreSQL 配置库 | 13.x | ✔️ | |||||||||
| GoldenDB 配置库 | ✔️(不支持迁移) | ||||||||||
| Kafka | 2.7.3.1 | ✔️ | 🔚 | ||||||||
| 3.8.1 | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ||
| Redis | 6.2 | ✔️ | ✔️ | 🔚 | 🔚 | 🔚 | 🔚 | 🔚 | 🔚 | 🔚 | 🔚 |
| 7.4.2 | ✔️ | ✔️ ⚠️ 当Arm操作系统内核版本为4.18.0-193.28.1.el7.aarch64时,不支持该版本 | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | 🔚 | |||
| 7.4.5 | ✔️ | ||||||||||
| Zookeeper | 3.6.4.1 | ✔️ | 🔚 | ||||||||
| 3.9.3 | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ||
| Schema_registry | 5.3.8.2 | ✔️ | 🔚 | ||||||||
| 7.7.1 | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
配置库权限说明
| 配置库类型 | 权限说明(最小权限) |
|---|---|
| MySQL配置库 | -- ``1``. 创建用户CREATE USER ``'dp_test'``@``'%' IDENTIFIED BY ``'xxx'``;-- ``2``. 对三个库分别授权GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, INDEX, EXECUTE ``ONdp-thrall.* TO ``'dp_test'``@``'%'``;GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, INDEX, EXECUTE ``ON coordinator.* TO ``'dp_test'``@``'%'``;GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, INDEX, EXECUTE ``ON connectdp.* TO ``'dp_test'``@``'%'``; |
| OceanBase配置库 | OceanBase - MySQL租户-- ``1``. 创建用户CREATE USER ``'dp_user'``@``'%' IDENTIFIED BY ``'xxx'``;-- ``2``. 对三个库分别授权GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, INDEX, EXECUTE ``ON dpthrall.* TO ``'dp_user'``@``'%'``;GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, INDEX, EXECUTE ``ON coordinator.* TO ``'dp_user'``@``'%'``;GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, INDEX, EXECUTE ``ON connectdp.* TO ``'dp_user'``@``'%'``;EXECUTE权限只能进行对象级赋权,不能进行库级赋权 OceanBase - Oracle租户 -- ``1``创建连接配置库用户(密码按需替换)CREATE USER ``"dp_test" IDENTIFIED BY ``123456``;-- ``2``创建配置库用户(密码按需替换)需创建两个 OceanBase-Oracle 用户:webservice、coordinator CREATE USER ``"webservice" IDENTIFIED BY ``123456``;CREATE USER ``"coordinator" IDENTIFIED BY ``123456``;-- ``3``分别对两个配置库用户进行授权GRANT CREATE ANY TABLE, ALTER ANY TABLE, CREATE ANY INDEX, CREATE ANY SEQUENCE,SELECT ANY TABLE, INSERT ANY TABLE, UPDATE ANY TABLE, DELETE ANY TABLE TO webservice/coordinator;GRANT CREATE SESSION TO webservice/coordinator;-- ``4``对连接配置库用户进行授权DECLARE ``v_sql VARCHAR2(``200``);BEGIN ``-- 遍历 SUMMER_WEB 模式下的所有序列 ``FOR seq_rec IN (SELECT SEQUENCE_NAME FROM ALL_SEQUENCES WHERE SEQUENCE_OWNER = ``'SUMMER_WEB'``) LOOP ``-- 构建 GRANT 语句 ``v_sql := ``'GRANT SELECT ON SUMMER_WEB.' ` |
| kingbase配置库 | -- ``1``. 创建用户并给用户复制槽权限CREATE USER dp_test WITH PASSWORD ``'123456'``;ALTER USER dp_test WITH REPLICATION;-- ``2``. 对coordinator.env,connectdp.env,manager.env,web.env四个配置文件中配置的数据库和schema进行赋权(需要在所有数据库和schema中执行下列语句)-- 授予数据库连接,创建权限,举例:GRANT CONNECT,CREATE ON DATABASE dp_test1 TO dp_test;GRANT CONNECT,CREATE ON DATABASE dp_test2 TO dp_test;-- 授予schema基础操作权限,举例GRANT USAGE, CREATE ON SCHEMA dp_test1 TO dp_test;-- 授予表级数据操作权限,举例GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA dp_test1 TO dp_test; |
| 达梦配置库 | -- ``1``. 创建用户(达梦用户即 schema)CREATE USER ``"DP_USER" IDENTIFIED BY ``"YourStrongPwd_123"-- ``2``. 对三个 schema 授予对象权限BEGIN ``FOR s IN (``'DPTHRALL'``,``'COORDINATOR'``,``'CONNECTDP'``) LOOP ``EXECUTE IMMEDIATE ``'GRANT SELECT, INSERT, UPDATE, DELETE, CREATE TABLE, ALTER, INDEX, EXECUTE ON SCHEMA ' ` |
| PostgreSQL | 使用新的用户来初始化-- ``1``. 创建用户CREATE USER dp_test WITH PASSWORD ``'Datapipeline123'``;-- ``2``. 给用户在postgres数据库创建schema的权限GRANT CREATE ON DATABASE postgres TO dp_test;切换pg用户-- ``1``. 创建用户CREATE USER dp_test WITH PASSWORD ``'Datapipeline123'``;-- ``2``. 授予 schema 权限GRANT USAGE, CREATE ON SCHEMA ``"dp-thrall"``, coordinator, connectdp TO dp_test;-- ``3``. 授予表权限GRANT SELECT, INSERT, UPDATE, DELETE, REFERENCES ON ALL TABLES IN SCHEMA ``"dp-thrall" TO dp_test;GRANT SELECT, INSERT, UPDATE, DELETE, REFERENCES ON ALL TABLES IN SCHEMA coordinator TO dp_test;GRANT SELECT, INSERT, UPDATE, DELETE, REFERENCES ON ALL TABLES IN SCHEMA connectdp TO dp_test;-- ``4``. 设置默认权限ALTER DEFAULT PRIVILEGES IN SCHEMA ``"dp-thrall" GRANT SELECT, INSERT, UPDATE, DELETE, REFERENCES ON TABLES TO dp_test;ALTER DEFAULT PRIVILEGES IN SCHEMA coordinator GRANT SELECT, INSERT, UPDATE, DELETE, REFERENCES ON TABLES TO dp_test;ALTER DEFAULT PRIVILEGES IN SCHEMA connectdp GRANT SELECT, INSERT, UPDATE, DELETE, REFERENCES ON TABLES TO dp_test; |
Zookeeper、Kafka组件开启Plain认证说明
从4.24.6版本开始,我们支持内部组件Zookeeper和Kafka,或者部分支持外部Kafka配置的节点,开启Plain认证。同时对所有支持节点高可用、Agent读取、Agent高可用的功能进行适配,保证在开启Plain后数据节点功能使用正常。
同时需要注意,因为 IBM 本身的限制,本次 DB2 以及 AS400 仍无法在开启 Plain 认证的环境使用部分功能。
适配范围
| 节点高可用 | 内部Kafka/ZK | 外部Kafka/ZK | 当前是否已支持plain认证 |
|---|---|---|---|
| oracle | ✅ | ❌ | ✅ |
| OB2(mysql/oracle) | ✅ | ❌ | ❓ |
| OB3(mysql/oracle) | ✅ | ❌ | ✅ |
| OB4(mysql/oracle) | ✅ | ❌ | ✅ |
| mysql | ✅ | ❌ | ✅ |
| newmysql | ✅ | ❌ | ✅ |
| dameng | ✅ | ❌ | ✅ |
| tdsql mysql(集中式/分布式) | ✅ | ❌ | ✅ |
| Agent高可用 | 内部Kafka/ZK | 外部Kafka/ZK | 是否已支持plain认证 |
|---|---|---|---|
| oracle agent | agent环境变量 | ✅ | |
| ob agent | agent环境变量 | ✅ |
| Agent节点 | 内部Kafka/ZK | 外部Kafka/ZK | 外部kafka支持的认证方式 | 是否已支持plain认证 |
|---|---|---|---|---|
| oracle agent | ✅ | ✅ | 信任认证、 口令认证(plain)、 scram认证、 kerberos认证 | ✅ |
| ob agent | ✅ | ❌ | ✅ | |
| tidb | ✅ 只配置kafka,不配置zk | ✅ 只配置kafka,不配置zk | 本次支持plain | ✅ |
| db2 | ✅ | ✅ | 不支持 | ❌不支持 受限于IBM CDC的API本身,无法支持认证。API 不具备重载方法可以设置Plain认证。 当DP的kafka开启plain认证,DB2设置另外不开启plain认证的kafka集群时,保证任务可以启动,一切功能正常(**❗️**当前不可用) |
| as400 | ✅ | ✅ | 不支持 |