Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix Linkis installation directories #363

Merged
merged 3 commits into from
Jun 22, 2022
Merged

Fix Linkis installation directories #363

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2bf82d5
Fix represent Linkis installation directories
Beacontownfc Jun 20, 2022
8d7eda4
resolve conflict
Beacontownfc Jun 22, 2022
1ccd8a2
Merge branch 'dev' into dir
Beacontownfc Jun 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions blog/2022-02-21-linkis-deploy.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,7 @@ If hive is not version 2.3.3, you need to modify the parameters:
##HIVE_VERSION, If the installed Hive version is not 2.3.3, it needs to be modified to the corresponding version, such as 2.3.4
HIVE_VERSION=2.3.4
```

If configured, it will actually be updated in the `{linkisInstallPath}/conf/linkis.properties` file after the installation and deployment are performed
```shell script
#wds.linkis.spark.engine.version=
Expand Down Expand Up @@ -264,6 +265,7 @@ Your default account password is \[hadoop/5e8e312b4]
### 3.3 Configuration modification
After the installation is complete, if you need to modify the configuration, you can re-execute the installation, or modify the corresponding ${InstallPath}/conf/*properties file and restart the corresponding service


### 3.4 Add mysql driver (>=1.0.3) version
Because of the license, mysql-connector-java is removed from the release package of linkis itself (the family bucket integrated by dss will be included, no need to manually add it), which needs to be added manually.
For details, see [Add mysql driver package](docs/1.0.3/deployment/quick_deploy#-44-Add mysql driver package)
Expand Down Expand Up @@ -362,6 +364,7 @@ Log in to the web terminal to view information
http://xx.xx.xx.xx:8188/#/login
Username/Password (check in {InstallPath}/conf/linkis-mg-gateway.properties)


```shell script
#When not using LDAP configuration
wds.linkis.admin.user= #User
Expand Down
2 changes: 1 addition & 1 deletion docs/development/linkis_debug.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ nohup java -DserviceName=linkis-cg-engineplugin -Xmx512M -XX:+UseG1GC -Xloggc:/d
#### 1.1 specify the service of the package to be debugged
Determine the service it belongs to according to the location of the code to be debugged

#### 1.2 enter {linkis_home}/sbin/ext, modify the module configuration file, and open the remote call port
#### 1.2 enter {LINKIS_HOME}/sbin/ext, modify the module configuration file, and open the remote call port
![e-port](https://user-images.githubusercontent.com/29391030/167366305-207da78b-48da-4eee-af40-947bd43050b9.png)

### 2. Restart the service to be debugged
Expand Down
2 changes: 1 addition & 1 deletion i18n/zh-CN/docusaurus-plugin-content-docs/current/deployment/quick_deploy.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -598,7 +598,7 @@ sh -x engineConnExec.sh
```
2.执行安装之后修改注册中心eureka端口
```
1. 进入${linkis_home}/conf目录
1. 进入${LINKIS_HOME}/conf目录

2. 执行grep -r 20303 ./* ,查询结果如下所示:
./application-eureka.yml: port: 20303
Expand Down
4 changes: 2 additions & 2 deletions i18n/zh-CN/docusaurus-plugin-content-docs/current/development/linkis_debug.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,11 +206,11 @@ nohup java -DserviceName=linkis-cg-engineplugin -Xmx512M -XX:+UseG1GC -Xloggc:/
#### 1.1 明确需要调试的包所在的服务
根据需要调试的代码位置,确定其所属的服务

#### 1.2 进入{linkis_home}/sbin/ext,修改模块配置文件开启远程调用端口
#### 1.2 进入{LINKIS_HOME}/sbin/ext,修改模块配置文件开启远程调用端口
![c-port](https://user-images.githubusercontent.com/29391030/167364775-4f5d2774-b6b9-4a65-b69c-69319db870c4.png)

### 2. 重启需要调试的服务
sh linkis-daemon.sh restart ps-publicservice(如果不确定服务名称,在 {linkis_home}/sbin/linkis-start-all.sh 内查询)
sh linkis-daemon.sh restart ps-publicservice(如果不确定服务名称,在 {LINKIS_HOME}/sbin/linkis-start-all.sh 内查询)

### 3. 编译器配置远程调试
如下图所示打开窗口并配置远程调试的端口,服务,以及模块
Expand Down
12 changes: 2 additions & 10 deletions i18n/zh-CN/docusaurus-plugin-content-docs/version-1.1.1/deployment/quick_deploy.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,6 @@ export ENABLE_SPARK=false
```bash
sh bin/install.sh
```

install.sh脚本会询问您是否需要初始化数据库并导入元数据。如果选择初始化，会把数据库中的表数据清空重新初始化。

<font color="red"> ** 第一次安装 **必须选清空数据库</font>
Expand Down Expand Up @@ -252,14 +251,11 @@ cp mysql-connector-java-5.1.49.jar {LINKIS_HOME}/lib/linkis-commons/public-modu
echo "wds.linkis.keytab.enable=true" >> linkis.properties
```
#### 3.3.2 Yarn的认证

执行spark任务时，需要使用到yarn的ResourceManager，通过配置项`YARN_RESTFUL_URL=http://xx.xx.xx.xx:8088 `控制。
执行安装部署时，会将`YARN_RESTFUL_URL=http://xx.xx.xx.xx:8088` 信息更新到数据库表中 `linkis_cg_rm_external_resource_provider`中时候，默认访问yarn资源是不需权限验证的，
如果yarn的ResourceManager开启了密码权限验证，请安装部署后，修改数据库表 `linkis_cg_rm_external_resource_provider` 中生成的yarn数据信息,
详细可以参考[查看yarn地址是否配置正确](#811-查看yarn地址是否配置正确)



#### 3.3.2 session
如果您是对Linkis的升级。同时部署DSS或者其他项目，但其它软件中引入的依赖linkis版本<1.1.1(主要看lib包中，所依赖的Linkis的linkis-module-x.x.x.jar包 <1.1.1），则需要修改位于`${LINKIS_HOME}/conf/linkis.properties`文件
```shell
Expand Down Expand Up @@ -539,15 +535,13 @@ hdfs dfs -chown hadoop:hadoop /apps-data
```

### 8.3 登陆密码问题

linkis默认是使用静态用户和密码,静态用户即部署用户，静态密码会在执行部署是随机生成一个密码串，存储于
`{LINKIS_HOME}/conf/linkis-mg-gateway.properties`(>=1.0.3版本)

### 8.4 版本兼容性问题

linkis默认支持的引擎，与dss兼容关系可以查看[此文档](https://github.com/apache/incubator-linkis/blob/master/README.md)


### 8.5 如何定位服务端异常日志

linkis的微服务比较多，若对系统不熟悉，有时候无法定位到具体哪个模块出现了异常，可以通过全局日志搜索方式
Expand Down Expand Up @@ -586,6 +580,7 @@ less logs/stdout
可以通过尝试手动执行脚本，进行调试
```
sh -x engineConnExec.sh

```

### 8.7 如何修改注册中心eureka的端口
Expand All @@ -610,7 +605,6 @@ sh -x engineConnExec.sh
3. 将对应位置的端口修改为新的端口,并且重启所有服务sh restart sbin/linkis-start-all.sh
```


### 8.8 CDH适配版本的注意事项

CDH本身不是使用的官方标准的hive/spark包,进行适配时，最好修改linkis的源码中的hive/spark版本的依赖，进行重新编译部署。
Expand Down Expand Up @@ -655,6 +649,4 @@ linkis官网文档正在不断的完善,可以在本官网查看/关键字搜索
- Linkis的技术博文集 https://github.com/apache/incubator-linkis/issues/1233
- 公众号技术博文https://mp.weixin.qq.com/mp/homepage?__biz=MzI4MDkxNzUxMg==&hid=1&sn=088cbf2bbed1c80d003c5865bc92ace8&scene=18
- 官网文档 https://linkis.apache.org/zh-CN/docs/latest/introduction
- bili技术分享视频 https://space.bilibili.com/598542776?spm_id_from=333.788.b_765f7570696e666f.2


- bili技术分享视频 https://space.bilibili.com/598542776?spm_id_from=333.788.b_765f7570696e666f.2
4 changes: 2 additions & 2 deletions .../zh-CN/docusaurus-plugin-content-docs/version-1.1.1/development/linkis_debug.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,11 +206,11 @@ nohup java -DserviceName=linkis-cg-engineplugin -Xmx512M -XX:+UseG1GC -Xloggc:/
#### 1.1 明确需要调试的包所在的服务
根据需要调试的代码位置,确定其所属的服务

#### 1.2 进入{linkis_home}/sbin/ext,修改模块配置文件开启远程调用端口
#### 1.2 进入{LINKIS_HOME}/sbin/ext,修改模块配置文件开启远程调用端口
![c-port](https://user-images.githubusercontent.com/29391030/167364775-4f5d2774-b6b9-4a65-b69c-69319db870c4.png)

### 2. 重启需要调试的服务
sh linkis-daemon.sh restart ps-publicservice(如果不确定服务名称,在 {linkis_home}/sbin/linkis-start-all.sh 内查询)
sh linkis-daemon.sh restart ps-publicservice(如果不确定服务名称,在 {LINKIS_HOME}/sbin/linkis-start-all.sh 内查询)

### 3. 编译器配置远程调试
如下图所示打开窗口并配置远程调试的端口,服务,以及模块
Expand Down
2 changes: 1 addition & 1 deletion versioned_docs/version-1.1.1/development/linkis_debug.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ nohup java -DserviceName=linkis-cg-engineplugin -Xmx512M -XX:+UseG1GC -Xloggc:/d
#### 1.1 specify the service of the package to be debugged
Determine the service it belongs to according to the location of the code to be debugged

#### 1.2 enter {linkis_home}/sbin/ext, modify the module configuration file, and open the remote call port
#### 1.2 enter {LINKIS_HOME}/sbin/ext, modify the module configuration file, and open the remote call port
![e-port](https://user-images.githubusercontent.com/29391030/167366305-207da78b-48da-4eee-af40-947bd43050b9.png)

### 2. Restart the service to be debugged
Expand Down