Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workflow tutorial #44

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions 09-tutorial-screenshoot/screenshot.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# 视频截帧在工作流中的实现
本示例中,将详细讲解如何使用七牛深度学习平台的[工作流](/05-tasks/5.6-workflow.md),实现将一个视频文件根据用户要求进行下载并截帧上传至七牛云存储Bucket中.用户可通过调整参数设定截帧频率和上传的bucket.工作流的基本操作请参考[5.6 工作流](/05-tasks/5.6-workflow.md).
gaobonan marked this conversation as resolved.
Show resolved Hide resolved

## 创建任务定义
本示例可以分成3个任务:

1. download-video:将视频下载并重命名(videoname)后放入共享文件夹(dir)
2. ffmpeg:利用ffmpeg工具对视频截帧,并存入准备上传的文件夹(src-dir)
3. upload-frame:将截图上传至指定的bucket中
* 任务定义所包含的输入参数及关系如下图所示:
![](/images/ch-09/workflow-structure.png)

**输入参数**
+ url:视频的url地址
+ videoname:视频名(输出的截图以 视频名-%d.JPG 命名)
+ dir:存放需要处理的视频及上传所需配置文件的目录(需在共享目录下)
houz42 marked this conversation as resolved.
Show resolved Hide resolved
+ fps:截帧的频率(fps=1时为每一秒截一张图,fps=1/20是为每二十秒截一张图)
+ scr-dir:截图输出文件夹(需要在共享目录下)
+ AK:AccessKey(左侧边栏-个人中心-密钥管理中获得)
+ SK:SecretKey(左侧边栏-个人中心-密钥管理中获得)
+ bucket:七牛云存储空间名

**镜像选择及执行入口**
+ download-video

镜像地址:
>reg.qiniu.com/common/public-mxnet:public-test

执行入口:
>`mkdir -p {{inputs.parameters.dir}} && cd {{inputs.parameters.dir}} && wget {{inputs.parameters.url}} -O {{inputs.parameters.videoname}}`

+ ffmpeg
镜像地址:
>reg.qiniu.com/avatest/detect-reader:20181212-31fbdc6

执行入口:
>`mkdir -p {{inputs.parameters.src-dir}} && cd {{inputs.parameters.dir}} && ffmpeg -i {{inputs.parameters.videoname}} -f image2 -vf fps=fps={{inputs.parameters.fps}} {{inputs.parameters.src-dir}}/{{inputs.parameters.videoname}}-%d.jpg`

+ upload-frame

镜像地址:
>reg.qiniu.com/avatest/qshell:20181211-2020

执行入口:
>`qshell account {{inputs.parameters.ak}} {{inputs.parameters.sk}} username && qshell qupload2 --src-dir={{inputs.parameters.src-dir}} --bucket={{inputs.parameters.bucket}}`
houz42 marked this conversation as resolved.
Show resolved Hide resolved

配置文件中的内容为:
houz42 marked this conversation as resolved.
Show resolved Hide resolved
>{
"src_dir" : "xxx"(与输入参数中的src-dir一致),
"bucket" : "xxx"
}

详情可参考[qshell](https://developer.qiniu.com/kodo/tools/1302/qshell)及[qshell qupload](https://github.com/qiniu/qshell/blob/master/docs/qupload.md)
houz42 marked this conversation as resolved.
Show resolved Hide resolved

## 创建工作流模版
![](/images/ch-09/create-template.png)
工作流模版结构如上图所示,任务定义中的输入参数需要绑定全局参数。本示例中,对不同任务所声明的相同输入参数需要绑定相同的全局参数。

## 创建工作流
选择创建的模版

输入参数的取值如下图所示:
![](/images/ch-09/input.png)
为任务选择完资源后,即可创建并运行该工作流。

截图会上传至配置文件中所指定的bucket中,结果如下图所示:
![](/images/ch-09/workflow-result.png)



1 change: 1 addition & 0 deletions 03-concepts/3.7-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,4 @@
- 循环执行 (即将支持);
- 脚本任务 (即将支持);
- 输入/输出文件传递 (即将支持);
查看如何管理\使用工作流.[5.6 工作流管理](/05-tasks/5.6-workflow.md)
houz42 marked this conversation as resolved.
Show resolved Hide resolved
4 changes: 2 additions & 2 deletions 09-faq/faq.md → 10-faq/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@

- 我明明关闭了所有的工作台窗口, 为什么还有 Jupyter Kernel 进程占用资源呢?

![](/images/ch-09/workspace-kernel-proc.png)
![](/images/ch-10/workspace-kernel-proc.png)

关闭浏览器窗口并不会终止 Jupyter Notebook 进程, 对应的 Jupyter Kernel 仍然在后台运行. 如需终止, 需要在 Notebook 页面, 或侧边栏 [Running] => [Kernel Sessions] 中点击 Shutdown. IPython Console 和 Terminal 进程同理.

![](/images/ch-09/workspace-shutdown-kernel.png)
![](/images/ch-10/workspace-shutdown-kernel.png)

## 训练

Expand Down
Binary file added images/ch-09/create-template.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/ch-09/input.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/ch-09/workflow-result.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/ch-09/workflow-structure.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.